Differences

This shows you the differences between two versions of the page.

--- calculate_python_expression [2020/05/18 16:25]
francisco [Description]
+++ calculate_python_expression [2026/07/18 02:30] (current)
hermann
@@ Line 1: / Line 1: @@
 ====== Calculate Python Expression ======
 ===== Description =====
-This functor runs an Python instance with the defined user expression. For an overview of how Dinamica and Python can be linked together check the documentation about [[python_coupling|Dinamica EGO and Python Coupling]].
+This is a **[[ego_script#container_functors|container functor]]** that runs a Python instance with the user-defined expression. Like the other calculator functors, data is connected through hook functors placed inside its ''<nowiki>{{ … }}</nowiki>'' block.
 ===== Inputs =====
 ^ Name ^ Type ^ Description ^
-| expression | Code  | The expression that will run on Python.  |
+| expression | [[ego_script#constants|Code]] | The expression that will run on Python. ''Code'' values cannot be written as plain text constants in EGO Script — see [[#writing_the_expression_in_ego_script|Writing the expression in EGO Script]] below. |
 ===== Optional Inputs =====
-None.
+^ Name ^ Type ^ Description ^
+| packages //(advanced)// | String | Required packages to be installed by PIP (one per line). Each package can be identified either by its name or by specifying a filename or URL pointing to the corresponding wheel file. Packages that are already installed will be ignored. |
 ===== Outputs =====
 ^ Name ^ Type ^ Description ^
-| result | [[struct_type|Struct]]| A struct containing the output values generated by the expression.  |
+| result | [[struct_type|Struct]] | A struct containing the output values generated by the expression. |
+===== Notes =====
+==== Expression inputs ====
+Data is passed into the expression through **hook** functors placed inside the container's ''<nowiki>{{ … }}</nowiki>'' block:
+  * Tables and lookup tables → [[Number Table]] → available in the expression as ''dinamica.inputs["t1"]'', ''dinamica.inputs["t2"]'', …, ''dinamica.inputs["t100"]''
+  * Scalar values → [[Number Value]] → available as ''dinamica.inputs["v1"]'', ''dinamica.inputs["v2"]'', …, ''dinamica.inputs["v100"]''
+Maps cannot be connected — this functor has no cell context. There is no shorthand notation. See [[calculate_functors|Calculate Functors — Complete Operator Documentation]] for the general hook mechanism and syntax.
+Every table or lookup table arriving through ''dinamica.inputs'' is represented in Python as a **list of lists**: the first inner list contains the column names (the header row) and every subsequent inner list is a row of data. The header name of each key column has an asterisk (''*'') appended to it.
+==== Expression outputs ====
+Values are returned to Dinamica by assigning them into ''dinamica.outputs'', keyed by the desired output name. Every assigned value becomes an entry in the output ''result'' struct:
+<code>
+// Scalar values are assigned directly
+dinamica.outputs["patchCount"] = 42
+dinamica.outputs["totalArea"] = 1530.5
+</code>
+Tables and lookup tables cannot be assigned directly — they must first be converted using the [[#utilities|utilities]] described below. If a table arriving from an input already carries ''*'' markers on its key column names, it can be assigned directly to an output without conversion.
+==== Retrieving outputs ====
+''CalculatePythonExpression'' returns a single [[struct_type|Struct]] value (via its ''result'' output port) containing every entry assigned to ''dinamica.outputs''. To retrieve individual values from that struct, use the following functors from the //Integration// group:
+^ Functor ^ Retrieves ^
+| [[Extract Struct Number]] | A numeric value (''int'' or ''float'' assigned to ''dinamica.outputs'') |
+| [[Extract Struct String]] | A string value |
+| [[Extract Struct Table]] | A table produced by ''dinamica.prepareTable()'' |
+| [[Extract Struct Lookup Table]] | A lookup table produced by ''dinamica.prepareLookupTable()'' |
+| [[Extract Struct Tuple]] | A tuple value |
+Each functor takes two inputs: the ''Struct'' returned by ''CalculatePythonExpression'', and the name of the entry to extract as a string constant. For example, to retrieve a numeric output named ''patchCount'' and a table output named ''filteredPatches'':
+<code>
+result := CalculatePythonExpression (String $"(
+myTable = [["PatchId*", "Area"], [1, 12.4], [2, 8.7]]
+dinamica.outputs['patchCount'] = 42
+dinamica.outputs['filteredPatches'] = dinamica.prepareTable(myTable, 1)
+)") {{ }};
+patchCount      := ExtractStructNumber result "patchCount";
+filteredPatches := ExtractStructTable  result "filteredPatches";
+</code>
+==== Utilities ====
+=== dinamica.package() ===
+Installs (if needed via pip) and imports the requested module.
+^ Parameter ^ Type ^ Default ^ Description ^
+| ''packageName'' | str | — | Identifies the package. Used as the default for both ''installPath'' and ''loadPath'' when those are omitted. |
+| ''installPath'' | str | ''packageName'' | What is passed to ''pip install''. Can be a plain name, a version-pinned requirement, a wheel filename or URL, a ''git+'' URL, or a name followed by extra pip flags. |
+| ''loadPath'' | str | ''packageName'' | The name used to ''import'' the module in Python. |
+=== dinamica.prepareTable() ===
+Converts a list of lists into a table ready to be assigned to an output. The first inner list must be the header row.
+^ Parameter ^ Type ^ Default ^ Description ^
+| ''inputTable'' | list of lists | — | The table data. First inner list is the header row; subsequent inner lists are data rows. |
+| ''numKeys'' | int | — | Number of key columns. Key column names will have ''*'' appended in the output. |
+=== dinamica.prepareLookupTable() ===
+Converts a list of lists into a lookup table ready to be assigned to an output. The first inner list must be the header row.
+^ Parameter ^ Type ^ Default ^ Description ^
+| ''lut'' | list of lists | — | The lookup table data. First inner list is the header row; subsequent inner lists are data rows. |
+=== dinamica.toTable() ===
+Converts several Python data shapes into a valid Dinamica table for output.
+^ Parameter ^ Type ^ Default ^ Description ^
+| ''inputTable'' | list of lists; dict of lists; list of tuples; flat list; ''pandas.DataFrame''; ''numpy.array'' | — | The data to convert. A flat list produces a lookup table with sequential keys. A ''numpy.array'' must have its header as the first row. |
+=== Installing packages ===
+Packages can be installed in two ways:
+  * Calling ''dinamica.package(...)'' from within the expression.
+  * Listing package names on the //packages// input port, one per line.
+These two approaches differ in timing: ''dinamica.package(...)'' runs **during** the expression, so it can be called conditionally; the //packages// port always runs **before** the expression executes. Because ''dinamica.package(...)'' runs together with the script, it may fail if a package is already loaded in an incompatible version.
+''dinamica.package(packageName, installPath=None, loadPath=None)'':
+  * ''packageName'' identifies the package and is the default for both following parameters when omitted.
+  * ''installPath'' is passed to ''pip install''. It can be a version-pinned requirement, a wheel filename or URL, a ''git+'' URL, or a name plus extra pip flags.
+  * ''loadPath'' is the name used to ''import'' the module. It defaults to ''packageName''.
+Simply install and load ''numpy'':
+<code>
+dinamica.package("numpy")
+</code>
+Specify a version:
+<code>
+dinamica.package("numpy", "numpy==1.19.5")
+</code>
+Install a package whose importable name differs from its pip name:
+<code>
+dinamica.package("segment_anything_py", "segment_anything_py", "segment_anything")
+</code>
+Use arbitrary pip parameters, such as installing from a remote wheel with a custom index:
+<code>
+dinamica.package("segment_anything_py", "https://files.pythonhosted.org/packages/43/2f/dabe75d90a7eb54a0a609a0fc5c36d1933256319beaea5d6b2f176e213a2/segment_anything_py-1.0-py3-none-any.whl --index-url https://download.pytorch.org/whl/cu118", "segment_anything")
+</code>
+Chain several installs:
+<code>
+dinamica.package("cython")
+dinamica.package("numpy")
+dinamica.package("pycocotools", "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI")
+</code>
+The //packages// port takes one package identifier per line, in any form accepted by ''pip''. Unlike ''dinamica.package()'', it does not support specifying separate install and import names, and every listed package is installed before the expression runs:
+<code>
+numpy
+requests==2.31.0
+torchvision==0.19.1 --index-url https://download.pytorch.org/whl/cu121
+</code>
+==== Examples ====
+The following examples use a consistent set of inputs:
+  * ''t1'' — a table of land cover patches, with columns ''PatchId*'', ''Area'', and ''CategoryId''
+  * ''t2'' — a lookup table mapping category identifiers to category names
+  * ''v1'' — a scalar minimum area threshold
+Install and import ''numpy'', then inspect all inputs passed in by Dinamica:
+<code>
+dinamica.package("numpy")
+print(dinamica.inputs)
+</code>
+Print the rows of both connected tables to verify their contents:
+<code>
+for row in dinamica.inputs["t1"]:
+    print(row)
+for row in dinamica.inputs["t2"]:
+    print(row)
+</code>
+Count the patches whose area meets the minimum threshold and sum their total area:
+<code>
+patchCount = 0
+totalArea = 0.0
+for row in dinamica.inputs["t1"][1:]:
+    if row[1] >= dinamica.inputs["v1"]:
+        patchCount += 1
+        totalArea += row[1]
+dinamica.outputs["patchCount"] = patchCount
+dinamica.outputs["totalArea"] = totalArea
+</code>
+Return a filtered table containing only the patches above the threshold:
+<code>
+filtered = [dinamica.inputs["t1"][0]]
+for row in dinamica.inputs["t1"][1:]:
+    if row[1] >= dinamica.inputs["v1"]:
+        filtered.append(row)
+dinamica.outputs["filteredPatches"] = dinamica.prepareTable(filtered, 1)
+</code>
+Compute the total area per category and return it as a lookup table:
+<code>
+areaSums = {}
+for row in dinamica.inputs["t1"][1:]:
+    categoryId = row[2]
+    areaSums[categoryId] = areaSums.get(categoryId, 0.0) + row[1]
+lut = [["CategoryId*", "TotalArea"]]
+for catId, total in areaSums.items():
+    lut.append([catId, total])
+dinamica.outputs["areaByCategory"] = dinamica.prepareLookupTable(lut)
+</code>
+==== Writing the expression in EGO Script ====
+When writing a script by hand, the ''expression'' input cannot be filled in directly as a text constant: the [[ego_script#constants|Code]] type is represented in the underlying script using base64 encoding, which is impractical to write or edit directly. Instead, connect a ''String'' carrier functor containing the expression text to the ''expression'' port — its output is accepted wherever a ''Code'' value is expected. Since only this one output is needed, the carrier can be [[ego_script#inline_syntax|inlined]] directly into the call.
+This limitation is specific to hand-written EGO Script. In the Dinamica EGO GUI, the ''expression'' port has a dedicated code editor that edits the ''Code'' value directly — the ''String'' carrier workaround is only necessary when writing or editing the ''.ego'' file as text.
+The following counts the patches in a land cover areas table whose area meets a minimum threshold:
+<code>
+result := CalculatePythonExpression (String $"(
+total = 0
+for row in dinamica.inputs['t1'][1:]:
+    if row[1] >= dinamica.inputs['v1']:
+        total += 1
+dinamica.outputs['patchCount'] = total
+)") {{
+    NumberTable landCoverAreas 1;
+    NumberValue minimumArea    1;
+}};
+patchCount := ExtractStructNumber result "patchCount";
+</code>
+''CalculatePythonExpression'' returns a ''Struct'' containing all values assigned to ''dinamica.outputs''. The ''ExtractStructNumber'' functor then pulls the ''patchCount'' entry out of that struct by name. See [[#retrieving_outputs|Retrieving outputs]] for the full list of extraction functors.
+A more involved example: installing ''numpy'' to compute area statistics for a land cover patches table and flag outlier patches, then retrieving both the scalar statistics and the resulting table:
+<code>
+result := CalculatePythonExpression (String $"(
+dinamica.package('numpy')
+areas = numpy.array([row[1] for row in dinamica.inputs['t1'][1:]])
+meanArea = float(numpy.mean(areas))
+stdArea = float(numpy.std(areas))
+isOutlier = numpy.abs(areas - meanArea) > 2 * stdArea
+dinamica.outputs['meanArea'] = meanArea
+dinamica.outputs['stdArea'] = stdArea
+header = dinamica.inputs['t1'][0]
+outlierRows = [row for row, flagged in zip(dinamica.inputs['t1'][1:], isOutlier) if flagged]
+outlierTable = [header] + outlierRows
+dinamica.outputs['outlierPatches'] = dinamica.prepareTable(outlierTable, 1)
+)") {{
+    NumberTable landCoverPatches 1;
+}};
+meanArea       := ExtractStructNumber result "meanArea";
+stdArea        := ExtractStructNumber result "stdArea";
+outlierPatches := ExtractStructTable  result "outlierPatches";
+</code>
+Here ''landCoverPatches'' is bound to ''t1'' through a single [[Number Table]] hook.
 ===== Group =====
-[[Functor List#Integration | Integration]]
+[[Functor List#Integration|Integration]]
 ===== Internal Name =====
 CalculatePythonExpression
+===== See Also =====
+  * [[ego_script#container_functors|EGO Script — Container Functors]]
+  * [[ego_script#verbose_form|EGO Script — Verbose Form (Hooks)]]
+  * [[calculate_functors|Calculate Functors — Complete Operator Documentation]]