Calculate Python Expression

Description

This is a container functor that runs a Python instance with the user-defined expression. Like the other calculator functors, data is connected through hook functors placed inside its {{ … }} block. See also Calculate R Expression for the equivalent functor using R.

Inputs

Name	Type	Description
expression	Code	The expression that will run on Python. Code values cannot be written as plain text constants in EGO Script — see Writing the expression in EGO Script below.

Optional Inputs

Name	Type	Description
packages (advanced)	String	Required packages to be installed by PIP (one per line). Each package can be identified either by its name or by specifying a filename or URL pointing to the corresponding wheel file. Packages that are already installed will be ignored.

Outputs

Name	Type	Description
result	Struct	A struct containing the output values generated by the expression.

Notes

Expression inputs

Data is passed into the expression through hook functors placed inside the container's {{ … }} block — the same verbose form hook mechanism used by the calculator functors:

Tables and lookup tables → Number Table → available in the expression as dinamica.inputs[“t1”], dinamica.inputs[“t2”], …, dinamica.inputs[“t100”]

Scalar values → Number Value → available as dinamica.inputs[“v1”], dinamica.inputs[“v2”], …, dinamica.inputs[“v100”]

Strings → Number String → available as dinamica.inputs[“s1”], dinamica.inputs[“s2”], …, dinamica.inputs[“s100”]

Maps cannot be connected — this functor has no cell context. There is no shorthand notation. See Calculate Functors — Complete Operator Documentation for the general hook mechanism and syntax.

Every table or lookup table arriving through dinamica.inputs is represented in Python as a list of lists: the first inner list contains the column names (the header row) and every subsequent inner list is a row of data. The header name of each key column has an asterisk (*) appended to it.

Expression outputs

Values are returned to Dinamica by assigning them into dinamica.outputs, keyed by the desired output name. Every assigned value becomes an entry in the output result struct:

// Scalar values are assigned directly
dinamica.outputs["patchCount"] = 42
dinamica.outputs["totalArea"] = 1530.5

Tables and lookup tables cannot be assigned directly — they must first be converted using the utilities described below. If a table arriving from an input already carries * markers on its key column names, it can be assigned directly to an output without conversion.

Retrieving outputs

CalculatePythonExpression returns a single Struct value (via its result output port) containing every entry assigned to dinamica.outputs. To retrieve individual values from that struct, use the following functors from the Integration group:

Functor	Retrieves
Extract Struct Number	A numeric value (int or float assigned to dinamica.outputs)
Extract Struct String	A string value
Extract Struct Table	A table produced by dinamica.prepareTable()
Extract Struct Lookup Table	A lookup table produced by dinamica.prepareLookupTable()
Extract Struct Tuple	A tuple value

Each functor takes two inputs: the Struct returned by CalculatePythonExpression, and the name of the entry to extract as a string constant. For example, to retrieve a numeric output named patchCount and a table output named filteredPatches:

result := CalculatePythonExpression (String $"(
myTable = [["PatchId*", "Area"], [1, 12.4], [2, 8.7]]
dinamica.outputs['patchCount'] = 42
dinamica.outputs['filteredPatches'] = dinamica.prepareTable(myTable, 1)
)") {{ }};
patchCount := ExtractStructNumber result "patchCount";
filteredPatches := ExtractStructTable result "filteredPatches";

Utilities

dinamica.package()

Installs (if needed via pip) and imports the requested module.

Parameter	Type	Default	Description
packageName	str	—	Identifies the package. Used as the default for both installPath and loadPath when those are omitted.
installPath	str	packageName	What is passed to pip install. Can be a plain name, a version-pinned requirement, a wheel filename or URL, a git+ URL, or a name followed by extra pip flags.
loadPath	str	packageName	The name used to import the module in Python.

dinamica.prepareTable()

Converts a list of lists into a table ready to be assigned to an output. The first inner list must be the header row.

Parameter	Type	Default	Description
inputTable	list of lists	—	The table data. First inner list is the header row; subsequent inner lists are data rows.
numKeys	int	—	Number of key columns. Key column names will have * appended in the output.

dinamica.prepareLookupTable()

Converts a list of lists into a lookup table ready to be assigned to an output. The first inner list must be the header row.

Parameter	Type	Default	Description
lut	list of lists	—	The lookup table data. First inner list is the header row; subsequent inner lists are data rows.

dinamica.toTable()

Converts several Python data shapes into a valid Dinamica table for output.

Parameter	Type	Default	Description
inputTable	list of lists; dict of lists; list of tuples; flat list; pandas.DataFrame; numpy.array	—	The data to convert. A flat list produces a lookup table with sequential keys. A numpy.array must have its header as the first row.
numKeys	int	—	Number of key columns. Key column names will have * appended in the output. Ignored for a flat list, which always produces a lookup table with sequential keys.

Installing packages

Packages can be installed in two ways:

Calling dinamica.package(...) from within the expression.

Listing package names on the packages input port, one per line.

These two approaches differ in timing: dinamica.package(...) runs during the expression, so it can be called conditionally; the packages port always runs before the expression executes. Because dinamica.package(...) runs together with the script, it may fail if a package is already loaded in an incompatible version.

dinamica.package(packageName, installPath=None, loadPath=None):

packageName identifies the package and is the default for both following parameters when omitted.

installPath is passed to pip install. It can be a version-pinned requirement, a wheel filename or URL, a git+ URL, or a name plus extra pip flags.

loadPath is the name used to import the module. It defaults to packageName.

Simply install and load numpy:

dinamica.package("numpy")

Specify a version:

dinamica.package("numpy", "numpy==1.19.5")

Install a package whose importable name differs from its pip name:

dinamica.package("segment_anything_py", "segment_anything_py", "segment_anything")

Use arbitrary pip parameters, such as installing from a remote wheel with a custom index:

dinamica.package("segment_anything_py", "https://files.pythonhosted.org/packages/43/2f/dabe75d90a7eb54a0a609a0fc5c36d1933256319beaea5d6b2f176e213a2/segment_anything_py-1.0-py3-none-any.whl --index-url https://download.pytorch.org/whl/cu118", "segment_anything")

Chain several installs:

dinamica.package("cython")
dinamica.package("numpy")
dinamica.package("pycocotools", "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI")

The packages port takes one package identifier per line, in any form accepted by pip. It only installs — unlike dinamica.package(), it does not also import the module, and it does not support specifying a separate install name and import name. Every listed package is installed before the expression runs, but the expression must still import each one itself, using its actual importable name (which may differ from the name given to pip):

numpy
requests==2.31.0
torchvision==0.19.1 --index-url https://download.pytorch.org/whl/cu121

import numpy
import requests
import torchvision

Examples

The following examples use a consistent set of inputs:

t1 — a table of land cover patches, with columns PatchId*, Area, and CategoryId

t2 — a lookup table mapping category identifiers to category names

v1 — a scalar minimum area threshold

Install and import numpy, then inspect all inputs passed in by Dinamica:

dinamica.package("numpy")
print(dinamica.inputs)

Note: The print() call is visible in Dinamica EGO's Message Log, shown as a Result-level message. Log levels are ordered Unconditional, Error, Warning, Result, Info, Info2, Debug, Debug2 — messages printed from Python are only shown when the Message Log level is set to Result or a more verbose level; at Unconditional, Error, or Warning they are suppressed.

Print the rows of both connected tables to verify their contents:

for row in dinamica.inputs["t1"]:
print(row)
for row in dinamica.inputs["t2"]:
print(row)

Count the patches whose area meets the minimum threshold and sum their total area:

patchCount = 0
totalArea = 0.0
for row in dinamica.inputs["t1"][1:]:
if row[1] >= dinamica.inputs["v1"]:
patchCount += 1
totalArea += row[1]
dinamica.outputs["patchCount"] = patchCount
dinamica.outputs["totalArea"] = totalArea

Return a filtered table containing only the patches above the threshold:

filtered = [dinamica.inputs["t1"][0]]
for row in dinamica.inputs["t1"][1:]:
if row[1] >= dinamica.inputs["v1"]:
filtered.append(row)
dinamica.outputs["filteredPatches"] = dinamica.prepareTable(filtered, 1)

Compute the total area per category and return it as a lookup table:

areaSums = {}
for row in dinamica.inputs["t1"][1:]:
categoryId = row[2]
areaSums[categoryId] = areaSums.get(categoryId, 0.0) + row[1]
lut = [["CategoryId*", "TotalArea"]]
for catId, total in areaSums.items():
lut.append([catId, total])
dinamica.outputs["areaByCategory"] = dinamica.prepareLookupTable(lut)

Writing the expression in EGO Script

When writing a script by hand, the expression input cannot be filled in directly as a text constant: the Code type is represented in the underlying script using base64 encoding, which is impractical to write or edit directly. Instead, connect a String carrier functor containing the expression text to the expression port — its output is accepted wherever a Code value is expected. Since only this one output is needed, the carrier can be inlined directly into the call.

This limitation is specific to hand-written EGO Script. In the Dinamica EGO GUI, the expression port has a dedicated code editor that edits the Code value directly — the String carrier workaround is only necessary when writing or editing the .ego file as text.

The following counts the patches in a land cover areas table whose area meets a minimum threshold:

result := CalculatePythonExpression (String $"(
total = 0
for row in dinamica.inputs['t1'][1:]:
if row[1] >= dinamica.inputs['v1']:
total += 1
dinamica.outputs['patchCount'] = total
)") {{
NumberTable landCoverAreas 1;
NumberValue minimumArea 1;
}};
patchCount := ExtractStructNumber result "patchCount";

CalculatePythonExpression returns a Struct containing all values assigned to dinamica.outputs. The ExtractStructNumber functor then pulls the patchCount entry out of that struct by name. See Retrieving outputs for the full list of extraction functors.

A more involved example: installing numpy to compute area statistics for a land cover patches table and flag outlier patches, then retrieving both the scalar statistics and the resulting table:

result := CalculatePythonExpression (String $"(
dinamica.package('numpy')
areas = numpy.array([row[1] for row in dinamica.inputs['t1'][1:]])
meanArea = float(numpy.mean(areas))
stdArea = float(numpy.std(areas))
isOutlier = numpy.abs(areas - meanArea) > 2 * stdArea
dinamica.outputs['meanArea'] = meanArea
dinamica.outputs['stdArea'] = stdArea
header = dinamica.inputs['t1'][0]
outlierRows = [row for row, flagged in zip(dinamica.inputs['t1'][1:], isOutlier) if flagged]
outlierTable = [header] + outlierRows
dinamica.outputs['outlierPatches'] = dinamica.prepareTable(outlierTable, 1)
)") {{
NumberTable landCoverPatches 1;
}};
meanArea := ExtractStructNumber result "meanArea";
stdArea := ExtractStructNumber result "stdArea";
outlierPatches := ExtractStructTable result "outlierPatches";

Here landCoverPatches is bound to t1 through a single Number Table hook.

Group

Integration

Internal Name

CalculatePythonExpression