Guides

Algorithm Manifests

Complete guide to defining algorithm manifests

Algorithm Manifests

An algorithm manifest is a JSON document that describes your algorithm's requirements, capabilities, and behavior. The platform validates this manifest against the manifest schema when you register an algorithm.

Complete Example

Here's a complete manifest for a device visits algorithm:

{
  "manifest_version": "0.1.0",
  "metadata": {
    "name": "Device Visits",
    "description": "Produce a list of AOI visits per device, based on geolocation device pings.",
    "tags": ["device_visits", "foot_traffic"]
  },
  "interface": {
    "interface_type": "FILESYSTEM_TASK_WORKER"
  },
  "inputs": [
    {
      "data_type_name": "pings",
      "min_count": 1,
      "max_count": 1
    }
  ],
  "outputs": [
    {
      "data_type_name": "device_visits",
      "observation_value_columns": ["visit_count"]
    }
  ],
  "container_parameters": {
    "image": "orbitalinsight/device_visits:13131a4e",
    "command": ["python", "/app/device_visits.py"],
    "resource_request": {
      "gpu": 0,
      "memory_gb": 5,
      "cpu_millicore": 200
    }
  },
  "parameters": [
    {
      "name": "look_back_time",
      "type": "integer",
      "units": "seconds",
      "description": "Number of seconds to look back before the first recorded ping for each device.",
      "min": 0,
      "max": 2592000,
      "default": 3600
    },
    {
      "name": "look_forward_time",
      "type": "integer",
      "units": "seconds",
      "description": "Number of seconds to look forward after the last recorded ping for each device.",
      "min": 0,
      "max": 2592000,
      "default": 3600
    },
    {
      "name": "override_visit_time",
      "type": "boolean",
      "description": "If enabled, set the look back and look forward timestamp to be equal to the start and end of the given observation.",
      "default": false
    }
  ]
}

Manifest Structure

The following six fields are required: manifest_version, metadata, interface, inputs, outputs, and container_parameters. The rest (parameters, selection, parallelization, and the metadata blocks below) are optional.

manifest_version (required)

Specifies which version of the manifest schema to use, in semantic-version form.

{
  "manifest_version": "0.1.0"
}

Current version: "0.1.0"

The platform validates your manifest against the schema for this version.

metadata (required)

Provides high-level information about your algorithm.

{
  "metadata": {
    "name": "Railcar Detection",
    "description": "Detect and count railcars in optical imagery.",
    "tags": ["railcar", "rail", "object_detection"],
    "release_date": "2024-10-30",
    "indicator": "Railcars",
    "training_location_coverage": "Tested on ~100,000 marked railcars across rail yards worldwide.",
    "best_input_location": "Works best at rail yards and train stations.",
    "known_issues": "Reduced accuracy on heavily shadowed or hazy imagery."
  }
}

Fields:

  • description (string, required) - Clear description of the algorithm's purpose
  • name (string, optional) - Human-readable algorithm name
  • tags (array of strings, optional) - Keywords for categorization and search
  • release_date (string, optional) - Release date, YYYY-MM-DD
  • indicator (string, optional) - What is observed that indicates activity at the AOIs
  • training_location_coverage (string, optional) - Coverage and applicability of the algorithm
  • best_input_location (string, optional) - Where/when the algorithm works best
  • known_issues (string, optional) - Known limitations

Note: there is no version field in metadata. Algorithm versions are identified by the SHA-256 hash of the manifest — see Manifest Versioning.

interface (required)

Specifies how the algorithm container communicates with the platform.

{
  "interface": {
    "interface_type": "FILESYSTEM_TASK_WORKER"
  }
}

Fields:

  • interface_type (string, required) - one of:
    • FILESYSTEM_TASK_WORKER - algorithm reads algo_input.json and writes algo_output.json on the filesystem (the standard interface for most algorithms)
    • SYNCHRONOUS_TASK_HTTP_WORKER - algorithm runs an embedded HTTP server that the platform calls
    • NO_OP_WORKER - no algorithm container runs (used for pass-through / analysis-only steps)
  • adapter (string, optional) - sidecar input adapter; currently only AIRBUS_ICD_INPUT_ADAPTER

For FILESYSTEM_TASK_WORKER, the platform provides:

  • Input via an algo_input.json file
  • Output via an algo_output.json file written to the supplied output_path
  • The input file path in the ALGORITHM_INPUT_PATH environment variable

inputs (required)

Declares the Data Types your algorithm consumes. This is an array — one entry per input Data Type.

{
  "inputs": [
    {
      "data_type_name": "pings",
      "data_source_ids": ["safegraph_pings", "xmode_pings"],
      "min_count": 1,
      "max_count": 1,
      "parameters": {
        "sampling": {
          "min_count": 1,
          "max_count": 10,
          "min_interval": "P1DT1H",
          "max_interval": "P7DT1H"
        }
      }
    }
  ]
}

Fields per input:

  • data_type_name (string, required) - Name of the Data Type (must exist in the platform; retrieve with DataTypesGetRequest)
  • data_source_ids (array of strings, optional) - Specific data sources to use for this Data Type
  • min_count (integer, optional, default 1) - Minimum number of data sources required
  • max_count (integer, optional, default 1) - Maximum number of data sources allowed
  • parameters (object, optional) - Data-Type query parameters conforming to the Data Type's query schema. The sampling block controls how many data points per date range are required:
    • min_count (default 1) - minimum data points; 0 means "don't error if none available"
    • max_count (default 1) - maximum data points
    • min_interval / max_interval (ISO 8601 durations, default PT0S) - spacing between data points

Notes:

  • Most algorithms have a single input Data Type.
  • Use multiple inputs if your algorithm combines different data types.
  • Set min_count: 0 for optional inputs.

outputs (required)

Declares the Data Types your algorithm produces. This is an array — one entry per output Data Type (each entry's own observation_value_columns lists that type's summary metrics).

{
  "outputs": [
    {
      "data_type_name": "device_visits",
      "observation_value_columns": ["visit_count", "total_dwell_time"]
    }
  ]
}

Fields per output:

  • data_type_name (string, required) - The output Data Type
  • observation_value_columns (array of strings, optional) - The full list of summary/aggregate columns the algorithm writes into each observation's observation_values
  • classes (array of strings, optional) - Sub-classes the algorithm classifies (e.g. ["fighter", "bomber"])
  • output_geometry (string, optional) - Geometry type the algorithm outputs (e.g. "Point")
  • sample_result (string, optional) - Path to a sample image of the algorithm's output
  • skip_export (boolean, optional) - If true, results are not exported for download

Understanding observation_value_columns:

These are the aggregated values your algorithm includes in each observation, separate from the detailed measurement data. For example, if you declare ["visit_count", "total_dwell_time"], each observation you write should populate those keys:

{
  "observation_start_ts": 1614607200,
  "observation_values": {
    "visit_count": 42,
    "total_dwell_time": 3600
  },
  "measurements": [
    {"measurement_path": "visits_abc123.parquet"}
  ]
}

See Algorithm Output Schema for the full output structure.

container_parameters (required)

Specifies the Docker container that runs your algorithm.

{
  "container_parameters": {
    "image": "myorg/my-algorithm:v1.0.0",
    "command": ["python", "-u", "/app/main.py"],
    "resource_request": {
      "gpu": 0,
      "memory_gb": 8,
      "cpu_millicore": 1000,
      "max_input_gb": 1
    }
  }
}

Fields:

  • image (string, required) - Full container image reference (registry/name:tag)
  • command (array of strings, required) - Container command in exec form (e.g. ["python", "main.py"], not "python main.py"). Required for FILESYSTEM_TASK_WORKER.
  • resource_request (object, optional) - Resource requirements:
    • gpu (number) - Number of GPUs (0 for CPU-only)
    • memory_gb (number) - Memory in gigabytes
    • cpu_millicore (number) - CPU in millicores (1000 = 1 core)
    • max_input_gb (number) - Maximum size in GB for the algorithm input

Best practices:

  • Use specific image tags, not latest; include the image digest for reproducibility.
  • Request resources conservatively but adequately; monitor actual usage during test runs.
  • Use the -u flag with Python for unbuffered output.

parameters (optional)

Defines configurable parameters users can set when creating an Algorithm Config.

{
  "parameters": [
    {
      "name": "confidence_threshold",
      "type": "double",
      "description": "Minimum confidence score for detections",
      "units": "probability",
      "min": 0.0,
      "max": 1.0,
      "default": 0.7
    },
    {
      "name": "enable_filtering",
      "type": "boolean",
      "description": "Apply post-processing filters",
      "default": true
    },
    {
      "name": "model_variant",
      "type": "string",
      "description": "Which model variant to use",
      "allowed_values": ["fast", "standard", "accurate"],
      "default": "standard"
    }
  ]
}

Fields per parameter:

  • name (string, required) - Parameter identifier
  • type (string, required) - One of integer, double, number, string, boolean, array
  • description (string, required) - Clear explanation of what the parameter does
  • units (string, optional) - Units for the parameter (e.g. "seconds", "meters")
  • min (number, optional) - Minimum allowed value (numeric types)
  • max (number, optional) - Maximum allowed value (numeric types)
  • allowed_values (array, optional) - Restricted set of allowed values
  • default (optional) - Default value used when a config doesn't override the parameter

Best practices:

  • Provide sensible defaults and use min/max to prevent invalid values.
  • Include units for clarity and write descriptions that explain each parameter's impact.

selection (optional)

A declarative scene-selection block consumed by the platform's scene-selection step. Use it for algorithms that need scenes grouped or paired in a particular way (e.g. stereo pairs, before/after change detection). When present, selection.min_count is authoritative over inputs[].min_count.

{
  "selection": {
    "grouping_strategy": "pair",
    "temporal_strategy": "window",
    "spatial_strategy": "independent",
    "min_count": 2,
    "lookback_windows": 0,
    "spacing": {
      "min_separation": "P10D",
      "max_separation": "P40D"
    },
    "data_source_filters": ["cloud_cover <= 20"]
  }
}

Fields:

  • grouping_strategy (required) - how scenes are grouped: single, independent, pair, cross
  • temporal_strategy (required) - how time is sliced: all, window, event
  • spatial_strategy (required) - how AOIs combine with grouping: independent, cross_aoi
  • min_count (integer, optional, default 1) - minimum scenes per group (authoritative when this block is present)
  • spacing (object, optional) - temporal spacing between scenes within a group (min_separation / max_separation, ISO 8601 durations); only valid for pair or cross
  • data_source_filters (array, optional) - CQL2 filter expressions that replace the data source's default constraints for this algorithm (user cql2_filters AND on top)
  • grouping_filters (array, optional) - CQL2 filter expressions for grouping, using reference.* / target.* prefixes
  • before_search / after_search (ISO 8601 durations) - required when temporal_strategy is event
  • lookback_windows (integer) - required when temporal_strategy is window; 0 means current window only (e.g. stereo pairs)

parallelization (optional)

Lists the Dimensions the algorithm can be parallelized by, and how each may be subdivided.

{
  "parallelization": [
    {
      "dimension": "AOI",
      "subdividible": true,
      "units": "geohash",
      "size": 2
    },
    {
      "dimension": "TIME_RANGE_GROUP",
      "subdividible": false
    }
  ]
}

Fields per entry:

  • dimension (string, required) - the Dimension name (e.g. AOI, TIME_RANGE_GROUP)
  • subdividible (boolean, optional) - whether the Dimension can be subdivided
  • units (string, optional) - unit of the subdivisible component (e.g. geohash)
  • size (number, optional) - quantity of units of the subdivisible component

Other optional blocks

The schema also supports several optional metadata/behavior blocks:

  • restrictions - spatial / temporal / size restrictions on where and how the algorithm may run (spatial_restriction, temporal_restriction, size_restriction).
  • batching - how input data is grouped (group: None, Time, or AOI) and tiling requirements (tile_width, tile_height, overlap, tile_method).
  • performance_metrics - reported accuracy (overall_f1_score, overall_precision, overall_recall, pixel_accuracy, and per-subclass metrics).
  • evaluation_details - the training/evaluation datasets used (data_packet_collection_id, data_label_collection_id, etc.).

Validation

When you register an algorithm, the platform validates:

  1. Schema compliance - Manifest matches the schema for manifest_version
  2. Data Type existence - All referenced Data Types exist
  3. Data Source compatibility - Data sources support the specified Data Types (validated against DEM)
  4. Parameter validity - Parameters have valid types and constraints
  5. Resource constraints - Resources are within platform limits

Common validation errors:

Invalid manifest: 'outputs' must be an array
  → Fix: outputs is a list of {data_type_name, ...} objects, not an object

Data Type 'custom_type' not found
  → Fix: Create the Data Type first or use an existing one

Parameter 'threshold' missing required field 'description'
  → Fix: name, type, and description are required on every parameter

Manifest Versioning

Algorithm versions are identified by the SHA-256 hash of the manifest — there is no version number in the manifest itself.

  • Registering a manifest computes its hash. If that hash already exists for the algorithm, the existing Algorithm Version is returned unchanged (no duplicate is created).
  • Any change to the manifest produces a new hash and therefore a new Algorithm Version.
  • Algorithm Versions are immutable once registered, and are never deleted — only deactivated or deprecated.
  • The user-facing "version" label is the creation timestamp (ISO 8601) of the Algorithm Version; the latest version is the one with the most recent timestamp.

So to publish an update — a bug fix, a new model, changed parameters, or changed inputs/outputs — you simply register the updated manifest, which becomes a new Algorithm Version. For changed inputs/outputs in particular, consider whether a new Algorithm is more appropriate than a new version of the existing one.

Next Steps

Updated about 2 months ago

Did this page help you?