Operations

Operations are blocks of YAML code that translate a list of dates into fields.

join

The join is the process of combining data from several sources. Each source is expected to provide different variables for the same dates.

input:
  join:
    - source1:
        key: date
    - source2
    - ...

concat

Concatenation is the process of combining different sets of operations that handle different dates. This is typically used to build a dataset that spans several years, when several sources are involved, each providing a different period.

input:
  dataset:
    concat:
      - dataset: path_to_zarr_1
        start: 2025-03-01
        end: 2025-03-31
        frequency: 6h

      - dataset: path_to_zarr_2
        start: 2025-04-01
        end: 2025-04-30
        frequency: 6h

pipe

The pipe is the process of transforming fields using filters. The first step of a pipe is typically a source, a join, or another pipe. The following steps are filters.

input:
  pipe:
    - source
    - filter1
    - filter2
    - ...