Field

Fields: time series with explicit availability semantics.

A Field is one logical series of a dataset plus the rule for when each value becomes knowable. This is the primitive that makes look-ahead leakage impossible by construction: the DataFeed only ever serves values whose availability time is at or before the simulation clock.

Three kinds of field:

actual

Observations stamped by measurement time. Two availability rules:

• constant lag (default): a value stamped t becomes knowable at t + availability_lag — the right model for settled-only archives;

• bitemporal (knowledge_col=...): each row carries its own knowledge timestamp, and the same measurement time may appear multiple times with successive revisions (preliminary → settled). The feed serves, at any clock time, the latest revision already knowable — so a backtest sees the preliminary value exactly as a live participant did.

forecast

Predictions stamped by (issue_time, valid_time) — e.g. NWP runs. The run issued at issue_time becomes knowable at issue_time + availability_lag (dissemination delay). At any clock time the feed serves, per valid_time, the latest run already issued. Using tomorrow’s weather actuals as if they were forecasts is the classic energy-backtest leak; this kind exists to make that mistake unrepresentable.

static

Time-invariant metadata (capacities, coordinates). Always available.

class emflow.data.field.Field(name: str, frame: DataFrame, kind: str = 'actual', availability_lag: str | Timedelta = '0h', knowledge_col: str | None = None, description: str | None = None)

Bases: object

One logical time series + its availability rule.

Parameters

name:

Field name, unique within a Dataset.

frame:

The data. actual: a DataFrame with a tz-aware DatetimeIndex (measurement time). forecast: a DataFrame with a two-level MultiIndex (issue_time, valid_time) — use Field.forecast() to build one from columns. static: any DataFrame.

kind:

"actual" (default), "forecast" or "static".

availability_lag:

Delay between a value’s timestamp (actuals) or a run’s issue time (forecasts) and the moment it becomes knowable. Anything pd.Timedelta accepts; default "0h".

knowledge_col:

Actuals only: name of a column holding each row’s knowledge timestamp (bitemporal storage — rows may repeat an index timestamp with revisions). Mutually exclusive with a non-zero availability_lag.

description:

Optional human-readable description (surfaced in manifests).

name: str

frame: DataFrame

kind: str = 'actual'

availability_lag: str | Timedelta = '0h'

knowledge_col: str | None = None

description: str | None = None

classmethod forecast(name, frame, issue_col, valid_col, availability_lag='0h', description=None) → Field

Build a forecast field from a flat frame with issue/valid columns.

classmethod static(name, frame, description=None) → Field

property settlement_lag: Timedelta

How long after a timestamp its value is finally knowable.

Environments use this to decide when an origin can settle. For constant-lag fields it’s availability_lag; for bitemporal fields it’s the largest observed revision delay, so settlement waits for the settled (last) revision rather than scoring against a preliminary one.

property start: Timestamp | None

property end: Timestamp | None