Metrics

Metrics: pure scoring functions with one uniform signature.

Every metric implements calculate(y_true, y_pred) -> float and elementwise(y_true, y_pred). The canonical prediction format everywhere in emflow is a DataFrame indexed by target time whose columns are either

• ["point"] — a point forecast, or

• quantile levels as floats (0.1 ... 0.9) — a probabilistic forecast.

y_true is a Series (or single-column DataFrame) on the same index. NaNs are ignored pairwise. A metric is not what a problem ranks on — that’s the Objective, which wraps a metric with a direction.

emflow.problems.metrics.quantile_columns(y_pred: DataFrame) → List[float]

class emflow.problems.metrics.Metric

Bases: ABC

Base class for metrics. Stateless; safe to share across problems.

aggregate: str = 'mean'

How per-origin scores combine into an overall score: “mean” (pooled by scored timestamps — error metrics) or “sum” (revenue-type metrics).

property name: str

abstract elementwise(y_true, y_pred)

Per-timestamp scores (NaN where either side is missing).

calculate(y_true, y_pred) → float

class emflow.problems.metrics.MeanAbsoluteError

Bases: Metric

elementwise(y_true, y_pred)

Per-timestamp scores (NaN where either side is missing).

class emflow.problems.metrics.MeanSquaredError(squared: bool = True)

Bases: Metric

property name

elementwise(y_true, y_pred)

Per-timestamp scores (NaN where either side is missing).

calculate(y_true, y_pred) → float

class emflow.problems.metrics.RootMeanSquaredError

Bases: MeanSquaredError

class emflow.problems.metrics.MeanAbsolutePercentageError

Bases: Metric

elementwise(y_true, y_pred)

Per-timestamp scores (NaN where either side is missing).

class emflow.problems.metrics.PinballLoss(quantiles: Sequence[float] | None = None)

Bases: Metric

Average pinball (quantile) loss over the prediction’s quantile columns.

y_pred must carry quantile levels as float column names. If the metric was constructed with explicit quantiles, the prediction must provide exactly those columns (competition contract); otherwise whatever quantile columns are present are scored.

elementwise(y_true, y_pred)

Per-timestamp pinball loss, averaged across quantiles.

class emflow.problems.metrics.TradingRevenue(imbalance_penalty: float = 0.07)

Bases: Metric

Day-ahead trading revenue under HEFTCom24 settlement: per period,

bid × DA_price + (actual − bid) × SS_price − λ·(actual − bid)²

with λ = 0.07 — the exact formula reproduces the official archive’s per-period revenues to 1e-10 (the quadratic term penalizes large imbalances).

Needs market prices, which live on the environment’s clock — so this metric is settled by TradingEnv, not computed from (y_true, y_pred) alone. It exists as a Metric for naming, direction and aggregation (“sum”: competitions rank total revenue).

aggregate: str = 'sum'

How per-origin scores combine into an overall score: “mean” (pooled by scored timestamps — error metrics) or “sum” (revenue-type metrics).

settled_by_env = True

revenue(actuals, bids, da_price, ss_price)

Elementwise revenue given aligned series (the env calls this).

elementwise(y_true, y_pred)

Per-timestamp scores (NaN where either side is missing).

class emflow.problems.metrics.PeakTimingError

Bases: Metric

BigDEAL-2022-style peak-timing score: per day, the absolute distance (in hours) between the true and predicted peak hour of y.

Both series are grouped by calendar day on their index; days missing from either side are skipped.

elementwise(y_true, y_pred)

Per-timestamp scores (NaN where either side is missing).