Registry

Problem discovery and loading.

Problems live in packages under emflow.benchmarks (competition problems — the product) and emflow.examples (tutorials). A package participates by providing a problem module with either

• list_problem_variants() -> list[str] + one get_problem_<variant>() factory per variant, or

• a single get_problem() factory.

Factories must return a Problem. A benchmark whose data has not been ingested yet raises ProblemNotIngestedError from its factory — it still shows up in list_problems() (marked in problem_status()), so the full roster is visible.

Packages whose name starts with _ (e.g. examples/_legacy) are skipped.

exception emflow.problems.registry.ProblemNotIngestedError

Bases: NotImplementedError

Raised by a registered benchmark whose dataset is not yet ingested.

emflow.problems.registry.list_problems() → List[str]

All registered problem names (package or package:variant).

emflow.problems.registry.cache_problem_data(name: str, include_private: bool = False, token: str | None = None) → bool

Materialize a benchmark’s HF-hosted data into its local build cache so later loads (e.g. sandboxed evaluations running offline and without credentials) read from disk.

Benchmarks that follow the PUBLIC_REPO/PRIVATE_REPO/LOCAL_BUILD convention are downloaded via huggingface_hub (token from the argument or ambient env). Problems that ship packaged data have nothing to cache — returns False. include_private deliberately defaults to False: holdout labels should stay off local disk in sandboxed setups and be read straight from the gated repo by the credentialed verifier instead.

emflow.problems.registry.load_problem(name: str) → Problem

Load a problem by name; always returns a Problem.

Raises ProblemNotIngestedError for registered-but-not-ingested benchmarks, KeyError for unknown names.