transition-foundation-model / docs /external-validation.md
maxffarrell's picture
Release experimental Transition Foundation Model v0.1 with reproducible source and validation
1baf215 verified
|
Raw History Blame Contribute Delete
2.4 kB

External validation boundary

tfm.data.external provides native collectors for CityLearn 3 list-interface environments, BOPTEST direct-instance REST endpoints, and numerical Minari episodes. They preserve original names, controls, timestamps/frequency and source identifiers in isolated artifacts. They do not automatically map external observations onto the synthetic vocabulary, or combine normalization statistics.

CityLearn collector receives an already configured environment and reads env.action_names / env.action_space. A policy must act only through enabled native controls. Storage fractions are not physical power; thermostat setpoints may be absent. See CityLearn API.

BOPTEST collector receives an existing instance endpoint and records native input bounds, measurement metadata and forecast snapshots before each action. Initialize the desired weather/operating scenario outside the collector. Hosted service routing with a test ID may require a transport adapter; direct instance routes are the tested contract. See BOPTEST.

Minari collector requires an explicit physical-instance grouping function. Different episode IDs do not establish unseen physical systems; recorded policies may lack alternative-action coverage. See Minari dataset source.

Before deriving supervised windows from an external source:

  1. Inspect its enabled controls, units, sensor semantics, observation frequency, and physical instance grouping.
  2. Validate an explicit mapping to the controlled vocabulary. Add a new versioned field identity for a different quantity; do not relabel it to fit an existing schema.
  3. Distinguish forecast disturbances known at the anchor time from realized future outcomes. Never use future measured state as a forecast input.
  4. Validate normalization and missing-data handling on that source alone, with statistics from its training systems.
  5. Establish ordinary and intervention baselines within that source before sharing a backbone with the synthetic experiment.

The local collectors have contract tests with numerical fixtures. Live CityLearn/BOPTEST/Minari experiments have not been executed by the initial smoke run. No external transfer result is claimed.