| # generator_v6 — v5 plus an observation model |
| |
| v5 (`sarima-fullctx-v5b`) scored **0.07090** against the king's **0.10467** on the |
| revealed eval pool (`2026-07-24`, seed 42, heat budget). v6 keeps that engine |
| unchanged and adds the stage it was missing: a model of how a latent process is |
| actually *observed*. |
| |
| ## Why |
| |
| Measuring the revealed pool against the v5 corpus (`local_eval/scripts/fingerprint.py`) |
| showed three structural gaps — not small mismatches, but regimes the corpus |
| essentially never produced: |
|
|
| | property | eval pool | v5 corpus | v6 corpus | |
| |---|---|---|---| |
| | non-negative | 87.7% | 25.8% | 67.5% | |
| | integer-valued | 44.2% | 7.5% | 22.5% | |
| | heavy increments (kurtosis > 20) | 34.9% | 15.2% | 30.8% | |
| | marginal skew (median) | 0.36 | 0.05 | 0.35 | |
|
|
| Real series are overwhelmingly non-negative (six of the pool's seven domains are |
| 100% non-negative; only `energy` is signed) and often integer counts (`sales`, |
| `healthcare`, `transport` are entirely integer). Those properties change the |
| *shape* of the predictive distribution the model has to emit: a floor forces |
| asymmetric quantiles, and counts make the target atomic. A corpus of symmetric |
| signed processes never teaches either, which costs CRPS/WQL on most of the pool. |
|
|
| ## What v6 adds |
|
|
| `_observation_model()` runs after the measurement artifacts, on every family: |
|
|
| 1. **Positive-domain links** (`positive_rate`, default 0.62) — a signed row is |
| mapped through one of four links: `offset` (floor moved below the bulk, |
| dynamics untouched), `softplus` (smooth floor), `exponential` (lognormal |
| magnitudes), `censored` (hard floor with genuine zero runs, rainfall-like). |
| 2. **Integer counts** (`count_rate`, 0.34) — a non-negative row is rescaled to a |
| count level drawn log-uniformly over 0.8–3000 and rounded, so the corpus spans |
| tiny tally series (heavy ties, many zeros) through large tallies. |
| 3. **Sparse heavy-tailed events** (`spike_rate`, 0.22) — Student-t(3) marks at a |
| log-uniform rate, lifting increment kurtosis into the pool's range. |
|
|
| Cost is **1.02x** generation time — important, because both king and v5 already |
| hit the 30-minute wall clock at ~75% of the token budget, so the GPU is the |
| binding constraint and the corpus must not get slower. |
|
|
| ## Changing the prior |
|
|
| Three levels, cheapest first. |
|
|
| **1. Re-weight families — config only, no code.** `config.json` → |
| `family_weights`. Unspecified families keep their default. Weights are |
| normalised, so only ratios matter. |
|
|
| ```json |
| "family_weights": { "sarima": 0.12, "physical_sensors": 0.06 } |
| ``` |
|
|
| **2. Tune a stage — config only.** `positive_rate` / `count_rate` / |
| `spike_rate` above, and the trend/seasonal knobs (`tr_exc_*`, `gr_exc_*`, |
| `sa_clean_*`). |
|
|
| **3. Add a family — four edits, all in `generator.py`, and order matters:** |
|
|
| - append the name to `_FAMILIES` |
| - add its default to `_DEFAULT_WEIGHTS` |
| - write `def _my_family(rng, n, L) -> np.ndarray` returning `(n, L)` float64, |
| fully batched (a per-series Python loop will blow the time budget) |
| - append it to the `builders` tuple in `generate()` **at the same position** as |
| in `_FAMILIES` |
|
|
| If it emits magnitudes or counts, add the name to `_NONNEGATIVE_FAMILIES` so the |
| measurement stage will not flip its sign. (v5 hard-coded those as integer |
| indices, which silently broke if a family was inserted mid-list; v6 resolves them |
| by name.) |
|
|
| ## Validate |
|
|
| ```bash |
| cascade verify ../generator_v6 |
| cd /root/local_eval && ./scripts/compare.sh king v5 v6 |
| ``` |
|
|
