docs/ARCHITECTURE.md

# SyFox Architecture — the SI substrate core

This document specifies exactly what happens between "state in" and "decision
out", and which mechanisms are ported from the Synthetic-Intelligence (SI)
research substrate. The rule that governs this codebase:

> **The core is SI physics only.** No transformer, no neural network, no
> pattern-matching classifier. Everything below is field dynamics over an
> explicit concept graph. Tools (JSON, CLI, HTTP, calibration math) live
> outside the core and never rewrite it.

## 1. Concepts and acoustic mass

Every lowercase folded token interned by the engine becomes a **concept node**
(`core/si_substrate.hpp`, `Substrate::intern`). Nodes carry:

* **mass** — a raw occurrence count: `intern()` adds exactly `1.0` each time
  the concept appears in any lesson. It is **linear, not log-compressed**.
  *Lineage:* SI "acoustic mass" — heavier units respond less to the same
  stimulus. The sub-linear behavior lives at the **use sites**, not in the
  counter: injection deposits `energy / sqrt(mass)` and Hebbian bind strength
  scales by `1/sqrt(mass)`, so frequent concepts ("the", "please") cannot
  flood the field.
* **energy** — transient; injected and dissipated per decision.

(An earlier revision also carried a `salience` field. Audit found it was
write-only — incremented at injection, read nowhere; gating selects sources
by energy — so it was removed. Dead state in the core is doc debt.)

Token folding (`fold`) is deterministic string normalization (plural / `-ed` /
`-ing` / trailing `e`), applied identically to states, options, and
instructions. It is I/O hygiene, not linguistics and not pattern matching.

## 2. Lanes (Hebbian bindings)

Lanes are weighted edges between concepts, stored sparsely per node with a
per-node lane cap (weakest lane evicted, bounded memory).

* **Bind** (`hebbian_lesson`): for a labelled lesson, every active state
  concept binds to every outcome concept with
  `η · (1/√mass_a + 1/√mass_b) · scale`. Adjacent state tokens also bind
  (the co-occurrence fabric that lets energy diffuse inside a phrase).
  *Lineage:* SI Hebbian lane bindings in `physics.hpp`.
* **Weaken** (`weaken`): counter-evidence subtracts lane weight; a lane whose
  weight reaches zero dissolves. Used by Noul lessons labelled false.
* **Decay** (`lane_decay`): every lesson multiplies all lanes by 0.995 —
  unused routes fade.

Learning is substrate rewiring. There are no fitted weights anywhere in the
core; the calibration tool (below) is external and read-only.

## 3. Injection

A state is tokenized; every token that exists in the vocabulary deposits
`inject_energy / sqrt(mass)` on its node. Unknown tokens stay dark — this is
the first half of the honest-silence contract (a state of nothing but unknown
words settles zero energy and defers).

## 4. Settle (the System One pass)

`settle()` runs up to `k_settle` (default 8) dissipative diffusion passes:

1. Sources are gated by **source cap** (`source_cap`, 24 by default): the
   top-K nodes propagate each pass — ranked **by energy** by default, or by
   **salience** when `salience_gating` is on. Gated nodes keep their energy
   (they remain readout-visible); the cap gates flow, it does not
   annihilate. (An earlier version pruned energy outright and destroyed 97%
   of the field — the gating form is the correct physics and the tests pin
   it.)

   **SI salience lineage, restored after audit.** The SI substrate runs a
   cavity salience integrator, `s = tanh(s·decay + gain·|velocity|)`
   (`physics.hpp`, `salience_gain = 0.05`, `salience_decay = 0.995`), and
   its TSDA layer samples the live working cap from
   `[working_cap−4, working_cap]` = [5,9] (Miller 7 ± 2). SyFox ports both:
   the integrator runs **every pass** on the motion proxy `|ΔE|` (a node at
   rest is an exact fixed point, mirroring SI's sparsity guard), and
   `miller_window = true` samples the live cap from
   `[source_cap−4, source_cap]` each decision. One deliberate adaptation:
   SI draws the sample from a seeded mt19937 stream; SyFox derives it from
   the decision's state hash, so the same state always settles to the same
   field bit-for-bit (the persistence contract outranks stochasticity).
   `inject()` spikes salience to 1.0 on touch (TSDA spike semantics).

   *Benchmark (10 in-domain decisions, 3 domains).* default energy gating
   10/10 with calibrated-shaped confidences; salience ranking at the wide
   cap 4/10 (motion history is a poor selector when the cap rarely trims);
   salience + literal [5,9] window 10/10 argmax but probabilities saturate
   at 1.0 (calibration collapse — needs the v0.2 eval harness before
   adoption); Miller window at wide cap 9/10. Verdict: default stays
   energy-gated at a fixed 24; the SI-faithful modes ship behind config
   flags until a held-out eval says otherwise.
2. Each source retains `(1 − diffusion)` of its decayed energy and flows
   `diffusion` out along its lanes, split proportionally to lane weight:
   `next[b] += e·decay·diffusion·(w_ab / Σw_a)`.
3. Total energy is non-increasing (dissipative); the pass loop stops early
   when the field relaxes (`total_before − total_after < eps`).

One settle per decision — all questions read the same settled field. Adding
questions never re-settles the substrate: flat marginal latency by structure,
not by scheduling.

**Derivation, not retrieval.** Because diffusion is multi-hop, a state that
was never in any lesson still settles to a computed answer: energy crosses
the co-occurrence fabric into outcome lanes laid down by *other* lessons.
Probe (two lessons, `alpha beta`→x and `beta gamma`→y): the untaught state
`alpha` alone settles with y at 0.494 — lesson 2 never mentioned alpha; the
two-hop path alpha→beta→y carried the evidence. The boundary is the taught
vocabulary and lane fabric: concepts never interned have no node, unknown
words settle to zero energy and defer (§6). This is the substrate-level
kernel of the SI stack's explicit derivation layers (meta-learner
macro-rules, dreamer emergent resonances, analogical mapping) — not yet
ported; see ROADMAP v0.3.

## 5. Readout (resonance sweep, two lanes)

Probes read the settled field at their own concepts: direct node energy plus a
hop term through lanes. Two readout lanes exist because two question shapes
need two different questions of the field:

* **Specific** (`choice`, `score`): mean lane support —
  `hop · Σ w·e_n / Σ w`. Rewards precise coupling; a node bound diffusely to
  half the vocabulary cannot out-read a node bound exactly to this state.
  Options are then normalized by softmax at the fitted temperature.
* **Support** (`noul`): active lane mass — `hop · Σ w·e_n` (unnormalized).
  A statement may be corroborated broadly; what matters is how much
  lane-connected energy the field offers it. The ratio to total field energy
  is the statement's **support**.

Both are field measurements. No weights are fitted at readout; no classes
exist in the core.

## 6. Honest silence

If settled field energy is below `silence_floor` (0.05):

* empty state → `deferred: true, reason: "empty_state"`
* all-unknown vocabulary → `reason: "unknown_vocabulary"`
* settled but dark → `reason: "honest_silence"`

The engine refuses to guess from nothing. *Lineage:* the SI interface's
honest-silence dispatch — answers only when the readout actually collapsed.

## 7. Calibration (a tool, not core)

`fit_calibration` collects readout energies on labelled rows and fits, purely
outside the core:

* **temperature** per choice/score — golden-section search on NLL of
  `softmax(E_correct/T, E_wrong/T)`.
* **Platt scaling** for noul — logistic regression `sigmoid(a·support + b)`
  on supports, gradient descent to convergence (separable data needs a steep
  slope; the fit runs 4000 iterations with decaying lr).

Calibration changes how energies are *reported* as probabilities, never how
they are produced. Confidence for choice/score is the normalized-entropy
margin of the distribution; for noul it is `|p − 0.5|·2`.

## 8. Persistence

`save_model` writes `substrate.bin` (vocabulary + masses + lanes, binary),
`calibration.json`, and `meta.json`. Loading is exact — the same state settles
to the same readout bit-for-bit (pinned by tests).

## 9. Provenance

| Mechanism | Origin |
|---|---|
| Energy injection + field settle | SI `physics.hpp` substrate dynamics |
| Source capping (top-K, energy or salience) | SI working-set cap + TSDA live_cap ([cap−4, cap], Miller 7 ± 2) |
| Salience integrator tanh(s·decay + gain·motion) | SI `physics.hpp` cavity salience (verified live there) |
| Hebbian lane bindings + decay | SI lane learning |
| Honest silence | SI dispatch contract |
| Typed Choice/Score/Noul API | interface shape inspired by TypeSafe Jev |
| Temperature/Platt calibration | classical statistics (tool layer) |
| Derivation layer (§10) | SyFox-native design after studying SI meta_learner / dreamer / analogy as mechanism reference — no code copied; weight algebra on lanes instead of typed relation rules |
| No-regression gate (§11) | SyFox-native: transactional derive + bit-exact fabric snapshot/restore; policy refined by a full strength-scan (taught-row integrity + no manufactured certainty) |
| Jev-parity bench (§12) | SyFox-native: axis lineage from TypeSafe's published Jev material + community classification benchmarks; all numbers produced by this repo's binary |
| Recall (§13) | Hopfield-style associative memory reinterpreted as settled-field cosine — SI-physics-native, zero symbol-space similarity |

## 10. Derivation layer (offline, explicit)

SyFox's lanes so far carried only lived experience (`learn`). The derivation
layer lets the field grow knowledge **derived from** its own fabric — the
substrate-level kernel of what the upstream SI stack does with explicit rule
machinery (meta-learner macro-rules, dreamer emergent resonances, analogical
mapping). The primitives differ by constitution: SI composes typed relation
triples over a fact graph; SyFox composes **weighted lane paths** — weight
algebra and field dynamics, no classifiers. All engines are OFFLINE, explicit
CLI steps; `decide()` stays read-only and byte-identical unless the operator
derives on a model. Provenance: derived lanes carry a **generation counter**
(experienced / human-promoted = 0, each composition step +1, cap 3), persisted
as a versioned tail in `substrate.bin`; v1 files load clean (generation 0).

**1. Lane induction, compose mode** (`syfox derive --model DIR`): a two-hop
path A→B→C is evidence for A↔C, with three guards that each came out of a
real benchmark failure:
* *acoustic-mass damping* — path evidence is damped by `1/√mass(B)`, the same
  law `settle()` applies to energy through B; without it, hub tokens ("the",
  "command") fabricated corroboration between unrelated concepts and two demo
  argmaxes flipped;
* *top-K corroboration* (K=3) — a dense fabric offers hundreds of two-hop
  paths; treating them as independent evidence saturated the field (noul
  collapsed to 0.999 everywhere);
* *ghost-guarded verifier* — every derived lane is re-checked against the
  LIVE fabric (snapshot ghosts of already-dissolved lanes cannot justify
  anything), over-claims are healed down to their justification, unsupported
  ones dissolve. Composition is raise-only: observed lanes are never weakened.

**2. Lane induction, harvest mode** (`syfox derive --model DIR --examples
FILE.jsonl`): static algebra guesses; the settled field knows. Replay states
through inject→settle and record which concept pairs genuinely **co-activate**.
Dissipation does the filtering — cross-scenario energy decays before it
registers. Observed lanes are untouchable; gen-1 lanes must be re-nominated by
every run or they dissolve (justification is recomputable, hence deterministic).
Labels are not needed: the field learns structure from unlabelled exposure.

**3. Dreamer** (`syfox dream --model DIR [--steps N] [--seed S]`): inject
small random energy patterns, settle, and watch for undriven nodes that lit
up **without any direct lane to the driven set** — reached only through the
topology's own interference. Candidates go to `model-*/mutations.jsonl`. The
substrate is never modified by dreaming; a human sets `validated:true` on a
line and `syfox promote` applies it as a premise-grade lane. Same seed → same
dream, bit for bit.

**4. Analogy** (`syfox analogs --model DIR --concept WORD`, read-only): a
concept's signature is its lane neighbourhood (who it touches, both
directions); structural isomorphism = signature Jaccard; novelty potential
follows the L9 formulation `Phi = I·exp(k·d)`. High Phi = structurally aligned
AND fabric-distant: a transfer *hypothesis* for the human, never auto-applied.

*Benchmark (v0.2 → v2.1, honest).* The v0.2 verdict was measured on
resubstitution (eval rows = training rows): compose sharpened tickets demos
(technical conf 0.251→0.454) but flipped game/guard rows; harvest lifted
billing 0.604→0.912 but flipped a close call; no strength avoided taught-row
flips on the mixed-label fabrics. v2.1 replaces the measurement basis: the
gate now replays the HELD-OUT split (generalization safety, not memorized
behavior) and the harness decides per model — game ships derived (253 lanes,
0 held-out flips, accuracy held 0.667→0.667), tickets (9 flips) and guard
(1 flip) ship un-derived with the reason logged. Same enforcement philosophy,
honest measurement basis.

## 11. The no-regression gate (transactional derivation)

Derivation writes into the fabric; the gate (`core/gate.hpp`, CLI
`syfox derive --gate FILE.jsonl`) makes that write safe by construction:

1. replay every labelled row + generated close-call probes (cross-domain
   state mixtures, deterministic generator shared with `bench`) → baseline
   decision signatures;
2. snapshot the fabric **bit-exactly** — `snapshot_fabric()` copies each
   node's lane vector in insertion order plus the provenance map, because
   `settle()` sums over lanes in vector order (a weight set is not the
   state);
3. run the derivation (conservative presets: harvest co_floor 0.15/gain
   0.35/budgets 8·512, compose gain 0.25/top-2/budget 4);
4. replay the same probe set → compare;
5. **policy**: (a) any argmax flip on a TAUGHT row reverts — labelled
   knowledge must survive derivation; (b) any RISE in mean confidence on the
   MIXED probes reverts — close-call states are ambiguous by construction,
   their argmax may re-resolve, but manufactured certainty about them is the
   exact failure the gate exists to kill;
6. revert = `restore_fabric(snapshot)` → the fabric is bit-identical and
   `decide()` output is byte-identical (pinned by test); commit = the caller
   saves the model.

Strength-scan evidence (scripts/syfox_gate_scan.cpp, all strengths from
default down to 32 lanes): tickets reverts everywhere (8–18 taught flips —
its seed data mixes refund/invoice labels across billing and sales), guard
reverts at every useful strength, game commits at the conservative strength.
Per-model outcome: `model-game` ships derived (668 lanes, 0 taught flips,
mixed mean confidence 0.975→0.886 — the fabric got MORE honest about close
calls; demo argmaxes 3/3 unchanged; `make models` reproduces). `model-tickets`
and `model-guard` ship un-derived because the gate proves it, not because a
human remembered to check.

## 12. The Jev-parity benchmark (`syfox bench`)

`core/bench.hpp` measures the axes the System One model class is judged on
(axis lineage verified against TypeSafe's published Jev material and the
community classification benchmarks, Sept 2026): decision accuracy, latency
percentiles, calibration (ECE, 10 bins, + conf-when-correct vs
conf-when-wrong gap), honesty (OOD probes built from deterministic nonsense
vocabulary checked against the model's own — untaught tokens must defer),
guardrail hold precision/recall (noul confusion at the 0.5 threshold),
determinism (full replay compared byte-wise), and close-call margin
distribution (mixed cross-domain states; the same generator the gate
replays). Accuracy/calibration/guardrail aggregate over LABELLED rows only;
unlabelled mixed probes never pollute accuracy denominators (a real bug the
e2e suite caught). Every report carries a `jev_reference` block with the
PUBLISHED figures (67.8% workflow accuracy, 70–500 ms, pi-warden 88% hold
precision) for side-by-side reading — axes shared, numbers not compared
against toy-data resubstitution.

Seed-model results, v2.1 methodology — HELD-OUT split (70/30, stratified,
deterministic; `make bench-heldout`), post-gate derivation state. The v0.2
table that stood here measured in-domain resubstitution (eval rows = training
rows) and is retired; in-sample numbers remain available via `make bench` as
a regression contrast only.

| model | held-out choice acc (n) | score acc (n) | choice ECE | OOD defer | p50 / p95 |
|---|---|---|---|---|---|
| tickets | 1.000 (6) | 0.667 (6) | 0.012 | 1.00 | 64 / 80 µs |
| game (derived, gate PASS) | 0.667 (3) | — | 0.329 | 1.00 | 29 / 40 µs |
| guard | 0.750 (4) | — | 0.200 | 1.00 | 34 / 45 µs |

Reading: the held-out gap against in-sample (tickets 1.000 vs 1.000; guard
0.750 vs 0.778) is now measured instead of assumed. Calibration is fitted on
held-out rows only (2–3 scalars; multi-class NLL objective, 1-bit ECE
adoption guard) — tickets reaches ECE 0.012; game/guard are held above the
0.1 bar by a single held-out error each, which is arithmetic (1 wrong of
3–4 emitted forces ECE ≥ ~0.2), not a tuning failure. The headline metric is
the coverage-vs-accuracy curve (`make coverage`): tickets trades 100%
coverage @ 83.3% accuracy for 50% coverage @ 100% — the confident subset is
trustworthy, measurably. Paraphrase augmentation was implemented, measured on
the held-out split, and REJECTED by default (intern mass is linear, injection
damps 1/√mass: re-teaching near-duplicates re-weights the field toward the
training surface forms; tickets 1.000 → 0.500). The derivation gate replays
held-out rows and decides per model: game ships derived, tickets/guard ship
un-derived. Honesty stays structural (1.00 OOD defer everywhere); latency is
three to four orders of magnitude under the Jev band with the same one-pass
shape.

## 13. Recall (associative retrieval in energy space)

`core/recall.hpp`, CLI `syfox recall --state '...' --memories FILE.jsonl`.
Content-addressable memory the only way the constitution allows: settle the
query into an energy fingerprint (one float per node, L2-normalized), settle
each stored memory the same way, rank by cosine of the two settled fields.
No token comparison, no string distance, no n-gram overlap, no embedding
table, no transformer — similarity is measured in the substrate's own state
space, so two states resonate exactly as far as the field routes them
together. Unknown vocabulary resonates with nothing (empty hits — honest
silence, since `inject` skips unknown tokens). Read-only, bit-deterministic;
memory stores accept `{"state","label"}` rows or the training schema
(label = first label value). On the derived game model the zombie query
recalls the flee lessons at 0.9696–0.9421 with fight lessons at ≤0.856.

## 14. v3 — the hidden-test firewall and the evidence ledger

### 14.1 Firewall (core/firewall.hpp)

The 70/15/15 train/cal/hidden contract is enforced where a typo cannot
bypass it: the CLI consults the file's split role (decided by its filename
suffix — `_train`, `_cal`, `_hidden`, `_heldout`, anything else is
unclassified and stays readable, because worksheets and paraphrase files
must remain learnable) before opening anything.

| role      | suffix            | learn | calibrate | derive --gate | bench |
|-----------|-------------------|-------|-----------|---------------|-------|
| train     | `_train.jsonl`    | YES   | yes       | yes           | yes   |
| heldout   | `_heldout.jsonl`  | yes   | YES       | yes           | yes   |
| cal       | `_cal.jsonl`      | no    | YES       | yes           | yes   |
| hidden    | `_hidden.jsonl`   | NO    | NO        | NO            | YES   |

Rationale: calibration files exist to fit 2–3 scalars (temperature/Platt)
on rows the fabric never learned from. Hidden files exist to be SCORED,
once. A hidden row that leaks into any build path is not a mistake — it is
a different (worthless) number. `make firewall-check` runs the four
refusals against the real binary.

### 14.2 Evidence ledger (v3, Milestone 3)

Every lane key carries a `LaneEvidence` record: `support_events`,
`counter_events` (anti-Hebbian dissolutions), `generation` (derivation
provenance), `first_seq` / `last_seq` (teach-event window), and the
engine-set context tag (which substrate/model owned the event). The ledger
persists as an optional tail on `substrate.bin`; v2 files load unchanged
(the tail is detected at EOF) and v3 files load on v2 readers that stop at
the v2 end.

`decide --evidence` projects the ledger for the lanes the decision
actually used — a READ-ONLY view. The field is driven by lane weights;
the ledger never feeds back into physics. It exists so a decision can be
audited after the fact: which lanes, how strong, taught when, by which
corpus, disputed how many times.

### 14.3 Contradictions

Teaching the same (state, question) with a different outcome is a
contradiction: it increments the learn report's `contradictions` counter,
writes a contradiction record (old outcome, new outcome, both seq numbers,
state hash, context), and marks the affected lanes with a counter event —
the anti-Hebbian term weakens the disputed binding exactly as the physics
specifies. There is no last-write-wins: the field keeps the binding its
weights support, and the dispute stays on the record (`contested: true`
in the decide evidence view). Silent override is a design error, and the
adversarial suite (§15 of the README; `make adv-big`) tests for it with a
control arm that separates contradiction-specific damage from the generic
cost of re-teaching.

## 15. v3 — multicore settle and the GPU gate (M5)

### 15.1 CSR mirror

The settle hot path reads a compressed-sparse-row mirror of the out-lane
fabric: `csr_off_` (n+1 offsets), `csr_dst_`, `csr_w_` — contiguous
buffers, rebuilt lazily when the fabric mutates. The build copies each
source's lane vector verbatim, so per-source iteration order is exactly
the map order the sequential settle used: every float sum keeps its exact
operand order and results are bit-identical. This is the layout a GPU
port generalizes.

### 15.2 Deterministic OpenMP settle

`make omp` builds the same sources with `-fopenmp`. The diffusion pass
partitions sources with a static schedule (contiguous ascending chunks);
each thread scatters into its own buffer; the buffers are combined in
ascending thread order; therefore every accumulator sees contributions in
ascending SOURCE order — the sequential order. Bit-identity is a gate,
not a hope: `make omp-identity` fails the build if sequential and
`--threads 2` reports differ in any block (routing, calibration, honesty,
guardrail). Measured honestly: below ~10k nodes on a 2-core sandbox the
deterministic parallel settle is slower than sequential (thread
coordination exceeds the gain); it pays on denser fabrics.

### 15.3 Batched throughput and the GPU gate

`bench --throughput N` measures decisions/sec with N worker threads, each
owning a PRIVATE engine copy (decisions mutate the field, so workers never
share a substrate). It is a performance axis, not an accuracy headline —
a non-zero checksum proves the work happened. Measured on the 3,518-node
tickets fabric: 2,390 → 3,714 decisions/sec at N=2 on 2 cores.

The GPU port gate is MEASURED, not assumed: `make density` reports per-
fabric density and mean out-degree (tickets 0.0067 @ degree 23.5, game
0.0458, guard 0.0485). A GPU port happens when dense substrates make the
transfer costs pay for themselves — until then it stays design, and the
SoA/CSR buffers above are the only GPU-specific preparation the core
carries. No neural network enters the core at any point; the parallelism
is the SAME physics, executed on more cores.

## 16. v3.2 — the semantic field, context-sensitive lanes, retrieval by default, the two-stage router

The largest boundary-layer addition since v3.0, and the first that adds a
second field to the substrate. Everything below is deterministic, seeded by
nothing, fitted to nothing — there is still no neural network, no
transformer, and no pattern-matching classifier in the core.

### Stage 1 + 4 — the semantic vector field (64-dim, resonance)

Every concept node carries a 64-dim semantic vector built in two
deterministic steps. The LEXICAL LAYER hashes the concept string's character
trigrams into 64 buckets with FNV-1a-signed weights and L2-normalizes —
"card_arrival" and "card_delivery" share trigrams, so their base vectors
align; unrelated words stay near-orthogonal. FABRIC GROUNDING then runs two
smoothing passes over the Hebbian lanes (v_i <- normalize(v_i + 0.5 *
lane-weighted mean of v_j)): the fabric's own co-occurrence structure shapes
the vectors, which is meaning diffusing along the lanes, not a fit. From the
vectors, three derived quantities:

- **omega_semantic** (Stage 1, "concept frequency encoding") — a scalar
  projection onto one fixed deterministic direction, mapped to [0,1]. Two
  nodes with close omega resonate harder, like coupled oscillators that are
  nearly tuned; the fixed axis makes "card" map to the same omega in every
  fabric.
- **resonance edges** (Stage 4, "semantic field") — the top-6 cosine
  neighbours with cos >= 0.50, stored as a CSR adjacency in the substrate's
  v4 file tail. During settle, every SOURCE (and only sources — the working
  set stays capped) leaks a fraction (sem_coupling 0.12) of its post-diffusion
  energy along these edges, distributed proportionally to cos x
  frequency-match. Energy is CONSERVED exactly: the pool is subtracted from
  the source and distributed; decay applies next pass as always.
- **typed reads** (Option 2) — cos >= 0.82 reads as synonym-grade kin, >=
  0.66 as related; `syfox analogs` and the evidence surface expose the type.
  The v2.1 synonym folding table remains the string-level canonicalization;
  the vectors add the fabric-shaped layer above it.

Readout gains a semantic-neighbour term (sem_hop 0.10 x cosine x neighbour
energy): energy resting on semantically similar nodes counts for a probe
even where no lane connects them.

### Stage 2 — context-sensitive lanes

A lane may carry a CONTEXT SIGNATURE: required and forbidden context words,
learned from the co-occurrence statistics of the lessons that laid it
(`add_ctx_support` accumulates counts at learn time; `finalize_contexts`
promotes surviving counts — top-4 with count >= 2 — into required sets, and
cross-lane diffs (strong in a sibling lane's context, absent here) into
forbidden words, top-2). At settle, a lane's effective weight is
w x factor: any forbidden word present in the decision's own injected
tokens -> x0.20; required words missing -> x0.60..1.0 interpolated by match
count; all present -> x1.0 (bit-identical to the plain path). The node's
total outflow scales by sum(w x f)/sum(w), so damped lanes carry LESS and
the energy stays home — context-mismatched routes fade physically instead of
being pruned by a rule. The same "card" can now route toward card_arrival
when arrive/receive are in the state and toward card_delivery_estimate when
estimate/how-long is.

### Persistence and the replay contract

The semantic field, resonance edges and lane contexts live in a magic-
guarded v4 tail of substrate.bin. Pre-v3.2 files simply end at the v3
evidence tail: has_semantics() is false and every semantic code path is
inert, so every v2/v3 model replays bit-for-bit under the new binary
(test-enforced). `intern()` invalidates the field when the fabric grows —
a stale top-k neighbourhood must never be indexed — and the next
`save_model()` rebuilds it deterministically. `--no-semantics` /
`set_semantics(false)` is the runtime kill switch.

### Retrieval by default

A model dir may ship `memories.jsonl` (rows {"label": ..., "state": ...}).
load_model fingerprints each memory once (its own settled field, the
recall.hpp Hopfield-style mechanism); every decide then ranks memories by
settled-field cosine against the query and injects the top-5 outcomes at a
faint dose (0.30 x inject_energy) BEFORE the state settles — a physical
prior from lived experience, deterministic, bit-reproducible for the same
state + memories. No memories file => the whole path is inert (pre-3.2
behavior). Flags: --memories FILE (explicit store), --retrieval-topk,
--retrieval-dose, --no-retrieval. usage.retrieval discloses what primed the
decision, in the CLI, the C API and the HTTP bridge alike.

### Stage 3 — the semantic hierarchy

A model dir may ship `hierarchy.json` {"intents": {intent: category},
"categories": {category: {"criteria": ...}}, "floor": 0.35}. Both levels are
TRAINED anchors in one fabric; at decide, stage-1 category energies scale the
stage-2 intent candidates by floor + (1-floor) x cat/max_cat — a readout-side
two-stage classification. MEASURED on bank77 (cal, 1,003 rows): gate OFF
0.133 > floor 0.85: 0.129 > 0.5: 0.113 > 0.35: 0.111 — the category readout
is not additive on this fabric (consistent with the v3.1.0 finding that the
hierarchical first-token split was measured-negative), so the shipped
model-b77-sem carries floor 1.0 = gate off, kept as documentation. The
mechanism stays available (--no-hierarchy to disable elsewhere).

### The dedicated bank77 fabric + the two-stage router

The v3.2 answer to high-cardinality collapse is ARCHITECTURAL, as the
v3.0 design intended: a DEDICATED fabric per hard domain, routed by a small
dedicated router fabric.

- **model-b77-sem** (dedicated bank77): 9,000 intent lessons + 9,000
  category lessons (12 chunks), 1,648 nodes / 217,729 lanes, 7,728 resonance
  edges, 141,920 lane contexts, 256 retrieval memories, calibrated. Hidden
  (3,080 rows, scored once, energy-norm): **0.158** — vs 0.081 (v3.1.0
  dedicated+opaque) and 0.023 (77-way in the shared giant fabric). Ablations
  on cal: --no-semantics 0.039 (the semantic field is the jump: 0.039 ->
  0.133), retrieval neutral, hierarchy gate negative (above).
- **model-router16** (two-stage, stage 1): a 4,742-node fabric trained ONLY
  on router questions (16 anchors, one per giant domain, 60 rows/domain).
  Cal 16-way routing 0.444 (chance 0.0625), ECE 0.380 -> 0.086.
  `decide --router model-router16 ...` settles the state on the router
  fabric, picks the domain anchor, then the mapped domain fabric (router.json
  "models") decides the real questions; the route is disclosed in the output
  as route:{anchor,confidence,top,model}. Physics-based routing, no
  classifier — the two-stage architecture from the v3.0 design, now measured.

### Zero-shot, re-measured under v3.2 (docs/ZEROSHOT.md Part 4)

Email spam on the tickets-only fabric (never taught spam): 6/6 (plain) and
6/6 (semantic rebuild). SMS spam: 5/6 both. Snake on the giant fabric: 1/8
both; Tic-tac-toe: picks 1 at conf 0 — multi-constraint composition is
still not a fabric property, and the semantic field does not fake it.
Honest as ever: the semantic field moves TRAINED-discrimination and
vocabulary-overlap zero-shot, not symbolic reasoning.