Download docs/ARCHITECTURE.md from Mr-DS-ML-85/SyFox: direct link, hf CLI and curl.
- Browser
- Download file 31.6 kB
-
https://huggingface.co/Mr-DS-ML-85/SyFox/resolve/main/docs/ARCHITECTURE.md
- Command line
-
hf download hf://Mr-DS-ML-85/SyFox/docs/ARCHITECTURE.md
-
curl -L -o ARCHITECTURE.md https://huggingface.co/Mr-DS-ML-85/SyFox/resolve/main/docs/ARCHITECTURE.md
SyFox Architecture — the SI substrate core
This document specifies exactly what happens between "state in" and "decision out", and which mechanisms are ported from the Synthetic-Intelligence (SI) research substrate. The rule that governs this codebase:
The core is SI physics only. No transformer, no neural network, no pattern-matching classifier. Everything below is field dynamics over an explicit concept graph. Tools (JSON, CLI, HTTP, calibration math) live outside the core and never rewrite it.
1. Concepts and acoustic mass
Every lowercase folded token interned by the engine becomes a concept node
(core/si_substrate.hpp, Substrate::intern). Nodes carry:
- mass — a raw occurrence count:
intern()adds exactly1.0each time the concept appears in any lesson. It is linear, not log-compressed. Lineage: SI "acoustic mass" — heavier units respond less to the same stimulus. The sub-linear behavior lives at the use sites, not in the counter: injection depositsenergy / sqrt(mass)and Hebbian bind strength scales by1/sqrt(mass), so frequent concepts ("the", "please") cannot flood the field. - energy — transient; injected and dissipated per decision.
(An earlier revision also carried a salience field. Audit found it was
write-only — incremented at injection, read nowhere; gating selects sources
by energy — so it was removed. Dead state in the core is doc debt.)
Token folding (fold) is deterministic string normalization (plural / -ed /
-ing / trailing e), applied identically to states, options, and
instructions. It is I/O hygiene, not linguistics and not pattern matching.
2. Lanes (Hebbian bindings)
Lanes are weighted edges between concepts, stored sparsely per node with a per-node lane cap (weakest lane evicted, bounded memory).
- Bind (
hebbian_lesson): for a labelled lesson, every active state concept binds to every outcome concept withη · (1/√mass_a + 1/√mass_b) · scale. Adjacent state tokens also bind (the co-occurrence fabric that lets energy diffuse inside a phrase). Lineage: SI Hebbian lane bindings inphysics.hpp. - Weaken (
weaken): counter-evidence subtracts lane weight; a lane whose weight reaches zero dissolves. Used by Noul lessons labelled false. - Decay (
lane_decay): every lesson multiplies all lanes by 0.995 — unused routes fade.
Learning is substrate rewiring. There are no fitted weights anywhere in the core; the calibration tool (below) is external and read-only.
3. Injection
A state is tokenized; every token that exists in the vocabulary deposits
inject_energy / sqrt(mass) on its node. Unknown tokens stay dark — this is
the first half of the honest-silence contract (a state of nothing but unknown
words settles zero energy and defers).
4. Settle (the System One pass)
settle() runs up to k_settle (default 8) dissipative diffusion passes:
Sources are gated by source cap (
source_cap, 24 by default): the top-K nodes propagate each pass — ranked by energy by default, or by salience whensalience_gatingis on. Gated nodes keep their energy (they remain readout-visible); the cap gates flow, it does not annihilate. (An earlier version pruned energy outright and destroyed 97% of the field — the gating form is the correct physics and the tests pin it.)SI salience lineage, restored after audit. The SI substrate runs a cavity salience integrator,
s = tanh(s·decay + gain·|velocity|)(physics.hpp,salience_gain = 0.05,salience_decay = 0.995), and its TSDA layer samples the live working cap from[working_cap−4, working_cap]= [5,9] (Miller 7 ± 2). SyFox ports both: the integrator runs every pass on the motion proxy|ΔE|(a node at rest is an exact fixed point, mirroring SI's sparsity guard), andmiller_window = truesamples the live cap from[source_cap−4, source_cap]each decision. One deliberate adaptation: SI draws the sample from a seeded mt19937 stream; SyFox derives it from the decision's state hash, so the same state always settles to the same field bit-for-bit (the persistence contract outranks stochasticity).inject()spikes salience to 1.0 on touch (TSDA spike semantics).Benchmark (10 in-domain decisions, 3 domains). default energy gating 10/10 with calibrated-shaped confidences; salience ranking at the wide cap 4/10 (motion history is a poor selector when the cap rarely trims); salience + literal [5,9] window 10/10 argmax but probabilities saturate at 1.0 (calibration collapse — needs the v0.2 eval harness before adoption); Miller window at wide cap 9/10. Verdict: default stays energy-gated at a fixed 24; the SI-faithful modes ship behind config flags until a held-out eval says otherwise.
Each source retains
(1 − diffusion)of its decayed energy and flowsdiffusionout along its lanes, split proportionally to lane weight:next[b] += e·decay·diffusion·(w_ab / Σw_a).Total energy is non-increasing (dissipative); the pass loop stops early when the field relaxes (
total_before − total_after < eps).
One settle per decision — all questions read the same settled field. Adding questions never re-settles the substrate: flat marginal latency by structure, not by scheduling.
Derivation, not retrieval. Because diffusion is multi-hop, a state that
was never in any lesson still settles to a computed answer: energy crosses
the co-occurrence fabric into outcome lanes laid down by other lessons.
Probe (two lessons, alpha beta→x and beta gamma→y): the untaught state
alpha alone settles with y at 0.494 — lesson 2 never mentioned alpha; the
two-hop path alpha→beta→y carried the evidence. The boundary is the taught
vocabulary and lane fabric: concepts never interned have no node, unknown
words settle to zero energy and defer (§6). This is the substrate-level
kernel of the SI stack's explicit derivation layers (meta-learner
macro-rules, dreamer emergent resonances, analogical mapping) — not yet
ported; see ROADMAP v0.3.
5. Readout (resonance sweep, two lanes)
Probes read the settled field at their own concepts: direct node energy plus a hop term through lanes. Two readout lanes exist because two question shapes need two different questions of the field:
- Specific (
choice,score): mean lane support —hop · Σ w·e_n / Σ w. Rewards precise coupling; a node bound diffusely to half the vocabulary cannot out-read a node bound exactly to this state. Options are then normalized by softmax at the fitted temperature. - Support (
noul): active lane mass —hop · Σ w·e_n(unnormalized). A statement may be corroborated broadly; what matters is how much lane-connected energy the field offers it. The ratio to total field energy is the statement's support.
Both are field measurements. No weights are fitted at readout; no classes exist in the core.
6. Honest silence
If settled field energy is below silence_floor (0.05):
- empty state →
deferred: true, reason: "empty_state" - all-unknown vocabulary →
reason: "unknown_vocabulary" - settled but dark →
reason: "honest_silence"
The engine refuses to guess from nothing. Lineage: the SI interface's honest-silence dispatch — answers only when the readout actually collapsed.
7. Calibration (a tool, not core)
fit_calibration collects readout energies on labelled rows and fits, purely
outside the core:
- temperature per choice/score — golden-section search on NLL of
softmax(E_correct/T, E_wrong/T). - Platt scaling for noul — logistic regression
sigmoid(a·support + b)on supports, gradient descent to convergence (separable data needs a steep slope; the fit runs 4000 iterations with decaying lr).
Calibration changes how energies are reported as probabilities, never how
they are produced. Confidence for choice/score is the normalized-entropy
margin of the distribution; for noul it is |p − 0.5|·2.
8. Persistence
save_model writes substrate.bin (vocabulary + masses + lanes, binary),
calibration.json, and meta.json. Loading is exact — the same state settles
to the same readout bit-for-bit (pinned by tests).
9. Provenance
| Mechanism | Origin |
|---|---|
| Energy injection + field settle | SI physics.hpp substrate dynamics |
| Source capping (top-K, energy or salience) | SI working-set cap + TSDA live_cap ([cap−4, cap], Miller 7 ± 2) |
| Salience integrator tanh(s·decay + gain·motion) | SI physics.hpp cavity salience (verified live there) |
| Hebbian lane bindings + decay | SI lane learning |
| Honest silence | SI dispatch contract |
| Typed Choice/Score/Noul API | interface shape inspired by TypeSafe Jev |
| Temperature/Platt calibration | classical statistics (tool layer) |
| Derivation layer (§10) | SyFox-native design after studying SI meta_learner / dreamer / analogy as mechanism reference — no code copied; weight algebra on lanes instead of typed relation rules |
| No-regression gate (§11) | SyFox-native: transactional derive + bit-exact fabric snapshot/restore; policy refined by a full strength-scan (taught-row integrity + no manufactured certainty) |
| Jev-parity bench (§12) | SyFox-native: axis lineage from TypeSafe's published Jev material + community classification benchmarks; all numbers produced by this repo's binary |
| Recall (§13) | Hopfield-style associative memory reinterpreted as settled-field cosine — SI-physics-native, zero symbol-space similarity |
10. Derivation layer (offline, explicit)
SyFox's lanes so far carried only lived experience (learn). The derivation
layer lets the field grow knowledge derived from its own fabric — the
substrate-level kernel of what the upstream SI stack does with explicit rule
machinery (meta-learner macro-rules, dreamer emergent resonances, analogical
mapping). The primitives differ by constitution: SI composes typed relation
triples over a fact graph; SyFox composes weighted lane paths — weight
algebra and field dynamics, no classifiers. All engines are OFFLINE, explicit
CLI steps; decide() stays read-only and byte-identical unless the operator
derives on a model. Provenance: derived lanes carry a generation counter
(experienced / human-promoted = 0, each composition step +1, cap 3), persisted
as a versioned tail in substrate.bin; v1 files load clean (generation 0).
1. Lane induction, compose mode (syfox derive --model DIR): a two-hop
path A→B→C is evidence for A↔C, with three guards that each came out of a
real benchmark failure:
- acoustic-mass damping — path evidence is damped by
1/√mass(B), the same lawsettle()applies to energy through B; without it, hub tokens ("the", "command") fabricated corroboration between unrelated concepts and two demo argmaxes flipped; - top-K corroboration (K=3) — a dense fabric offers hundreds of two-hop paths; treating them as independent evidence saturated the field (noul collapsed to 0.999 everywhere);
- ghost-guarded verifier — every derived lane is re-checked against the LIVE fabric (snapshot ghosts of already-dissolved lanes cannot justify anything), over-claims are healed down to their justification, unsupported ones dissolve. Composition is raise-only: observed lanes are never weakened.
2. Lane induction, harvest mode (syfox derive --model DIR --examples FILE.jsonl): static algebra guesses; the settled field knows. Replay states
through inject→settle and record which concept pairs genuinely co-activate.
Dissipation does the filtering — cross-scenario energy decays before it
registers. Observed lanes are untouchable; gen-1 lanes must be re-nominated by
every run or they dissolve (justification is recomputable, hence deterministic).
Labels are not needed: the field learns structure from unlabelled exposure.
3. Dreamer (syfox dream --model DIR [--steps N] [--seed S]): inject
small random energy patterns, settle, and watch for undriven nodes that lit
up without any direct lane to the driven set — reached only through the
topology's own interference. Candidates go to model-*/mutations.jsonl. The
substrate is never modified by dreaming; a human sets validated:true on a
line and syfox promote applies it as a premise-grade lane. Same seed → same
dream, bit for bit.
4. Analogy (syfox analogs --model DIR --concept WORD, read-only): a
concept's signature is its lane neighbourhood (who it touches, both
directions); structural isomorphism = signature Jaccard; novelty potential
follows the L9 formulation Phi = I·exp(k·d). High Phi = structurally aligned
AND fabric-distant: a transfer hypothesis for the human, never auto-applied.
Benchmark (v0.2 → v2.1, honest). The v0.2 verdict was measured on resubstitution (eval rows = training rows): compose sharpened tickets demos (technical conf 0.251→0.454) but flipped game/guard rows; harvest lifted billing 0.604→0.912 but flipped a close call; no strength avoided taught-row flips on the mixed-label fabrics. v2.1 replaces the measurement basis: the gate now replays the HELD-OUT split (generalization safety, not memorized behavior) and the harness decides per model — game ships derived (253 lanes, 0 held-out flips, accuracy held 0.667→0.667), tickets (9 flips) and guard (1 flip) ship un-derived with the reason logged. Same enforcement philosophy, honest measurement basis.
11. The no-regression gate (transactional derivation)
Derivation writes into the fabric; the gate (core/gate.hpp, CLI
syfox derive --gate FILE.jsonl) makes that write safe by construction:
- replay every labelled row + generated close-call probes (cross-domain
state mixtures, deterministic generator shared with
bench) → baseline decision signatures; - snapshot the fabric bit-exactly —
snapshot_fabric()copies each node's lane vector in insertion order plus the provenance map, becausesettle()sums over lanes in vector order (a weight set is not the state); - run the derivation (conservative presets: harvest co_floor 0.15/gain 0.35/budgets 8·512, compose gain 0.25/top-2/budget 4);
- replay the same probe set → compare;
- policy: (a) any argmax flip on a TAUGHT row reverts — labelled knowledge must survive derivation; (b) any RISE in mean confidence on the MIXED probes reverts — close-call states are ambiguous by construction, their argmax may re-resolve, but manufactured certainty about them is the exact failure the gate exists to kill;
- revert =
restore_fabric(snapshot)→ the fabric is bit-identical anddecide()output is byte-identical (pinned by test); commit = the caller saves the model.
Strength-scan evidence (scripts/syfox_gate_scan.cpp, all strengths from
default down to 32 lanes): tickets reverts everywhere (8–18 taught flips —
its seed data mixes refund/invoice labels across billing and sales), guard
reverts at every useful strength, game commits at the conservative strength.
Per-model outcome: model-game ships derived (668 lanes, 0 taught flips,
mixed mean confidence 0.975→0.886 — the fabric got MORE honest about close
calls; demo argmaxes 3/3 unchanged; make models reproduces). model-tickets
and model-guard ship un-derived because the gate proves it, not because a
human remembered to check.
12. The Jev-parity benchmark (syfox bench)
core/bench.hpp measures the axes the System One model class is judged on
(axis lineage verified against TypeSafe's published Jev material and the
community classification benchmarks, Sept 2026): decision accuracy, latency
percentiles, calibration (ECE, 10 bins, + conf-when-correct vs
conf-when-wrong gap), honesty (OOD probes built from deterministic nonsense
vocabulary checked against the model's own — untaught tokens must defer),
guardrail hold precision/recall (noul confusion at the 0.5 threshold),
determinism (full replay compared byte-wise), and close-call margin
distribution (mixed cross-domain states; the same generator the gate
replays). Accuracy/calibration/guardrail aggregate over LABELLED rows only;
unlabelled mixed probes never pollute accuracy denominators (a real bug the
e2e suite caught). Every report carries a jev_reference block with the
PUBLISHED figures (67.8% workflow accuracy, 70–500 ms, pi-warden 88% hold
precision) for side-by-side reading — axes shared, numbers not compared
against toy-data resubstitution.
Seed-model results, v2.1 methodology — HELD-OUT split (70/30, stratified,
deterministic; make bench-heldout), post-gate derivation state. The v0.2
table that stood here measured in-domain resubstitution (eval rows = training
rows) and is retired; in-sample numbers remain available via make bench as
a regression contrast only.
| model | held-out choice acc (n) | score acc (n) | choice ECE | OOD defer | p50 / p95 |
|---|---|---|---|---|---|
| tickets | 1.000 (6) | 0.667 (6) | 0.012 | 1.00 | 64 / 80 µs |
| game (derived, gate PASS) | 0.667 (3) | — | 0.329 | 1.00 | 29 / 40 µs |
| guard | 0.750 (4) | — | 0.200 | 1.00 | 34 / 45 µs |
Reading: the held-out gap against in-sample (tickets 1.000 vs 1.000; guard
0.750 vs 0.778) is now measured instead of assumed. Calibration is fitted on
held-out rows only (2–3 scalars; multi-class NLL objective, 1-bit ECE
adoption guard) — tickets reaches ECE 0.012; game/guard are held above the
0.1 bar by a single held-out error each, which is arithmetic (1 wrong of
3–4 emitted forces ECE ≥ ~0.2), not a tuning failure. The headline metric is
the coverage-vs-accuracy curve (make coverage): tickets trades 100%
coverage @ 83.3% accuracy for 50% coverage @ 100% — the confident subset is
trustworthy, measurably. Paraphrase augmentation was implemented, measured on
the held-out split, and REJECTED by default (intern mass is linear, injection
damps 1/√mass: re-teaching near-duplicates re-weights the field toward the
training surface forms; tickets 1.000 → 0.500). The derivation gate replays
held-out rows and decides per model: game ships derived, tickets/guard ship
un-derived. Honesty stays structural (1.00 OOD defer everywhere); latency is
three to four orders of magnitude under the Jev band with the same one-pass
shape.
13. Recall (associative retrieval in energy space)
core/recall.hpp, CLI syfox recall --state '...' --memories FILE.jsonl.
Content-addressable memory the only way the constitution allows: settle the
query into an energy fingerprint (one float per node, L2-normalized), settle
each stored memory the same way, rank by cosine of the two settled fields.
No token comparison, no string distance, no n-gram overlap, no embedding
table, no transformer — similarity is measured in the substrate's own state
space, so two states resonate exactly as far as the field routes them
together. Unknown vocabulary resonates with nothing (empty hits — honest
silence, since inject skips unknown tokens). Read-only, bit-deterministic;
memory stores accept {"state","label"} rows or the training schema
(label = first label value). On the derived game model the zombie query
recalls the flee lessons at 0.9696–0.9421 with fight lessons at ≤0.856.
14. v3 — the hidden-test firewall and the evidence ledger
14.1 Firewall (core/firewall.hpp)
The 70/15/15 train/cal/hidden contract is enforced where a typo cannot
bypass it: the CLI consults the file's split role (decided by its filename
suffix — _train, _cal, _hidden, _heldout, anything else is
unclassified and stays readable, because worksheets and paraphrase files
must remain learnable) before opening anything.
| role | suffix | learn | calibrate | derive --gate | bench |
|---|---|---|---|---|---|
| train | _train.jsonl |
YES | yes | yes | yes |
| heldout | _heldout.jsonl |
yes | YES | yes | yes |
| cal | _cal.jsonl |
no | YES | yes | yes |
| hidden | _hidden.jsonl |
NO | NO | NO | YES |
Rationale: calibration files exist to fit 2–3 scalars (temperature/Platt)
on rows the fabric never learned from. Hidden files exist to be SCORED,
once. A hidden row that leaks into any build path is not a mistake — it is
a different (worthless) number. make firewall-check runs the four
refusals against the real binary.
14.2 Evidence ledger (v3, Milestone 3)
Every lane key carries a LaneEvidence record: support_events,
counter_events (anti-Hebbian dissolutions), generation (derivation
provenance), first_seq / last_seq (teach-event window), and the
engine-set context tag (which substrate/model owned the event). The ledger
persists as an optional tail on substrate.bin; v2 files load unchanged
(the tail is detected at EOF) and v3 files load on v2 readers that stop at
the v2 end.
decide --evidence projects the ledger for the lanes the decision
actually used — a READ-ONLY view. The field is driven by lane weights;
the ledger never feeds back into physics. It exists so a decision can be
audited after the fact: which lanes, how strong, taught when, by which
corpus, disputed how many times.
14.3 Contradictions
Teaching the same (state, question) with a different outcome is a
contradiction: it increments the learn report's contradictions counter,
writes a contradiction record (old outcome, new outcome, both seq numbers,
state hash, context), and marks the affected lanes with a counter event —
the anti-Hebbian term weakens the disputed binding exactly as the physics
specifies. There is no last-write-wins: the field keeps the binding its
weights support, and the dispute stays on the record (contested: true
in the decide evidence view). Silent override is a design error, and the
adversarial suite (§15 of the README; make adv-big) tests for it with a
control arm that separates contradiction-specific damage from the generic
cost of re-teaching.
15. v3 — multicore settle and the GPU gate (M5)
15.1 CSR mirror
The settle hot path reads a compressed-sparse-row mirror of the out-lane
fabric: csr_off_ (n+1 offsets), csr_dst_, csr_w_ — contiguous
buffers, rebuilt lazily when the fabric mutates. The build copies each
source's lane vector verbatim, so per-source iteration order is exactly
the map order the sequential settle used: every float sum keeps its exact
operand order and results are bit-identical. This is the layout a GPU
port generalizes.
15.2 Deterministic OpenMP settle
make omp builds the same sources with -fopenmp. The diffusion pass
partitions sources with a static schedule (contiguous ascending chunks);
each thread scatters into its own buffer; the buffers are combined in
ascending thread order; therefore every accumulator sees contributions in
ascending SOURCE order — the sequential order. Bit-identity is a gate,
not a hope: make omp-identity fails the build if sequential and
--threads 2 reports differ in any block (routing, calibration, honesty,
guardrail). Measured honestly: below ~10k nodes on a 2-core sandbox the
deterministic parallel settle is slower than sequential (thread
coordination exceeds the gain); it pays on denser fabrics.
15.3 Batched throughput and the GPU gate
bench --throughput N measures decisions/sec with N worker threads, each
owning a PRIVATE engine copy (decisions mutate the field, so workers never
share a substrate). It is a performance axis, not an accuracy headline —
a non-zero checksum proves the work happened. Measured on the 3,518-node
tickets fabric: 2,390 → 3,714 decisions/sec at N=2 on 2 cores.
The GPU port gate is MEASURED, not assumed: make density reports per-
fabric density and mean out-degree (tickets 0.0067 @ degree 23.5, game
0.0458, guard 0.0485). A GPU port happens when dense substrates make the
transfer costs pay for themselves — until then it stays design, and the
SoA/CSR buffers above are the only GPU-specific preparation the core
carries. No neural network enters the core at any point; the parallelism
is the SAME physics, executed on more cores.
16. v3.2 — the semantic field, context-sensitive lanes, retrieval by default, the two-stage router
The largest boundary-layer addition since v3.0, and the first that adds a second field to the substrate. Everything below is deterministic, seeded by nothing, fitted to nothing — there is still no neural network, no transformer, and no pattern-matching classifier in the core.
Stage 1 + 4 — the semantic vector field (64-dim, resonance)
Every concept node carries a 64-dim semantic vector built in two deterministic steps. The LEXICAL LAYER hashes the concept string's character trigrams into 64 buckets with FNV-1a-signed weights and L2-normalizes — "card_arrival" and "card_delivery" share trigrams, so their base vectors align; unrelated words stay near-orthogonal. FABRIC GROUNDING then runs two smoothing passes over the Hebbian lanes (v_i <- normalize(v_i + 0.5 * lane-weighted mean of v_j)): the fabric's own co-occurrence structure shapes the vectors, which is meaning diffusing along the lanes, not a fit. From the vectors, three derived quantities:
- omega_semantic (Stage 1, "concept frequency encoding") — a scalar projection onto one fixed deterministic direction, mapped to [0,1]. Two nodes with close omega resonate harder, like coupled oscillators that are nearly tuned; the fixed axis makes "card" map to the same omega in every fabric.
- resonance edges (Stage 4, "semantic field") — the top-6 cosine neighbours with cos >= 0.50, stored as a CSR adjacency in the substrate's v4 file tail. During settle, every SOURCE (and only sources — the working set stays capped) leaks a fraction (sem_coupling 0.12) of its post-diffusion energy along these edges, distributed proportionally to cos x frequency-match. Energy is CONSERVED exactly: the pool is subtracted from the source and distributed; decay applies next pass as always.
- typed reads (Option 2) — cos >= 0.82 reads as synonym-grade kin, >=
0.66 as related;
syfox analogsand the evidence surface expose the type. The v2.1 synonym folding table remains the string-level canonicalization; the vectors add the fabric-shaped layer above it.
Readout gains a semantic-neighbour term (sem_hop 0.10 x cosine x neighbour energy): energy resting on semantically similar nodes counts for a probe even where no lane connects them.
Stage 2 — context-sensitive lanes
A lane may carry a CONTEXT SIGNATURE: required and forbidden context words,
learned from the co-occurrence statistics of the lessons that laid it
(add_ctx_support accumulates counts at learn time; finalize_contexts
promotes surviving counts — top-4 with count >= 2 — into required sets, and
cross-lane diffs (strong in a sibling lane's context, absent here) into
forbidden words, top-2). At settle, a lane's effective weight is
w x factor: any forbidden word present in the decision's own injected
tokens -> x0.20; required words missing -> x0.60..1.0 interpolated by match
count; all present -> x1.0 (bit-identical to the plain path). The node's
total outflow scales by sum(w x f)/sum(w), so damped lanes carry LESS and
the energy stays home — context-mismatched routes fade physically instead of
being pruned by a rule. The same "card" can now route toward card_arrival
when arrive/receive are in the state and toward card_delivery_estimate when
estimate/how-long is.
Persistence and the replay contract
The semantic field, resonance edges and lane contexts live in a magic-
guarded v4 tail of substrate.bin. Pre-v3.2 files simply end at the v3
evidence tail: has_semantics() is false and every semantic code path is
inert, so every v2/v3 model replays bit-for-bit under the new binary
(test-enforced). intern() invalidates the field when the fabric grows —
a stale top-k neighbourhood must never be indexed — and the next
save_model() rebuilds it deterministically. --no-semantics /
set_semantics(false) is the runtime kill switch.
Retrieval by default
A model dir may ship memories.jsonl (rows {"label": ..., "state": ...}).
load_model fingerprints each memory once (its own settled field, the
recall.hpp Hopfield-style mechanism); every decide then ranks memories by
settled-field cosine against the query and injects the top-5 outcomes at a
faint dose (0.30 x inject_energy) BEFORE the state settles — a physical
prior from lived experience, deterministic, bit-reproducible for the same
state + memories. No memories file => the whole path is inert (pre-3.2
behavior). Flags: --memories FILE (explicit store), --retrieval-topk,
--retrieval-dose, --no-retrieval. usage.retrieval discloses what primed the
decision, in the CLI, the C API and the HTTP bridge alike.
Stage 3 — the semantic hierarchy
A model dir may ship hierarchy.json {"intents": {intent: category},
"categories": {category: {"criteria": ...}}, "floor": 0.35}. Both levels are
TRAINED anchors in one fabric; at decide, stage-1 category energies scale the
stage-2 intent candidates by floor + (1-floor) x cat/max_cat — a readout-side
two-stage classification. MEASURED on bank77 (cal, 1,003 rows): gate OFF
0.133 > floor 0.85: 0.129 > 0.5: 0.113 > 0.35: 0.111 — the category readout
is not additive on this fabric (consistent with the v3.1.0 finding that the
hierarchical first-token split was measured-negative), so the shipped
model-b77-sem carries floor 1.0 = gate off, kept as documentation. The
mechanism stays available (--no-hierarchy to disable elsewhere).
The dedicated bank77 fabric + the two-stage router
The v3.2 answer to high-cardinality collapse is ARCHITECTURAL, as the v3.0 design intended: a DEDICATED fabric per hard domain, routed by a small dedicated router fabric.
- model-b77-sem (dedicated bank77): 9,000 intent lessons + 9,000 category lessons (12 chunks), 1,648 nodes / 217,729 lanes, 7,728 resonance edges, 141,920 lane contexts, 256 retrieval memories, calibrated. Hidden (3,080 rows, scored once, energy-norm): 0.158 — vs 0.081 (v3.1.0 dedicated+opaque) and 0.023 (77-way in the shared giant fabric). Ablations on cal: --no-semantics 0.039 (the semantic field is the jump: 0.039 -> 0.133), retrieval neutral, hierarchy gate negative (above).
- model-router16 (two-stage, stage 1): a 4,742-node fabric trained ONLY
on router questions (16 anchors, one per giant domain, 60 rows/domain).
Cal 16-way routing 0.444 (chance 0.0625), ECE 0.380 -> 0.086.
decide --router model-router16 ...settles the state on the router fabric, picks the domain anchor, then the mapped domain fabric (router.json "models") decides the real questions; the route is disclosed in the output as route:{anchor,confidence,top,model}. Physics-based routing, no classifier — the two-stage architecture from the v3.0 design, now measured.
Zero-shot, re-measured under v3.2 (docs/ZEROSHOT.md Part 4)
Email spam on the tickets-only fabric (never taught spam): 6/6 (plain) and 6/6 (semantic rebuild). SMS spam: 5/6 both. Snake on the giant fabric: 1/8 both; Tic-tac-toe: picks 1 at conf 0 — multi-constraint composition is still not a fabric property, and the semantic field does not fake it. Honest as ever: the semantic field moves TRAINED-discrimination and vocabulary-overlap zero-shot, not symbolic reasoning.