# Native SHARED looped MoE with Fable response SFT

This single **112,237,056-parameter** model descends from the native looped MoE
trained from random initialization for **7,217 updates / 920,770,653 retained
targets**, followed by the SHARED MELT arm's **32 updates / 4,087,238 targets**.
The architecture and tokenizer are unchanged: six physical blocks, width 384,
three loops, 24 routed plus two shared experts, four routed plus both shared
experts active per token, and 1,023-token context.

The additional SFT stage completed **3 epochs / 21
updates**, processing **163,953 input-token exposures**
and **147,597 supervised target exposures**.
It starts a fresh FP32 Adam optimizer from the saved SHARED model. Repeated
epochs reuse the same **68 unique training examples in 19 task families**.
Validation has 5 examples in 3 families; test has 12 in 3 families. These are small,
task-grouped diagnostics, not a broad reasoning or coding benchmark.

Training uses public uploader-labelled Fable **visible explanatory responses**
from TheFusionCube/Fable-5-CoT-Traces, with decoys, ambiguous model metadata,
truncation and unsuitable tasks excluded. The declared chat conversion is not
added as more data. Original answers are preserved verbatim. This is **not
authenticated private-CoT distillation** or proof of Fable-level capabilities.
Teacher responses supply SFT labels; no online teacher or KL loss is used.

Assistant-target CE: validation 5.669114 before
and 5.327462 after SFT; test 5.504149.
These scores do not establish general reasoning improvement. Source receipts,
actual training summary and checkpoint validation remain in provenance/.

Run from this folder with the dependencies in requirements.txt:

```bash
python run.py --prompt "Explain why regex backreferences are not regular." --max-new-tokens 64
```

The CLI wraps the raw question in the exact `### Context` / `### Response` training format.
The default is CPU FP32 reference inference. Optional `--device cuda` uses
the unchanged BF16 parameter-view engine with FP32 masters, buffers and routing.
Formatted prompt plus continuation must fit 1,023 tokens. Use a fresh process.

Only this SHARED model's weights are included. The unchanged source/runtime and
tokenizer are bundled with file hashes. Optimizer/checkpoint payloads, original
training data and cloud credentials are not needed for inference; the resumable
checkpoint remains in the verified local retrieval. Historical paths in passive
provenance are not runtime dependencies. Source license: source/LICENSE.

This public distribution adds comment-only change notices to six modified
upstream files and an Apache-2.0 notice to the original MELT-memory extension.
Runtime integrity constants and distribution receipts are rebound to those
files; all verification gates remain active. Model weights, tokenizer bytes
and inference algorithms are unchanged. Original training manifests and
source-integrity receipts are preserved outside this bundle under release/.
Model weights and project-added runtime are distributed under Apache-2.0,
with bundled third-party notices retained.