Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -53,6 +53,19 @@ batch 8 × grad_accum 4, on a single A100-PCIE-40GB.
|
|
| 53 |
- The M2 line's verdict so far is honest-null on reasoning mechanisms; this
|
| 54 |
checkpoint is the language-line scaling artifact, not a capability claim.
|
| 55 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
## Config
|
| 57 |
|
| 58 |
`config.json` carries the full MTLNNConfig (d_model 2080, 34 layers, 16 heads,
|
|
|
|
| 53 |
- The M2 line's verdict so far is honest-null on reasoning mechanisms; this
|
| 54 |
checkpoint is the language-line scaling artifact, not a capability claim.
|
| 55 |
|
| 56 |
+
## SFT checkpoints (post-training experiments, 2026-10)
|
| 57 |
+
|
| 58 |
+
| file | recipe | val PPL | JevBench public (official harness) |
|
| 59 |
+
|---|---|---|---|
|
| 60 |
+
| `t2b_sft_v1.pt` | alpaca-short 5.4k pairs (<=128B), 3 epochs, lr 1e-4 | 3.95 | - |
|
| 61 |
+
| `t2b_sft_v2.pt` | alpaca-short512 19.6k pairs (<=512B), 1+5k steps, lr 2e-5, rope_scale 4 | **3.20** | raw 31.3% (48 easy + 72 original decisions) |
|
| 62 |
+
|
| 63 |
+
Boundaries: instruction-SFT on **short** pairs; held-out generation went from
|
| 64 |
+
0/8 (base) to 4/8 format-correct, but decision-task accuracy remains the open
|
| 65 |
+
problem (best family configuration: 2B frozen core + decision head, 39.6%
|
| 66 |
+
easy-tier raw / ECE 0.20 / 0.168 s - see github.com/AwareLiquid/O1-Flash).
|
| 67 |
+
Serving interface for all of the above: byte-level, `max_seq_len` per config.
|
| 68 |
+
|
| 69 |
## Config
|
| 70 |
|
| 71 |
`config.json` carries the full MTLNNConfig (d_model 2080, 34 layers, 16 heads,
|