| # OxMini v2 validation record |
|
|
| | Requirement | Measured evidence | |
| |---|---| |
| | Distinct v2 architecture | `configs/oxmini_v2.yaml`: 8 blocks, 144 width, 6 heads, 320-character context, 6 KDA-lite + 2 MLA-lite layers, 4 mHC streams with Sinkhorn routing. | |
| | Compact parameter budget | 2,752,568 parameters after fitting the 117-character vocabulary. | |
| | Broader pretraining | TinyStories revision pinned; 3,000,000 characters; zero unknown tokens; corpus hash recorded in `V2_RESULTS.md`. | |
| | Improved optimizer path | Native/accumulated effective-batch support, warm-up + cosine decay, finite-loss gates, gradient clipping, best/final checkpoints. | |
| | Stable long contexts | KDA state has a per-head Frobenius-norm bound. Exact formerly overflowing 320-character SFT cases are finite and covered by regression tests. | |
| | Post-training | Response-only SFT masks prompts/padding and mixes a 0.5-weight TinyStories replay loss to reduce forgetting. | |
| | Held-out post-training gain | Response CE 2.5387 -> 2.0018; PPL 12.6628 -> 7.4027; character accuracy 35.19% -> 45.57%. | |
| | Retention decision | Replay-SFT TinyStories PPL 3.6272 versus 5.4063 for pure SFT and 3.2908 for base; replay checkpoint selected. | |
| | Cache correctness | Cached and full-forward logits match in tests; 100-character greedy outputs match exactly. | |
| | Cache performance | 7.59x measured speedup on 100 greedy characters; evaluator samples around 200 chars/s. | |
| | Automated tests | `pytest -q`: 12 passed, including causality, cache equivalence, stability, SFT masking, serialization, and budgets. | |
| | Architecture image | `assets/oxmini-v2-architecture.png`, 1536x1024 PNG, visually inspected; SHA-256 `9480f25ea0636ae2f0a23d525c8920a3cfa2d4fd3352b1a950595de81552b225`. | |
| | Hugging Face publication | Public at [`Shivam3002/OxMini-v2`](https://huggingface.co/Shivam3002/OxMini-v2), remote commit `84bf61184f215eb2d247651c5434df36c3c90af0`. Anonymous forced download, exact weight/image hashes, disposable installation, and inference all passed. | |
|
|
| The recurring `pynvml` warning comes from the host PyTorch installation's CUDA |
| discovery path. All measured OxMini v2 training and evaluation commands used CPU. |
|
|
