OxMini v2 validation record
| Requirement | Measured evidence |
|---|---|
| Distinct v2 architecture | configs/oxmini_v2.yaml: 8 blocks, 144 width, 6 heads, 320-character context, 6 KDA-lite + 2 MLA-lite layers, 4 mHC streams with Sinkhorn routing. |
| Compact parameter budget | 2,752,568 parameters after fitting the 117-character vocabulary. |
| Broader pretraining | TinyStories revision pinned; 3,000,000 characters; zero unknown tokens; corpus hash recorded in V2_RESULTS.md. |
| Improved optimizer path | Native/accumulated effective-batch support, warm-up + cosine decay, finite-loss gates, gradient clipping, best/final checkpoints. |
| Stable long contexts | KDA state has a per-head Frobenius-norm bound. Exact formerly overflowing 320-character SFT cases are finite and covered by regression tests. |
| Post-training | Response-only SFT masks prompts/padding and mixes a 0.5-weight TinyStories replay loss to reduce forgetting. |
| Held-out post-training gain | Response CE 2.5387 -> 2.0018; PPL 12.6628 -> 7.4027; character accuracy 35.19% -> 45.57%. |
| Retention decision | Replay-SFT TinyStories PPL 3.6272 versus 5.4063 for pure SFT and 3.2908 for base; replay checkpoint selected. |
| Cache correctness | Cached and full-forward logits match in tests; 100-character greedy outputs match exactly. |
| Cache performance | 7.59x measured speedup on 100 greedy characters; evaluator samples around 200 chars/s. |
| Automated tests | pytest -q: 12 passed, including causality, cache equivalence, stability, SFT masking, serialization, and budgets. |
| Architecture image | assets/oxmini-v2-architecture.png, 1536x1024 PNG, visually inspected; SHA-256 9480f25ea0636ae2f0a23d525c8920a3cfa2d4fd3352b1a950595de81552b225. |
| Hugging Face publication | Public at Shivam3002/OxMini-v2, remote commit 84bf61184f215eb2d247651c5434df36c3c90af0. Anonymous forced download, exact weight/image hashes, disposable installation, and inference all passed. |
The recurring pynvml warning comes from the host PyTorch installation's CUDA
discovery path. All measured OxMini v2 training and evaluation commands used CPU.