youssK
/

Boris-2 / RECIPE.md
youssK's picture KlondikeDev's picture
Duplicate from opencerebral/Boris-2
c5cfc0e
|
Raw History Blame Contribute Delete
772 Bytes

Boris-2 release candidate (2026-11-11)

Official-style Open SLM Index 34.19 (HellaSwag 50.26, ARC-E 59.81, ARC-C 32.00, PIQA 72.91, ArithMark-3 official 45.10). Lineage:

  1. Boris-2-0929 final.pt (200B tokens, WSD 1-sqrt decay, doubled batch from 170B).
  2. "Nudge": 30M tokens, mix climbmix .55 / cosmo2_how .30 / cosmo_how_v1_tf .05 / ultra_math_l3 .07 / metamath_gsm .02 / orca_math .01 (per-doc draw weights), LR 0.1x peak, 1-sqrt to 0, seed 1234 (probe/pilot_train.py --peak 0.1).
  3. 10M tokens of LibreMath v1 only (decontaminated: 13-gram + 8-gram ArithMark/SVAMP/ASDiv bans), LR 0.02x peak, 1-sqrt to 0, fresh optimizer state (probe/pilot_train.py --peak 0.02). Selected from ~10 short final-stage candidates by benchmark score -- disclose in the model card.