Transformers
pretraining

14M Random β€” uniform WSD v2

Controlled trigger-injection pretraining with the shared M/C/G stream. Models are trained from scratch; see the configuration and supervision guide.

Planned conditions: MCG/M/C/G Γ— 32/16/8/4/2/1 injections per billion tokens, with stable checkpoints and independent 763-update cooldowns. Full states retain weights, AdamW, scheduler, RNG and injection counters. There are no new constant-LR control branches. Checkpoint publication is incremental; repository existence does not imply completion.

uniform-wsd-v2/14m/<run>/eval512/ contains paired JSON and per-token NPZ scores for the frozen 1,536-example Random512 teacher-forced evaluation, without generation. Clean weights are reused from 294-288-pretrain-interaction/14M-fixed-v2 only after identity/parent checks. Imported baseline summaries identify the exact source revision and checkpoint hash. Shared-GPU pacing is recorded in checkpoint execution metadata.

analysis/results.json and analysis/primary.csv are periodically updated with raw and available matched-Clean results. Missing Clean matches remain null. Different execution layouts are reported explicitly.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support