RetNet-0.5B - from-scratch training line
Collection
From-scratch 0.5B RetNet (21.6 B tokens): epoch 1-3, repair v21.1, long-context v21.2, chat v21.3-chat, 16k/8k research exports. โข 16 items โข Updated
Pre-0.5B generations, archived from the local workspace before deletion.
Each file is a raw torch.save state dict (or a safetensors export for 100m_chat);
they are not from_pretrained-ready repositories, only weight dumps.
100m_chat/model.safetensors - 100M chat fine-tune97m_wikitext103/model.pt - 97M wikitext103 baseline23m/best_model.pt - 23M smoke-test model (may contain optimizer state)