Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior
Paper • 2609.39827 • Published • 12
This repository contains a 3B parameter causal language model based on the SmolLM3 architecture, presented in the paper Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior.
For more information, see the project page and the GitHub repository.