Pretrained on 4x more tokens than the previous releases (20b vs 5b). Instruct tuned versions are coming soon. Very interesting models are coming soon too (hint: super long context).
Just hit #14 and #15 with out FIRST models on Open SLM Leaderboard. The models were trained on 5B tokens, while competing with similarly sized models trained on more than 6-20x the data.
A new base model Speck1.5-140M being trained right now on a higher quality corpus and will be released soon. SpeckChat3 is coming very soon with 1 million samples, specifically designed to post train small base models.
Also, just to clarify stuff, we will NOT release anything that is NOT MIT licensed EVER. Openness is needed in small language research.
Thanks to everyone supporting the project, and stay tuned for new releases!
SPECK UPDATES: 1 New instruct model tuned on top of Speck1-140M: specklabs/Speck1-140M-Instruct 2 Instruction tuning datasets 2 GGUFs
Much more coming soon: Speck1.1-140M-Instruct that is post trained on SpeckChat2 will be coming very soon New base model Speck1.5-140M is coming with a much higher quality corpus
Thanks to everyone who is already supporting the project, and stay tuned for new releases!
If an agent can build the obvious demo, the obvious demo probably isnโt worth building anymore. For years, turning a research repo into something people could actually try was valuable by itself. That part is becoming automated โ and thatโs a good thing.
Which means the interesting work moves elsewhere: finding the weird use case, the right interaction, the unexpected model combination โ or simply knowing which paper is worth anyoneโs attention.
The demo used to be the product. Now it needs a point of view.
new models coming very soon (both instruct and much better models), with much much higher training scale as i am getting marenostrum5 access soon! we will be looking at 100b-2t token budgets :)
So far I've been pointing it at Markdown Minimap, an Obsidian plugin that adds a scrollable IDE-style minimap to your notes. This week I've been clearing a backlog of user-reported issues on it, with Claude often handling them end to end.
Introducing Inflect-v2, two exceptionally small, open-weight English TTS models at just 3.9M and 9.3M parameters. Both generate speech multiple times faster than real-time on CPU. Despite their size, Inflect-v2 delivers quality that is competitive with much larger lightweight TTS systems, including KittenTTS, Piper, and Supertonic-3.
CPU, CUDA, PyTorch, and ONNX are supported. Apache 2.0.