The SLM Consortium has begun work on a safety dataset for Small Language Models, with the creation of the dataset being headed by @wayneworkman2012
The dataset will focus on refusals and redirects surrounding dangerous or extreme sexual content, designed to be shaped sized appropriately for SLMs, without significantly lowering benchmark performance.
Hello! Tomorrow, OpenCerebral will be releasing Boris-1.7-D60M-n30M — an experimental architecture. It will be testing a new data mixture, a new tokenizer, and testing Qwen4-like n-gram embeddings.
Following this will be Boris-1.8-D60M-n30M, which will test both the n-gram embeddings AND a new architecture.
The Small Language Model Consortium has reached 10 members!
Anybody is welcome to join, via two methods:
1. Starting a community discussion on the group’s README 2. Receiving an invite from me: @KlondikeDev
The Discord Community is also live! Feel free to join, even if you aren’t in the consortium, to discuss Small Language Models, or even just AI in general!
Boris-2-75M: Trained on 26B tokens -- estimated to start training on August 12th. Boris-2-125M: Trained on 90B tokens -- Estimated to start training on August 20th. Boris-2-250M: Trained on 60B tokens -- Estimated to start training on September 10th.
Why does 125M get more tokens than 250M?
Well, the straight answer is time. It saves time, while still allowing the 250M model to exceed the 125M model.
Furthermore, we are attempting a unique architecture and layering scheme to hopefully end up around the strength of SmolLM2-135M. Fingers crossed!