metadata
library_name: transformers
pipeline_tag: text-generation
tags:
- qwen3
- byte-level
- pretraining
- symbolic
license: apache-2.0
datasets:
- karpathy/climbmix-400b-shuffle
- openbmb/Ultra-FineWeb-L3
- dotlabs/rewrite
- nvidia/OpenMathInstruct-2
language:
- en
👋 Meet void.1
State of the art, small language model pretrained from scratch on a diverse set of high-quality texts and an internal symbolic kernel. This is the first step into a series of models designed for fine-grained understanding of abstract/symbolic reasoning on text while being grounded on english.
Comparison
| Model | Params | HellaSwag | PIQA | ARC-Easy | ARC-Challenge | ArithMark-3 | Intelligence Index |
|---|---|---|---|---|---|---|---|
| void.1* | 90.15M | 38.68% | 67.46% | 47.31% | 28.16% | 44.80% | 23.92 |
| 100M-exp | 98.16M | 37.78% | 66.97% | 49.83% | 27.22% | 40.00% | 22.47 |
| Rose-1.5-Medium | 98.28M | 38.09% | 64.80% | 47.22% | 27.13% | 40.70% | 21.07 |
| tinctura-v1 | 96.2M | 37.96% | 65.61% | 47.98% | 25.77% | 38.40% | 20.81 |
| Surjo-100m | 97.7M | 35.05% | 63.87% | 47.64% | 25.85% | 38.90% | 18.86 |
We used the revision on step 900,000 for evaluations which trained for around 120 billion bytes which is around 30 to 35 billion bpe tokens. For more details on the evals and inference, please have a look at the official notebook.
@misc{dotlabs,
title = {void: bytes is all you need},
author = {appvoid},
year = {2026},
url = {https://huggingface.co/dotlabs/void.1}
}