File size: 2,443 Bytes
9f0e0ef cf6ff9f 7ea7977 cf6ff9f 0082c28 cf6ff9f 9f0e0ef f7534c8 71a7bbd a7afe5b 79d2de8 f7534c8 939103d 9015a72 e4eef4a 9015a72 e4eef4a a569196 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
library_name: transformers
pipeline_tag: text-generation
tags:
- qwen3
- byte-level
- pretraining
- symbolic
license: apache-2.0
datasets:
- karpathy/climbmix-400b-shuffle
- openbmb/Ultra-FineWeb-L3
- dotlabs/rewrite
- nvidia/OpenMathInstruct-2
language:
- en
---
> [#1 on sub-100m on the open slm leaderboard!](https://huggingface.co/spaces/AxiomicLabs/Open_SLM_Leaderboard)
# 👋 Meet void.1
State of the art, small language model pretrained from scratch on a diverse set of high-quality texts and an internal symbolic kernel. This is the first step into a series of models designed for fine-grained understanding of abstract/symbolic reasoning on text while being grounded on english.
<video src="https://huggingface.co/dotlabs/void.1/resolve/main/teaser.mp4" autoplay loop width="100%"></video>
## Comparison
| Model | Params | HellaSwag | PIQA | ARC-Easy | ARC-Challenge | ArithMark-3 | Intelligence Index |
| ----------------------------------------------------------------- | ------ | --------- | ------ | -------- | ------------- | ----------- | ------------------ |
| [void.1](https://huggingface.co/dotlabs/void.1)\* | **90.15M** | **38.68%** | **67.46%** | 47.31% | **28.16%** | **44.80%** | **23.92** |
| [100M-exp](https://huggingface.co/User01110/100M-exp) | 98.16M | 37.78% | 66.97% | **49.83%** | 27.22% | 40.00% | 22.47 |
| [Rose-1.5-Medium](https://huggingface.co/GODELEV/Rose-1.5-Medium) | 98.28M | 38.09% | 64.80% | 47.22% | 27.13% | 40.70% | 21.07 |
| [tinctura-v1](https://huggingface.co/bench-labs/tinctura-v1) | 96.2M | 37.96% | 65.61% | 47.98% | 25.77% | 38.40% | 20.81 |
| [Surjo-100m](https://huggingface.co/SurjoLabs/Surjo-100m) | 97.7M | 35.05% | 63.87% | 47.64% | 25.85% | 38.90% | 18.86 |
We used the revision on step 900,000 for evaluations which trained for around 120 billion bytes which is around 30 to 35 billion bpe tokens. For more details on the evals and inference, please have a look at the [official notebook](https://huggingface.co/dotlabs/void.1/blob/main/evals.ipynb).
```bibtex
@misc{dotlabs,
title = {void: bytes is all you need},
author = {appvoid},
year = {2026},
url = {https://huggingface.co/dotlabs/void.1}
} |