Hugging Face's logo

dotlabs
/
void.1

Text Generation
Transformers
TensorBoard
Safetensors
English
qwen3
byte-level
pretraining
symbolic
text-generation-inference
Model card Files
xet
 Metrics Community
3
appvoid commited on
Commit
572d5ed
·
verified ·
1 Parent(s): 42b8d23

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -9,10 +9,12 @@ tags: [qwen3, byte-level, pretraining]
9
 
10
  # details
11
 
 
 
12
  90,148,352 tied-embedding parameters.
13
 
14
  259 tokens: bytes 0–255, PAD 256, BOS 257, EOS 258.
15
 
16
- pretrained from scratch on cortex (our internal symbolic kernel), rewrite, climbmix, openmath and ultrafineweb.
17
 
18
- we used revision `92c601560cc2179e3bfc33e067fdd3bf9254e163` (step 900,000) for evaluations, for more details on the evals, please have a look at the official notebook.
 
9
 
10
  # details
11
 
12
+ state of the art byte-level language model pretrained from scratch on rewrite, climbmix, openmath, ultrafineweb and cortex (our internal symbolic kernel).
13
+
14
  90,148,352 tied-embedding parameters.
15
 
16
  259 tokens: bytes 0–255, PAD 256, BOS 257, EOS 258.
17
 
18
+ we used revision `92c601560cc2179e3bfc33e067fdd3bf9254e163` (step 900,000) for evaluations.
19
 
20
+ for more details on the evals, please have a look at the official notebook.