appvoid commited on
Commit
8f062cc
·
verified ·
1 Parent(s): a569196

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -34,8 +34,9 @@ State of the art, small language model pretrained from scratch on a diverse set
34
  | [tinctura-v1](https://huggingface.co/bench-labs/tinctura-v1) | 96.2M | 37.96% | 65.61% | 47.98% | 25.77% | 38.40% | 20.81 |
35
  | [Surjo-100m](https://huggingface.co/SurjoLabs/Surjo-100m) | 97.7M | 35.05% | 63.87% | 47.64% | 25.85% | 38.90% | 18.86 |
36
 
 
37
 
38
- We used the revision on step 900,000 for evaluations which trained for around 120 billion bytes which is around 30 to 35 billion bpe tokens. For more details on the evals and inference, please have a look at the [official notebook](https://huggingface.co/dotlabs/void.1/blob/main/evals.ipynb).
39
 
40
  ```bibtex
41
  @misc{dotlabs,
 
34
  | [tinctura-v1](https://huggingface.co/bench-labs/tinctura-v1) | 96.2M | 37.96% | 65.61% | 47.98% | 25.77% | 38.40% | 20.81 |
35
  | [Surjo-100m](https://huggingface.co/SurjoLabs/Surjo-100m) | 97.7M | 35.05% | 63.87% | 47.64% | 25.85% | 38.90% | 18.86 |
36
 
37
+ ## Details
38
 
39
+ We used the revision on step 900,000 for evaluations which trained for ~12 on an RTX PRO 6000. It trained on 131.69 billion bytes which is between 32.92B and 43.90B standard tokens, around ~38 bpe tokens on average. For more details on the evals and inference, please have a look at the [official notebook](https://huggingface.co/dotlabs/void.1/blob/main/evals.ipynb).
40
 
41
  ```bibtex
42
  @misc{dotlabs,