Text Generation
Transformers
Safetensors
English
wildai
nanochat
scaling-laws
ai-generated-text
pretraining-data
Instructions to use pangram/WildAI-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use pangram/WildAI-models with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="pangram/WildAI-models")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("pangram/WildAI-models", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use pangram/WildAI-models with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "pangram/WildAI-models" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pangram/WildAI-models", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/pangram/WildAI-models
- SGLang
How to use pangram/WildAI-models with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "pangram/WildAI-models" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pangram/WildAI-models", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "pangram/WildAI-models" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "pangram/WildAI-models", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use pangram/WildAI-models with Docker Model Runner:
docker model run hf.co/pangram/WildAI-models
Model cards: cite russell2026wildai; root tokenizer (part 4)
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- 86.2m-g041-ai-r4.22/README.md +2 -2
- 86.2m-g041-ai-r8/README.md +2 -2
- 86.2m-g041-control/README.md +2 -2
- 86.2m-g041-human-r2/README.md +2 -2
- 86.2m-g042-ai-r1/README.md +2 -2
- 86.2m-g042-control/README.md +2 -2
- 86.2m-g043-ai-r1/README.md +2 -2
- 86.2m-g043-control/README.md +2 -2
- 86.2m-g044-ai-r0.025/README.md +2 -2
- 86.2m-g044-ai-r0.05/README.md +2 -2
- 86.2m-g044-ai-r0.1/README.md +2 -2
- 86.2m-g044-ai-r0.25/README.md +2 -2
- 86.2m-g044-ai-r0.5/README.md +2 -2
- 86.2m-g044-ai-r1/README.md +2 -2
- 86.2m-g044-ai-r2/README.md +2 -2
- 86.2m-g044-ai-r4/README.md +2 -2
- 86.2m-g044-ai-r8/README.md +2 -2
- 86.2m-g044-control/README.md +2 -2
- 86.2m-g044-human-r0.025/README.md +2 -2
- 86.2m-g044-human-r0.05/README.md +2 -2
- 86.2m-g044-human-r2/README.md +2 -2
- 86.2m-g045-ai-r0.05/README.md +2 -2
- 86.2m-g045-ai-r0.1/README.md +2 -2
- 86.2m-g045-ai-r0.25/README.md +2 -2
- 86.2m-g045-ai-r0.5/README.md +2 -2
- 86.2m-g045-ai-r1/README.md +2 -2
- 86.2m-g045-control/README.md +2 -2
- 86.2m-g046-ai-r0.05/README.md +2 -2
- 86.2m-g046-ai-r0.1/README.md +2 -2
- 86.2m-g046-ai-r0.25/README.md +2 -2
- 86.2m-g046-ai-r0.5/README.md +2 -2
- 86.2m-g046-ai-r1/README.md +2 -2
- 86.2m-g046-control/README.md +2 -2
- 86.2m-g047-ai-r0.05/README.md +2 -2
- 86.2m-g047-ai-r0.1/README.md +2 -2
- 86.2m-g047-ai-r0.25/README.md +2 -2
- 86.2m-g047-ai-r0.5/README.md +2 -2
- 86.2m-g047-ai-r1/README.md +2 -2
- 86.2m-g047-ai-r2.07/README.md +2 -2
- 86.2m-g047-ai-r4.2/README.md +2 -2
- 86.2m-g047-control/README.md +2 -2
- 86.2m-g048-ai-r0.25/README.md +2 -2
- 86.2m-g048-ai-r1/README.md +2 -2
- 86.2m-g048-control/README.md +2 -2
- 86.2m-g049-ai-r0.05/README.md +2 -2
- 86.2m-g049-ai-r0.1/README.md +2 -2
- 86.2m-g049-ai-r0.25/README.md +2 -2
- 86.2m-g049-ai-r0.5/README.md +2 -2
- 86.2m-g049-ai-r1/README.md +2 -2
- 86.2m-g049-ai-r2/README.md +2 -2
86.2m-g041-ai-r4.22/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g041-ai-r8/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g041-control/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g041-human-r2/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g042-ai-r1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g042-control/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g043-ai-r1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g043-control/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r0.025/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r0.05/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r0.1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r0.25/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r0.5/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r2/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r4/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-ai-r8/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-control/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-human-r0.025/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-human-r0.05/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g044-human-r2/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g045-ai-r0.05/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g045-ai-r0.1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g045-ai-r0.25/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g045-ai-r0.5/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g045-ai-r1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g045-control/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g046-ai-r0.05/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g046-ai-r0.1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g046-ai-r0.25/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g046-ai-r0.5/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g046-ai-r1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g046-control/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g047-ai-r0.05/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g047-ai-r0.1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g047-ai-r0.25/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g047-ai-r0.5/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g047-ai-r1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g047-ai-r2.07/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g047-ai-r4.2/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g047-control/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g048-ai-r0.25/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g048-ai-r1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g048-control/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g049-ai-r0.05/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g049-ai-r0.1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g049-ai-r0.25/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g049-ai-r0.5/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g049-ai-r1/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
86.2m-g049-ai-r2/README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
-
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
@@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
-
@article{
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|
|
|
|
| 18 |
|
| 19 |
> **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license.
|
| 20 |
|
| 21 |
+
A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training.
|
| 22 |
|
| 23 |
## Training data
|
| 24 |
|
|
|
|
| 95 |
## Citation
|
| 96 |
|
| 97 |
```bibtex
|
| 98 |
+
@article{russell2026wildai,
|
| 99 |
title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text},
|
| 100 |
author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley},
|
| 101 |
journal = {arXiv preprint arXiv:2609.40295},
|