diff --git a/35.8m-f07-ai-removed/README.md b/35.8m-f07-ai-removed/README.md index 4c3d0a5bae39d69db31b8b1c9cc16bdc2862768a..ddb70f48f22215ae7ed846bbaad39098563fb454 100644 --- a/35.8m-f07-ai-removed/README.md +++ b/35.8m-f07-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-f07-web-mix/README.md b/35.8m-f07-web-mix/README.md index 4954ca5bd8b973489df033e63a885bedac399e44..4c5ee4412ee8b37a1c45cadcfdd2cf498da27108 100644 --- a/35.8m-f07-web-mix/README.md +++ b/35.8m-f07-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-f08-ai-removed/README.md b/35.8m-f08-ai-removed/README.md index 49dfabc612888220f711d5cd2223c7018fc37e3d..7afe86b299bd137e031a73e745aad34d997ccacb 100644 --- a/35.8m-f08-ai-removed/README.md +++ b/35.8m-f08-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-f08-web-mix/README.md b/35.8m-f08-web-mix/README.md index 40f1323a96f84e1bff2608bf62dfc81113f8fee8..a36a73a8add0f37526575e6497022498fe178a63 100644 --- a/35.8m-f08-web-mix/README.md +++ b/35.8m-f08-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-f09-ai-removed/README.md b/35.8m-f09-ai-removed/README.md index 2c5a0c599e133f50bc62125b986197ae8b7b450c..fc613d7ce5a74300106b2667e0e47468e735d014 100644 --- a/35.8m-f09-ai-removed/README.md +++ b/35.8m-f09-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-f09-web-mix/README.md b/35.8m-f09-web-mix/README.md index 434010bef5893cbf735c739aab15a456a28111ad..ccdba903e982b30a49ac55648934c408d492eec7 100644 --- a/35.8m-f09-web-mix/README.md +++ b/35.8m-f09-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-f10-ai-removed/README.md b/35.8m-f10-ai-removed/README.md index 03494761637fbbe37f606ae24b5fb38608b867fd..d4f1ed6f915c2f774682f0b5136f7da2f108c898 100644 --- a/35.8m-f10-ai-removed/README.md +++ b/35.8m-f10-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-f10-web-mix/README.md b/35.8m-f10-web-mix/README.md index 246d9f903836ce2fcb6fa944ed53265f064fe642..7a41c76062a5372a6b3e599d242043f43a90b04a 100644 --- a/35.8m-f10-web-mix/README.md +++ b/35.8m-f10-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g021-ai-r0.05/README.md b/35.8m-g021-ai-r0.05/README.md index b14e7793253f72dcc82e63f8e541a8d0abc527ed..5f0d3bc62a730d2adb093f50aa2505f761bcd285 100644 --- a/35.8m-g021-ai-r0.05/README.md +++ b/35.8m-g021-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g021-ai-r0.1/README.md b/35.8m-g021-ai-r0.1/README.md index ee3258a585676f8198891db2944f475c282d2e98..61590eb51b8d8fc979f64e11a96002347d1c9eda 100644 --- a/35.8m-g021-ai-r0.1/README.md +++ b/35.8m-g021-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g021-ai-r0.25/README.md b/35.8m-g021-ai-r0.25/README.md index a121defaa01e9a759a56db2ca2532330b23431ac..81ea491690f864bb30b80b85dce0a11823c25c70 100644 --- a/35.8m-g021-ai-r0.25/README.md +++ b/35.8m-g021-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g021-ai-r0.5/README.md b/35.8m-g021-ai-r0.5/README.md index 5ef78290b5371140f1a07b04ba931e30bfc7dda3..3dd32d4744dd44ffd47bf0f263aea78f7ddbc973 100644 --- a/35.8m-g021-ai-r0.5/README.md +++ b/35.8m-g021-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g021-ai-r1/README.md b/35.8m-g021-ai-r1/README.md index 10c4c5e706157385fa805e1f1dcc018e4e3dc014..ce62cdc12ddb8ef03d6ee789ded7db4adb319a46 100644 --- a/35.8m-g021-ai-r1/README.md +++ b/35.8m-g021-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g021-control/README.md b/35.8m-g021-control/README.md index c912266e0de9acde94c1654ef0dc3aa3a72cf05b..73cce5e88e6bf258c1b564ea923e50952a667718 100644 --- a/35.8m-g021-control/README.md +++ b/35.8m-g021-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-ai-r0.0234/README.md b/35.8m-g022-ai-r0.0234/README.md index e5febb13ff50e362d4266d580de2fa3e4b2a209a..b463ed5a1aa5c58eb1c2e3c85584aedb608540c3 100644 --- a/35.8m-g022-ai-r0.0234/README.md +++ b/35.8m-g022-ai-r0.0234/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-ai-r0.0465/README.md b/35.8m-g022-ai-r0.0465/README.md index ddd5a9a0df2139203cb1308b01469f2cf48d9083..cc1bb1776fffdb1116b4331888ab0a2ef9cd8262 100644 --- a/35.8m-g022-ai-r0.0465/README.md +++ b/35.8m-g022-ai-r0.0465/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-ai-r17.6/README.md b/35.8m-g022-ai-r17.6/README.md index 0bffff5bc8a8cb96e3d602784012b246873b3b2e..569b3777a5844166f79d2d6ce0dc4933ea994787 100644 --- a/35.8m-g022-ai-r17.6/README.md +++ b/35.8m-g022-ai-r17.6/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-ai-r2.1/README.md b/35.8m-g022-ai-r2.1/README.md index 8474c663f290441c5da3e41452774c06389c9e8c..5fe13e938338f349e8f70c36efcd61f177db4615 100644 --- a/35.8m-g022-ai-r2.1/README.md +++ b/35.8m-g022-ai-r2.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-ai-r35/README.md b/35.8m-g022-ai-r35/README.md index d72a5ce882ed7f1e810d12f055aa8cf1ba373a55..b89b55cb5994717ad91edf179eee6433da6dc051 100644 --- a/35.8m-g022-ai-r35/README.md +++ b/35.8m-g022-ai-r35/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-ai-r4.31/README.md b/35.8m-g022-ai-r4.31/README.md index d9e12c53bb729dc638cdc5f7179144a7a2254704..e25b03d682867a21e0d2e3dac1e5768e71f1b6ea 100644 --- a/35.8m-g022-ai-r4.31/README.md +++ b/35.8m-g022-ai-r4.31/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-ai-r69.6/README.md b/35.8m-g022-ai-r69.6/README.md index a338da2e32c946717e250c6457aa4c6fb3823c70..a294d12b21f3116c461054646a0ebf324be3a67f 100644 --- a/35.8m-g022-ai-r69.6/README.md +++ b/35.8m-g022-ai-r69.6/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-ai-r8.76/README.md b/35.8m-g022-ai-r8.76/README.md index b0d506b962cf22624d1cca37681109e4614e9bd7..c8c069ccfcb1ae1fddcc581532b92e5761f6369f 100644 --- a/35.8m-g022-ai-r8.76/README.md +++ b/35.8m-g022-ai-r8.76/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-control/README.md b/35.8m-g022-control/README.md index bd3dd9e416f26380725b4e609a1a8f133fdfca09..2912c1025ad5764626fdba89c0582c237678448b 100644 --- a/35.8m-g022-control/README.md +++ b/35.8m-g022-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-human-r0.025/README.md b/35.8m-g022-human-r0.025/README.md index 1ced4aa74bdca30583dcd9f4f02cbbfb8c2035aa..a268fb6609a46c24f85e1902d9774733c5feae5d 100644 --- a/35.8m-g022-human-r0.025/README.md +++ b/35.8m-g022-human-r0.025/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-human-r0.05/README.md b/35.8m-g022-human-r0.05/README.md index 545042d447ee36c5ae3696489752d9870107de6c..5355168e6840bceb0bdacb3a3f7616038959b5f0 100644 --- a/35.8m-g022-human-r0.05/README.md +++ b/35.8m-g022-human-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g022-human-r4/README.md b/35.8m-g022-human-r4/README.md index 4474cd33bc63d95eb421cc2b4dbba35e8010878d..b78249eb201113738d992a3c7329aba44c2ff816 100644 --- a/35.8m-g022-human-r4/README.md +++ b/35.8m-g022-human-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r0.025/README.md b/35.8m-g023-ai-r0.025/README.md index 6fbe248b4e1345ade4a50ccecde4b393ce4f05dc..d2dada6c7c5ad3f7adc4111d3ce4ee42be8233cb 100644 --- a/35.8m-g023-ai-r0.025/README.md +++ b/35.8m-g023-ai-r0.025/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r0.05/README.md b/35.8m-g023-ai-r0.05/README.md index 10a5d49ce0f3743f55519cb7321a631cbd4f5382..56c8ad644ddd0b781330a1ac08124f3423f1bf77 100644 --- a/35.8m-g023-ai-r0.05/README.md +++ b/35.8m-g023-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r0.1/README.md b/35.8m-g023-ai-r0.1/README.md index a6001e84673791a8cc0c74d2e89d30c6a6b20c46..c86ca71a8da727de7214aea558de61a8048c1670 100644 --- a/35.8m-g023-ai-r0.1/README.md +++ b/35.8m-g023-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r0.25/README.md b/35.8m-g023-ai-r0.25/README.md index 52f8f0cd06edaf0c1315816e06da1af25a5c3ae8..e676d93c45ba8627340f190f6e373a119c6768e0 100644 --- a/35.8m-g023-ai-r0.25/README.md +++ b/35.8m-g023-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r0.5/README.md b/35.8m-g023-ai-r0.5/README.md index 2126dfe4fb0ee43e3d2542275973d4912a75a046..011e0fe8bf35f317ddeb56b44c48a55147570919 100644 --- a/35.8m-g023-ai-r0.5/README.md +++ b/35.8m-g023-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r1/README.md b/35.8m-g023-ai-r1/README.md index f255c6e1239a2aa0722a0d226a8760da6533c594..2c9006ed7e1a297509cd4ef0125ba36b10850599 100644 --- a/35.8m-g023-ai-r1/README.md +++ b/35.8m-g023-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r16/README.md b/35.8m-g023-ai-r16/README.md index fe820b8615201c763cf8021b4bfbb129c143812e..248e71599214ccd3a440a4cd7914cfd56e85f0a1 100644 --- a/35.8m-g023-ai-r16/README.md +++ b/35.8m-g023-ai-r16/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r2/README.md b/35.8m-g023-ai-r2/README.md index 6577b915fa2fd5c25918338f429edfa4e7596c73..69237cdf7b66cd3e340c8131e098d70684c1180c 100644 --- a/35.8m-g023-ai-r2/README.md +++ b/35.8m-g023-ai-r2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r32/README.md b/35.8m-g023-ai-r32/README.md index 68618693725200fd602c219b58f0a091aef887e4..68f06b1a87cef1859c6fb7b8493615a8aa7cfd2d 100644 --- a/35.8m-g023-ai-r32/README.md +++ b/35.8m-g023-ai-r32/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r4/README.md b/35.8m-g023-ai-r4/README.md index d659a3bc2bf50f77f3a024e00f48646ae891b5a3..51d02bf6187d061287910bf8fbc43d88662b60e3 100644 --- a/35.8m-g023-ai-r4/README.md +++ b/35.8m-g023-ai-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r64/README.md b/35.8m-g023-ai-r64/README.md index d76a49b79cce789e5a283944136c66bc52fb068c..78a72d5aa11f82ae8607c9947c19e1f2d81231a8 100644 --- a/35.8m-g023-ai-r64/README.md +++ b/35.8m-g023-ai-r64/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-ai-r8/README.md b/35.8m-g023-ai-r8/README.md index bc02eea228d8769513a6c22765ee1d995780bfda..4c8fe08034a413a3f87f80a0f50ee087037ab10f 100644 --- a/35.8m-g023-ai-r8/README.md +++ b/35.8m-g023-ai-r8/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-control/README.md b/35.8m-g023-control/README.md index f52cd881ed9a0c30dd0c3705e86ec1821398d5c7..90d45b104d2b19033da8b652e3ea4466b8627065 100644 --- a/35.8m-g023-control/README.md +++ b/35.8m-g023-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-human-r0.025/README.md b/35.8m-g023-human-r0.025/README.md index decae01bd5f7a2c54c2866172807fb512fd9664d..2751e2cca71fc9f4478521024b2d96dbdcc966de 100644 --- a/35.8m-g023-human-r0.025/README.md +++ b/35.8m-g023-human-r0.025/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-human-r0.05/README.md b/35.8m-g023-human-r0.05/README.md index 10dd6cd1cd1054c42b639401c9694aea64889988..d68a36c4426f6a49d9624ea691413ea9b66e4a4e 100644 --- a/35.8m-g023-human-r0.05/README.md +++ b/35.8m-g023-human-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g023-human-r4/README.md b/35.8m-g023-human-r4/README.md index 5d9f9ea4b8cc68ec39842871b3edb8288a2b414a..ef41cdac176e526d453225c99dc52e76c3d90566 100644 --- a/35.8m-g023-human-r4/README.md +++ b/35.8m-g023-human-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g024-ai-r0.05/README.md b/35.8m-g024-ai-r0.05/README.md index 81a88a5b4213683df73bcc3ec688ba2ff242f2b4..132b6df65a974bbda254e2822a82758caf2eedd3 100644 --- a/35.8m-g024-ai-r0.05/README.md +++ b/35.8m-g024-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g024-ai-r0.1/README.md b/35.8m-g024-ai-r0.1/README.md index 5bc5ad5228a019f17b7bd8b4c9635aa906c0e219..f36dd5ec883aeb04eb42cfd6c98a64adf0d55f7c 100644 --- a/35.8m-g024-ai-r0.1/README.md +++ b/35.8m-g024-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g024-ai-r0.25/README.md b/35.8m-g024-ai-r0.25/README.md index 1feb28f97799d59867bf5554dd9f9eb37633c742..7ebe7247f33c5b9955e1e783c355cfbd70a795c7 100644 --- a/35.8m-g024-ai-r0.25/README.md +++ b/35.8m-g024-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g024-ai-r0.5/README.md b/35.8m-g024-ai-r0.5/README.md index 13b626d6df7ebf52649e2262def61b2894de73cf..21c02a74d8132f315a35e91d5006998760e5ce5a 100644 --- a/35.8m-g024-ai-r0.5/README.md +++ b/35.8m-g024-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g024-ai-r1/README.md b/35.8m-g024-ai-r1/README.md index 4625f6769f02001bb7847f3e47ad59ffd6bd69bb..a572f11dc41758f1731d301f97a78e15c98ce6f1 100644 --- a/35.8m-g024-ai-r1/README.md +++ b/35.8m-g024-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g024-control/README.md b/35.8m-g024-control/README.md index e5a94e4ad9a19408898f107ab9a39541f9c2afe2..1c6cd0eef501e57ca1f446db72e6d881317e7db8 100644 --- a/35.8m-g024-control/README.md +++ b/35.8m-g024-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-ai-r0.0256/README.md b/35.8m-g025-ai-r0.0256/README.md index 5129b99ab688c81a0452645e353637f12fdd5942..7847127890fd397cff1e3a522190ab0995f39815 100644 --- a/35.8m-g025-ai-r0.0256/README.md +++ b/35.8m-g025-ai-r0.0256/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-ai-r0.0514/README.md b/35.8m-g025-ai-r0.0514/README.md index 1f3fa6608e88a096fcdf1248c3cf80fccf0da84d..8431e3b01c295b5041cadc90e6377423f07fa0f5 100644 --- a/35.8m-g025-ai-r0.0514/README.md +++ b/35.8m-g025-ai-r0.0514/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-ai-r17.2/README.md b/35.8m-g025-ai-r17.2/README.md index e20c0d31e2dbb61d27ea56280cfcdcec6207c7c1..1763f549c2870f7af3e4737bf8e618f02ef86b61 100644 --- a/35.8m-g025-ai-r17.2/README.md +++ b/35.8m-g025-ai-r17.2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-ai-r2.11/README.md b/35.8m-g025-ai-r2.11/README.md index be7b97fbdf2c853790c6433f62b9400f6328af60..638245b014299ed4bcbc1bf0508053d5097845dc 100644 --- a/35.8m-g025-ai-r2.11/README.md +++ b/35.8m-g025-ai-r2.11/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-ai-r34.4/README.md b/35.8m-g025-ai-r34.4/README.md index 29d5dc63eb4740137f755e7bf670b94e611cb540..b5ad3d84dbed04e8752ff9a188b2667ad64d9a08 100644 --- a/35.8m-g025-ai-r34.4/README.md +++ b/35.8m-g025-ai-r34.4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-ai-r4.28/README.md b/35.8m-g025-ai-r4.28/README.md index dd6c139a7ee3d3222a67e7f95fafdd4422659506..94604076ee136760c05eab62c313b6832d65f914 100644 --- a/35.8m-g025-ai-r4.28/README.md +++ b/35.8m-g025-ai-r4.28/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-ai-r8.61/README.md b/35.8m-g025-ai-r8.61/README.md index ec444018ab45d502d0b9d04d254699efbb97d58a..44d5d230ed9e3c3041ffc1140b379faedc836d25 100644 --- a/35.8m-g025-ai-r8.61/README.md +++ b/35.8m-g025-ai-r8.61/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-control/README.md b/35.8m-g025-control/README.md index 5796d221698a68e702d917b0ffe54afcffa1e7af..e2e2dfc3ddf96cb73921a75d9030b828c66da89a 100644 --- a/35.8m-g025-control/README.md +++ b/35.8m-g025-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-human-r0.025/README.md b/35.8m-g025-human-r0.025/README.md index b67ed9b8ed618112b9f3caf00bf6ba971b64d806..88513c1a36325f65b49f10f238646939f4e69ace 100644 --- a/35.8m-g025-human-r0.025/README.md +++ b/35.8m-g025-human-r0.025/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-human-r0.05/README.md b/35.8m-g025-human-r0.05/README.md index 14d7db6f21dd537d12c4b102eac6e1c7f17f93b3..cd9a156433f8ad800e2edc642e922eff595071d5 100644 --- a/35.8m-g025-human-r0.05/README.md +++ b/35.8m-g025-human-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-human-r2/README.md b/35.8m-g025-human-r2/README.md index 4b3ac57d33a4d43564a4c9d58a6d865a040ed4bb..74fabc59d602e25f712a5e463a09ae21dd8c67af 100644 --- a/35.8m-g025-human-r2/README.md +++ b/35.8m-g025-human-r2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g025-human-r4/README.md b/35.8m-g025-human-r4/README.md index c2175665fa9a63d91eb65389bcb669a0a214451f..60bd84db6fc07b942080dac0a13460d465a9911b 100644 --- a/35.8m-g025-human-r4/README.md +++ b/35.8m-g025-human-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r0.025/README.md b/35.8m-g026-ai-r0.025/README.md index 665694a3e7f8b709e09537c1824df8c538a765ed..59a3464c2d9fbdea8cb39d9ddbbcb575f3d71e40 100644 --- a/35.8m-g026-ai-r0.025/README.md +++ b/35.8m-g026-ai-r0.025/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r0.05/README.md b/35.8m-g026-ai-r0.05/README.md index 9b9a0d22da6fb67e2a6ea05fdfe400d9078c2e4d..af48d19883f63388288144ae0b4d2a5050168278 100644 --- a/35.8m-g026-ai-r0.05/README.md +++ b/35.8m-g026-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r0.1/README.md b/35.8m-g026-ai-r0.1/README.md index 097b686b7de19b37f0cc372dbe2bfc7a6743ce84..990bc3191f0a950c965a96bcc48ac334038f44ae 100644 --- a/35.8m-g026-ai-r0.1/README.md +++ b/35.8m-g026-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r0.25/README.md b/35.8m-g026-ai-r0.25/README.md index 4dd803065493477457a87a06e326c2e5f3f2619c..46a49626311d097d699cc05a9b0f7afa3837b8b9 100644 --- a/35.8m-g026-ai-r0.25/README.md +++ b/35.8m-g026-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r0.5/README.md b/35.8m-g026-ai-r0.5/README.md index c4a29c761f0dfb82bd3a6a0dfd07b713fb94e5cd..658e439af6a13bcff8a238bb230c773e4d203c4d 100644 --- a/35.8m-g026-ai-r0.5/README.md +++ b/35.8m-g026-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r1/README.md b/35.8m-g026-ai-r1/README.md index 35955b25c979610b984c78be8e1a4df9c0445912..1919e0d7ddcf669be2d5e2b277b4a667d428cc65 100644 --- a/35.8m-g026-ai-r1/README.md +++ b/35.8m-g026-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r16/README.md b/35.8m-g026-ai-r16/README.md index d3ba7d9073118ee5602cebf9a0254cd381e9cf9a..3383f580124a3c3a53dfb8bf9544aa9b030fb294 100644 --- a/35.8m-g026-ai-r16/README.md +++ b/35.8m-g026-ai-r16/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r2/README.md b/35.8m-g026-ai-r2/README.md index fce680b426d4cd3e8fb5584d163b80dc62b89dd2..5ca88a79da89ab164edd93f6f66b27f5b0b2a6f3 100644 --- a/35.8m-g026-ai-r2/README.md +++ b/35.8m-g026-ai-r2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r32/README.md b/35.8m-g026-ai-r32/README.md index 9637ca77c60695d9d150afae4903e24ce908d2d7..f12e9e5c3fa0a91521072453c164433936dd38ed 100644 --- a/35.8m-g026-ai-r32/README.md +++ b/35.8m-g026-ai-r32/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r4/README.md b/35.8m-g026-ai-r4/README.md index 53e046d6205997236e13ce37e889aa9a5f1f9dee..2f679fbace613522d80d7808cf50f505e297d448 100644 --- a/35.8m-g026-ai-r4/README.md +++ b/35.8m-g026-ai-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-ai-r8/README.md b/35.8m-g026-ai-r8/README.md index a9c70a97914280f1ca26ed8145b08f56e2067fc2..c94f776f794cd8386c88788edf3aadbe86e511ec 100644 --- a/35.8m-g026-ai-r8/README.md +++ b/35.8m-g026-ai-r8/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-control/README.md b/35.8m-g026-control/README.md index 0882a9232ec6d41b565ff3bea9820e0697ea11bf..d454ef5c2f0b268676d95a6b855bffd61761c821 100644 --- a/35.8m-g026-control/README.md +++ b/35.8m-g026-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-human-r0.025/README.md b/35.8m-g026-human-r0.025/README.md index df22d1b9fdf085074a4cb03cbbb2f1c307aeaa0e..a91abe851e084d262b5081f0266c06965900817f 100644 --- a/35.8m-g026-human-r0.025/README.md +++ b/35.8m-g026-human-r0.025/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-human-r0.05/README.md b/35.8m-g026-human-r0.05/README.md index 221f03385303274cdc3754da838b396cba60d4ec..83c0051b2425455d8b32f44c2d756222b40992e1 100644 --- a/35.8m-g026-human-r0.05/README.md +++ b/35.8m-g026-human-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-human-r2/README.md b/35.8m-g026-human-r2/README.md index 8d271fbdc836b9e030e1c5d6f45d8b9c21e74704..a2da361a65625c9fbf714a10c8b4f8d7e742a276 100644 --- a/35.8m-g026-human-r2/README.md +++ b/35.8m-g026-human-r2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-human-r4/README.md b/35.8m-g026-human-r4/README.md index 0975d9c7b32fd2b88e35e366382ca86c9bc10c2d..11295a81521b02cae87a1b2738d76bb2cda13869 100644 --- a/35.8m-g026-human-r4/README.md +++ b/35.8m-g026-human-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-repeat-r1/README.md b/35.8m-g026-repeat-r1/README.md index 4c69ce61fbfd37a6aef83b316c33cdad243ff5c5..9cb4a9e8131c5b30133b4d4604f3f73b9e9c6ab3 100644 --- a/35.8m-g026-repeat-r1/README.md +++ b/35.8m-g026-repeat-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-repeat-r4/README.md b/35.8m-g026-repeat-r4/README.md index 76c87cacb865f686af0a7a0c67423c44395ff969..f7e636ca90e6a4a5d653d7d422f81b9a665b717b 100644 --- a/35.8m-g026-repeat-r4/README.md +++ b/35.8m-g026-repeat-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g026-repeat-r8/README.md b/35.8m-g026-repeat-r8/README.md index 98ad35032cde9eb1466d29a7e97d0bcdeaffd930..53039f89b02bac83e580700d8f6bab91a1177025 100644 --- a/35.8m-g026-repeat-r8/README.md +++ b/35.8m-g026-repeat-r8/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g027-ai-r0.05/README.md b/35.8m-g027-ai-r0.05/README.md index 8eab10bad0006fb8df39599ba9dc6144dc9ca266..495e661d1017b12e1c2839f7fe0d9d999cbb50be 100644 --- a/35.8m-g027-ai-r0.05/README.md +++ b/35.8m-g027-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g027-ai-r0.1/README.md b/35.8m-g027-ai-r0.1/README.md index cf853bfa663548636f823304706f47eaebd143cd..73ce045c1f5ed97a57faf77c8b2f81c6565d2876 100644 --- a/35.8m-g027-ai-r0.1/README.md +++ b/35.8m-g027-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g027-ai-r0.25/README.md b/35.8m-g027-ai-r0.25/README.md index febc9febf8c39abbd186b42dee8f4059db732091..0278ad4e97ce0c2804e155a4dee9f524d24ac09f 100644 --- a/35.8m-g027-ai-r0.25/README.md +++ b/35.8m-g027-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g027-ai-r0.5/README.md b/35.8m-g027-ai-r0.5/README.md index 544dbbaf1752674e49408446ac0012bea3e5b857..3f92b01e915c59cacc60df462c953644530cad08 100644 --- a/35.8m-g027-ai-r0.5/README.md +++ b/35.8m-g027-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g027-ai-r1/README.md b/35.8m-g027-ai-r1/README.md index 4a1bd13d99fe8b5d84e22c14e587d5f9fd7a5dff..cd576e125e03305f6b0673bf56ea41bab6b03105 100644 --- a/35.8m-g027-ai-r1/README.md +++ b/35.8m-g027-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g027-control/README.md b/35.8m-g027-control/README.md index 9753b196890edbba167afa3b1e455261399615ea..ed6568455fbb7a4d18f8638b7fcf9ff4a5ed460a 100644 --- a/35.8m-g027-control/README.md +++ b/35.8m-g027-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g028-ai-r0.05/README.md b/35.8m-g028-ai-r0.05/README.md index 85409c79c02b5506a062f126b33e4648cc4ac15d..1d0171c31e57b582cb06d2c944cfc5567e41a49a 100644 --- a/35.8m-g028-ai-r0.05/README.md +++ b/35.8m-g028-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g028-ai-r0.1/README.md b/35.8m-g028-ai-r0.1/README.md index b73fdd54b78d203daa884305f95481f30e6aa773..6922a86d9fc3a22ee09bf8e31355e84c07a896f8 100644 --- a/35.8m-g028-ai-r0.1/README.md +++ b/35.8m-g028-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g028-ai-r0.25/README.md b/35.8m-g028-ai-r0.25/README.md index e8cbbf656910658b31a985986ad48f7811bb00b2..57ca19cc560d88641c499f0403e52f63b7498c4f 100644 --- a/35.8m-g028-ai-r0.25/README.md +++ b/35.8m-g028-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g028-ai-r0.5/README.md b/35.8m-g028-ai-r0.5/README.md index 10c92802f6962ca64e47d67cde9e8b7d78521efc..7314b2fa5ec4afedd1e237a7277b42b9154421e0 100644 --- a/35.8m-g028-ai-r0.5/README.md +++ b/35.8m-g028-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g028-ai-r1/README.md b/35.8m-g028-ai-r1/README.md index d4b735403514499c5c3db68deae07da68d8dc79e..5ae8d27dec0fc0e2a6a9588db17e91e4c9d4d67c 100644 --- a/35.8m-g028-ai-r1/README.md +++ b/35.8m-g028-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g028-control/README.md b/35.8m-g028-control/README.md index 7be9f22c8f886fca1a928399814c25d3bcf1019c..add3e674fa76702d8f4051f2a48d7738592c086a 100644 --- a/35.8m-g028-control/README.md +++ b/35.8m-g028-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g029-ai-r0.05/README.md b/35.8m-g029-ai-r0.05/README.md index 499fcf3dbc8f3f0cfb60febcf158fac47aa5052a..8ac7630eb831d4ccfa76cefad8ee67bd457357db 100644 --- a/35.8m-g029-ai-r0.05/README.md +++ b/35.8m-g029-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g029-ai-r0.1/README.md b/35.8m-g029-ai-r0.1/README.md index f8c9ec48dc15230016d4626cf209d967bba07cc6..d15b4fcd0278ee4c6b0764260afe83c6df61f2e7 100644 --- a/35.8m-g029-ai-r0.1/README.md +++ b/35.8m-g029-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g029-ai-r0.25/README.md b/35.8m-g029-ai-r0.25/README.md index 514b7342eb519c7755b39034e4a7b1572871bbd0..15e293dfaae621ec811d9e3abfc0df82e0ce0105 100644 --- a/35.8m-g029-ai-r0.25/README.md +++ b/35.8m-g029-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g029-ai-r0.5/README.md b/35.8m-g029-ai-r0.5/README.md index bd5454763f443475f3b606103c072c23e582edc1..e20c4e9e50f02886f2e361bf171caaed0930b700 100644 --- a/35.8m-g029-ai-r0.5/README.md +++ b/35.8m-g029-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g029-ai-r1/README.md b/35.8m-g029-ai-r1/README.md index be59f9c54a70007a21c93deeef063370bc4d6fb9..4678f9a375b29c3e1aa55f60d1e9cbae3f4537e6 100644 --- a/35.8m-g029-ai-r1/README.md +++ b/35.8m-g029-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g029-control/README.md b/35.8m-g029-control/README.md index d324e6cb3145c35745fe43069a996acb200f9250..e9483b9f12da13569ac44d388f07f6d608a1733e 100644 --- a/35.8m-g029-control/README.md +++ b/35.8m-g029-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g030-ai-r2.08/README.md b/35.8m-g030-ai-r2.08/README.md index fcdcebdfbc22699b6126056c54a43c25fccb4697..27fa17946adc4eb58b8e3776c3f1f8297a0850d7 100644 --- a/35.8m-g030-ai-r2.08/README.md +++ b/35.8m-g030-ai-r2.08/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g030-ai-r4.21/README.md b/35.8m-g030-ai-r4.21/README.md index 284b96b9a95cc79424d26b0b0dc0b161bb4cf526..db1d0d1f0119a250f5c645139e09b2f9defb6584 100644 --- a/35.8m-g030-ai-r4.21/README.md +++ b/35.8m-g030-ai-r4.21/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g030-ai-r8.51/README.md b/35.8m-g030-ai-r8.51/README.md index a31aaeed5a68f72447a48261c8b9bd10903816d2..e900fa30eb10c04171dd8450ddcab06013a0bb01 100644 --- a/35.8m-g030-ai-r8.51/README.md +++ b/35.8m-g030-ai-r8.51/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g030-control/README.md b/35.8m-g030-control/README.md index 2e907f4e9a10884cce78024039300d5acab86d63..1b4ad222ec69c4cf9394468967ecb09da6295e2e 100644 --- a/35.8m-g030-control/README.md +++ b/35.8m-g030-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g030-human-r2/README.md b/35.8m-g030-human-r2/README.md index 43f758fca710e0ef49e2826b7fcca338829f3f32..0346964db00f66ffbc9815860c854b515063b70f 100644 --- a/35.8m-g030-human-r2/README.md +++ b/35.8m-g030-human-r2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g031-ai-r0.25/README.md b/35.8m-g031-ai-r0.25/README.md index ecf2e51a6a83dcbc06b00002f6bf9ed847f51361..0dfbcdd8c31731fd1f844ac0cd2dbf772c6a6b10 100644 --- a/35.8m-g031-ai-r0.25/README.md +++ b/35.8m-g031-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g031-ai-r1/README.md b/35.8m-g031-ai-r1/README.md index 26737689d7d559304ba0d0fe0e6f7355d4e0bb13..c9b46d814fa79b94776de9494082dc94cdfeb3b0 100644 --- a/35.8m-g031-ai-r1/README.md +++ b/35.8m-g031-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g031-control/README.md b/35.8m-g031-control/README.md index 2bd428f6c8ce701b2b88af8ebb50be7183ca109c..5ce94e06565c27cd254d4cc8653f32736af4bdca 100644 --- a/35.8m-g031-control/README.md +++ b/35.8m-g031-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g032-ai-r0.05/README.md b/35.8m-g032-ai-r0.05/README.md index b75faf191807bfdc66db9e57561262b130b6808a..a6756888d21b4b3b8585e76538a507c1c2379539 100644 --- a/35.8m-g032-ai-r0.05/README.md +++ b/35.8m-g032-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g032-ai-r0.1/README.md b/35.8m-g032-ai-r0.1/README.md index 9bbca698b2407cbdfaced4e9094ee4a95f319165..dd1c5a63b1c78d7bd40ca7a188bbb021206e5108 100644 --- a/35.8m-g032-ai-r0.1/README.md +++ b/35.8m-g032-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g032-ai-r0.25/README.md b/35.8m-g032-ai-r0.25/README.md index f3a086e54ed3a3015f986fde747f872e17d248b3..dd7a61ee70f78935d0bc8ac597c0edef197ecdaf 100644 --- a/35.8m-g032-ai-r0.25/README.md +++ b/35.8m-g032-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g032-ai-r0.5/README.md b/35.8m-g032-ai-r0.5/README.md index 099123e4f2a2ed4287c42cd7d678d2c403206dcc..e8b92818a80b6384e0d30d94f11ee19e78743c14 100644 --- a/35.8m-g032-ai-r0.5/README.md +++ b/35.8m-g032-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g032-ai-r1/README.md b/35.8m-g032-ai-r1/README.md index dddc4453644522996fb10681a287abb790bb81f9..6d99cc763e4a32000717481abeda72ed919bdeb2 100644 --- a/35.8m-g032-ai-r1/README.md +++ b/35.8m-g032-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g032-control/README.md b/35.8m-g032-control/README.md index ddf1b7bdb9d7d7d6be2a8929b663b9e6eafdbfe2..28c43c33dad9e6b266a3586d9e0035d8f7663e8b 100644 --- a/35.8m-g032-control/README.md +++ b/35.8m-g032-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-ai-r0.05/README.md b/35.8m-g033-ai-r0.05/README.md index 96daf252b5c53d7753bfbd7b9645471f59a6e14f..1c80f96a7a71ab3a7623b274b0f88dcd43a076fb 100644 --- a/35.8m-g033-ai-r0.05/README.md +++ b/35.8m-g033-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-ai-r0.1/README.md b/35.8m-g033-ai-r0.1/README.md index 9b23ad819807061f31abfb6505d40f13098894cd..034059b3732a56b86a83c4d21048bfbc1777b9b7 100644 --- a/35.8m-g033-ai-r0.1/README.md +++ b/35.8m-g033-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-ai-r0.25/README.md b/35.8m-g033-ai-r0.25/README.md index 421155e716ba4a7413e68c5be6fa938b4d28c22d..0c3c143a79c980197bbe3564339b728f94e18e5c 100644 --- a/35.8m-g033-ai-r0.25/README.md +++ b/35.8m-g033-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-ai-r0.5/README.md b/35.8m-g033-ai-r0.5/README.md index 434e3392c989f7a7d81d1be26f7b6ddbf3718e0b..b4354c6e2e4a23ae2606633210682a2ab44d44d1 100644 --- a/35.8m-g033-ai-r0.5/README.md +++ b/35.8m-g033-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-ai-r1/README.md b/35.8m-g033-ai-r1/README.md index c80ed7eae4ad8092486b342c79acc8573916809c..69f2113488da819df4a6173effe7c923a2b90b7c 100644 --- a/35.8m-g033-ai-r1/README.md +++ b/35.8m-g033-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-ai-r2/README.md b/35.8m-g033-ai-r2/README.md index 0f5a28fec78bd80bf07705188e0e4e2b42402903..928a43ba6dc768f93e26d3bcfafa49fcfce19e72 100644 --- a/35.8m-g033-ai-r2/README.md +++ b/35.8m-g033-ai-r2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-ai-r4/README.md b/35.8m-g033-ai-r4/README.md index 55e6f2a2008ced5b93f77066eb49f061a4bb0a16..0204da30020d17df9a07d00fe567b1d96b09b590 100644 --- a/35.8m-g033-ai-r4/README.md +++ b/35.8m-g033-ai-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-ai-r8/README.md b/35.8m-g033-ai-r8/README.md index 0d86f8df59a5d1ff3c49842f5983fcdd2f2ffc21..d736bf1fac0ed7fa27b10125ec69dc71c5e0784f 100644 --- a/35.8m-g033-ai-r8/README.md +++ b/35.8m-g033-ai-r8/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-control/README.md b/35.8m-g033-control/README.md index c892559b445a8f694d844bca8361db2fdc5258d0..8da6f11610099745db75236d5738b5c7de9bcfa6 100644 --- a/35.8m-g033-control/README.md +++ b/35.8m-g033-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g033-human-r2/README.md b/35.8m-g033-human-r2/README.md index ec6d8712751938bbee3e23d4201396bd9e360048..1a2fd9996e94554e3f719d67c5ccc5f8f261f8a1 100644 --- a/35.8m-g033-human-r2/README.md +++ b/35.8m-g033-human-r2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g034-ai-r0.05/README.md b/35.8m-g034-ai-r0.05/README.md index 9e281604471c21b5433e920b981267142d95f426..bad47fbd3c84d03f2d0f782be7df3cb78de4278a 100644 --- a/35.8m-g034-ai-r0.05/README.md +++ b/35.8m-g034-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g034-ai-r0.1/README.md b/35.8m-g034-ai-r0.1/README.md index 69e3035e0ea84151b641962a8ef378152c15c7b1..a8d7050a842a49215f8f11b2ce30ede342ba286a 100644 --- a/35.8m-g034-ai-r0.1/README.md +++ b/35.8m-g034-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g034-ai-r0.25/README.md b/35.8m-g034-ai-r0.25/README.md index 8199c506f9722da8bccd62fdc0aebe99202161f1..c97e077bfb00a16598f0d6922b6df4b0afc3af85 100644 --- a/35.8m-g034-ai-r0.25/README.md +++ b/35.8m-g034-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g034-ai-r0.5/README.md b/35.8m-g034-ai-r0.5/README.md index c092eb881ab26eb4666a8c1b3fa3e249b3563deb..b06baa3421e888b082dd691578473e88a25e6dce 100644 --- a/35.8m-g034-ai-r0.5/README.md +++ b/35.8m-g034-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g034-ai-r1/README.md b/35.8m-g034-ai-r1/README.md index cd28ea3ffeaeb2231f92a122ce750f9f7ca30aaa..bf1122a9283abcaf0854d52db2d248b1731c31fc 100644 --- a/35.8m-g034-ai-r1/README.md +++ b/35.8m-g034-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g034-control/README.md b/35.8m-g034-control/README.md index 2ccab966a717e1793e7c6895d8981161b3be6d46..0609281f9489d3e0dda5e9614620e7035f41eae0 100644 --- a/35.8m-g034-control/README.md +++ b/35.8m-g034-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g035-ai-r0.05/README.md b/35.8m-g035-ai-r0.05/README.md index 7438720b3b9be5ddf314b61fd14c199ebe0e1abf..ca81d660fc51e5dfd6789cf30277d1bf0b2deffa 100644 --- a/35.8m-g035-ai-r0.05/README.md +++ b/35.8m-g035-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g035-ai-r0.1/README.md b/35.8m-g035-ai-r0.1/README.md index d84bc6a16804214b48b93687f54a14899478c68e..9489b79a8b529f84235b6be76cc613517e701409 100644 --- a/35.8m-g035-ai-r0.1/README.md +++ b/35.8m-g035-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g035-ai-r0.25/README.md b/35.8m-g035-ai-r0.25/README.md index 3bb649268c630f78f9caea30da8b2a7a18d170bd..2ef5f33113823470741c58891142a0923f1584e7 100644 --- a/35.8m-g035-ai-r0.25/README.md +++ b/35.8m-g035-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g035-ai-r0.5/README.md b/35.8m-g035-ai-r0.5/README.md index b6950f40d7351fa37ea210d490077acc21a04904..63c341efdcdef16c6694ad8f384c51aa327a0a18 100644 --- a/35.8m-g035-ai-r0.5/README.md +++ b/35.8m-g035-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g035-ai-r1/README.md b/35.8m-g035-ai-r1/README.md index f85ee1f710c767e8162097b5b80d8107bc8ba2d2..c89f2e8f802f1a6a7d83e4ecf06362b514b9360d 100644 --- a/35.8m-g035-ai-r1/README.md +++ b/35.8m-g035-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g035-control/README.md b/35.8m-g035-control/README.md index 1330210c05b395a49937d20055bc9d7b50182ef9..123ffecb1f5dffe05866b86cee4b4733325e7be7 100644 --- a/35.8m-g035-control/README.md +++ b/35.8m-g035-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g036-ai-r0.05/README.md b/35.8m-g036-ai-r0.05/README.md index 6b46506e281ee7d023a8900f0e4f2abc788338d7..8da296f7410392b05bae4fb115f2d25158f12626 100644 --- a/35.8m-g036-ai-r0.05/README.md +++ b/35.8m-g036-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g036-ai-r0.1/README.md b/35.8m-g036-ai-r0.1/README.md index e910dcf30e10f17cabd4bd3f79ecce3258ab16e6..438bdbe1335008b324abe11a3b61c6c30777d253 100644 --- a/35.8m-g036-ai-r0.1/README.md +++ b/35.8m-g036-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g036-ai-r0.25/README.md b/35.8m-g036-ai-r0.25/README.md index b8d65a8594a5d9a64bc08d06aadac5ca30dbeaf0..6b576c56ac2e5bdba8dda4cbeb96225cbf1c72f7 100644 --- a/35.8m-g036-ai-r0.25/README.md +++ b/35.8m-g036-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g036-ai-r0.5/README.md b/35.8m-g036-ai-r0.5/README.md index d06f7ab9956c5ce4936961e2b4161658bfd409f5..e5f21af25472dba98f37c52827c05aa3576c64fe 100644 --- a/35.8m-g036-ai-r0.5/README.md +++ b/35.8m-g036-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g036-ai-r1/README.md b/35.8m-g036-ai-r1/README.md index b90e8ebbbc6d900f34e61a5a2605f028f7e05c92..f1046c89f46b30e15b1f6078f5aa2ac1c8385e72 100644 --- a/35.8m-g036-ai-r1/README.md +++ b/35.8m-g036-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/35.8m-g036-control/README.md b/35.8m-g036-control/README.md index b0778f7fe999ed78c37b6dd5698833521c4bf314..f8a0381972e04fef9104c07c47c273a7f433521b 100644 --- a/35.8m-g036-control/README.md +++ b/35.8m-g036-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 35.8M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f11-ai-removed/README.md b/86.2m-f11-ai-removed/README.md index f9a5feb55e6548f93b82aa81b84b2b7b1b5b08de..22b77b65c01aeb51562b1a43f829807018e8023e 100644 --- a/86.2m-f11-ai-removed/README.md +++ b/86.2m-f11-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f11-web-mix/README.md b/86.2m-f11-web-mix/README.md index 0046ab0057a9497f57f7c3531da158e24a0d6b8d..ef8b3f532300cd9ac8de03fcc8b5290183d33fc5 100644 --- a/86.2m-f11-web-mix/README.md +++ b/86.2m-f11-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f12-ai-removed/README.md b/86.2m-f12-ai-removed/README.md index 67c29e675e1911bf468e25e73bcf5b1321a45661..16cdd37ae4d00c085c1aa5860fcbd36d4d2b962e 100644 --- a/86.2m-f12-ai-removed/README.md +++ b/86.2m-f12-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f12-web-mix/README.md b/86.2m-f12-web-mix/README.md index 732436a4be8ee5092b0c7d865253104fadb79982..f9a6dcd7f6c80d3b82d980ac40473ac268ed5f1f 100644 --- a/86.2m-f12-web-mix/README.md +++ b/86.2m-f12-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f13-ai-removed/README.md b/86.2m-f13-ai-removed/README.md index 189543d00e2f373757078c4fb7274b32e91489f0..cef8cb7b9943c9850e3553760b9c5a21635805e1 100644 --- a/86.2m-f13-ai-removed/README.md +++ b/86.2m-f13-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f13-web-mix/README.md b/86.2m-f13-web-mix/README.md index f540496effac4aaea78cd1f9ff44cf5e785c60dd..6fed09b2624be1c15367454a269ead3d36b43dca 100644 --- a/86.2m-f13-web-mix/README.md +++ b/86.2m-f13-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f14-ai-removed/README.md b/86.2m-f14-ai-removed/README.md index b4b01455d7321a0f1fd788d32f81a62021d9b0a1..20d60ae60467a5dfe47394cce175ed2991dc1cec 100644 --- a/86.2m-f14-ai-removed/README.md +++ b/86.2m-f14-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f14-web-mix/README.md b/86.2m-f14-web-mix/README.md index 1fc499de3522a1d08946e83882c57e57d8a5fc8c..5673253cd86107e213e7e0440aaebec679bdf3a2 100644 --- a/86.2m-f14-web-mix/README.md +++ b/86.2m-f14-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f15-ai-removed/README.md b/86.2m-f15-ai-removed/README.md index 93b67f5d667fc13a1e9b36223c062b657d5799eb..57dcf98ecd4c2a7ac253afd61c372642bf6068f1 100644 --- a/86.2m-f15-ai-removed/README.md +++ b/86.2m-f15-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f15-web-mix/README.md b/86.2m-f15-web-mix/README.md index a5c6ce8a14d1546e7f5a5dff76547a8e6deab844..bba8996b1f0cd947be8975891ee19eb39f8e0202 100644 --- a/86.2m-f15-web-mix/README.md +++ b/86.2m-f15-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f16-ai-removed/README.md b/86.2m-f16-ai-removed/README.md index 88cf7dd145477ea38ff2cf40c4c7b062f4d541e1..9969fdfc8817229cf08ce51f55b0d6ccd440e9af 100644 --- a/86.2m-f16-ai-removed/README.md +++ b/86.2m-f16-ai-removed/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-f16-web-mix/README.md b/86.2m-f16-web-mix/README.md index 787dd6c72ed8d5413ac277be76e18ab2cd0f57a2..3ad6f7a2b218fcc7d417dec806e25186047635dc 100644 --- a/86.2m-f16-web-mix/README.md +++ b/86.2m-f16-web-mix/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -71,7 +71,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g037-ai-r0.05/README.md b/86.2m-g037-ai-r0.05/README.md index 328e9618cc034ef128c2db2a969f37ede30746e7..9b7f3da14695176f96e61a337212ef300e13134c 100644 --- a/86.2m-g037-ai-r0.05/README.md +++ b/86.2m-g037-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g037-ai-r0.1/README.md b/86.2m-g037-ai-r0.1/README.md index 7632afcf2fcbb9ea56775755b4fa3e533f5c49c5..3e1cfddc7d3324fd961a84301554ca4c70f8543b 100644 --- a/86.2m-g037-ai-r0.1/README.md +++ b/86.2m-g037-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g037-ai-r0.25/README.md b/86.2m-g037-ai-r0.25/README.md index b6c5b17813c612a7e0f23643d7c04db0340bef3b..01351eab3a409e0b316e7740ffe56e4104118758 100644 --- a/86.2m-g037-ai-r0.25/README.md +++ b/86.2m-g037-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g037-ai-r0.5/README.md b/86.2m-g037-ai-r0.5/README.md index 062a952015f9e540ef98209472d97f8961cd5c2a..3d45d2cd07c723b814746b7fe327cb8aa00e39a3 100644 --- a/86.2m-g037-ai-r0.5/README.md +++ b/86.2m-g037-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g037-ai-r1/README.md b/86.2m-g037-ai-r1/README.md index 255da8950edeace1df50e0badc3c2553399f9648..05fc7de9f7ab7b0358a1973519840d75920af226 100644 --- a/86.2m-g037-ai-r1/README.md +++ b/86.2m-g037-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g037-control/README.md b/86.2m-g037-control/README.md index f3f299f27c87a312dc7d7117c3370a41d4a05d80..9e0ce6898edf66625855454ca33b7b18be6070e8 100644 --- a/86.2m-g037-control/README.md +++ b/86.2m-g037-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g038-ai-r17.3/README.md b/86.2m-g038-ai-r17.3/README.md index 384b6ab38f3f81e684c816894a98496ca0eb9679..70065f5b9e476ae6def4171d58ca60d8bffb8120 100644 --- a/86.2m-g038-ai-r17.3/README.md +++ b/86.2m-g038-ai-r17.3/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g038-ai-r2.12/README.md b/86.2m-g038-ai-r2.12/README.md index 265c70363218eb345dbc84f7d1ca75fb698d6078..259d80fb885081e560ca826d31ad6de07ffb4a3a 100644 --- a/86.2m-g038-ai-r2.12/README.md +++ b/86.2m-g038-ai-r2.12/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g038-ai-r34.5/README.md b/86.2m-g038-ai-r34.5/README.md index 8d1a169efdc5cc2ac1857b6d8902b0ddbc9b30b7..4da3f36baef6442a3d65ba16dd0a1be3e0aefac4 100644 --- a/86.2m-g038-ai-r34.5/README.md +++ b/86.2m-g038-ai-r34.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g038-ai-r4.3/README.md b/86.2m-g038-ai-r4.3/README.md index 413277dd08e46e55bba475b06348c4e8e38bcea9..4808f865b3149b8a583c7af7eabac032666f8351 100644 --- a/86.2m-g038-ai-r4.3/README.md +++ b/86.2m-g038-ai-r4.3/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g038-ai-r8.66/README.md b/86.2m-g038-ai-r8.66/README.md index 7f37edc219f8492fb89f4ceb4d7688b4de832a8d..944e2c9b6e8de164ef1431dce22ca022a22b4a0b 100644 --- a/86.2m-g038-ai-r8.66/README.md +++ b/86.2m-g038-ai-r8.66/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g038-control/README.md b/86.2m-g038-control/README.md index ef0ae738473dca532a6347b83b3ec52f0a6e8cfe..11da51944ef1849e54e1f480e0436e0ff53289e1 100644 --- a/86.2m-g038-control/README.md +++ b/86.2m-g038-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g038-human-r4/README.md b/86.2m-g038-human-r4/README.md index 4aa64038a03c15ac7d102775f4fffb287e65a1d9..23918231821596498e96434c4ff3628cfdc58379 100644 --- a/86.2m-g038-human-r4/README.md +++ b/86.2m-g038-human-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r0.025/README.md b/86.2m-g039-ai-r0.025/README.md index 88c755ec51938ab2e7c2afa07982d889c8f0a2e9..51974e68ff18e8cec4e5c0b5cd6fd38291d0fd9c 100644 --- a/86.2m-g039-ai-r0.025/README.md +++ b/86.2m-g039-ai-r0.025/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r0.05/README.md b/86.2m-g039-ai-r0.05/README.md index 82a968087affd035a5113e7f294b1731f1c01eb7..ac4e9287dcd0ce9828edf1eec4121867da368d47 100644 --- a/86.2m-g039-ai-r0.05/README.md +++ b/86.2m-g039-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r0.1/README.md b/86.2m-g039-ai-r0.1/README.md index 3465d9210c7a29c879287c0d0f1011c453ea7692..1df501bf92aa84edf9812320feaba561277e3f85 100644 --- a/86.2m-g039-ai-r0.1/README.md +++ b/86.2m-g039-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r0.25/README.md b/86.2m-g039-ai-r0.25/README.md index de70eec93e3ffda9ef0d507675e6a6aa168be8e8..e832e1d319efb55d7db4318a3b71b56baf19276a 100644 --- a/86.2m-g039-ai-r0.25/README.md +++ b/86.2m-g039-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r0.5/README.md b/86.2m-g039-ai-r0.5/README.md index 0405f221ffd5f64f634364c566051eb40599f3e5..7deee396ddaca80d6e118320737171790f1bbf33 100644 --- a/86.2m-g039-ai-r0.5/README.md +++ b/86.2m-g039-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r1/README.md b/86.2m-g039-ai-r1/README.md index 8b3fe4d278df09b88f4788395273e42f0fae8327..76648564dd43b428eba4dd25c1b1e4f9b9a40115 100644 --- a/86.2m-g039-ai-r1/README.md +++ b/86.2m-g039-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r16/README.md b/86.2m-g039-ai-r16/README.md index a7c7e1547b5f64ac85c9b90dbcf65328992100a0..f80c21a1837aebf2aecf337cb71dc2485cbde7d7 100644 --- a/86.2m-g039-ai-r16/README.md +++ b/86.2m-g039-ai-r16/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r2/README.md b/86.2m-g039-ai-r2/README.md index 74ede2962046679dba57ef0894563a5c1e29ffd5..de2f8edf379e1d95bdbb36f4a3cabb2320069d4f 100644 --- a/86.2m-g039-ai-r2/README.md +++ b/86.2m-g039-ai-r2/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r32/README.md b/86.2m-g039-ai-r32/README.md index ad817417119f96be7ffedd40df0aac69cf0eaf22..a49e2d0eba7e5923ba6cae78d20a798ed9b32701 100644 --- a/86.2m-g039-ai-r32/README.md +++ b/86.2m-g039-ai-r32/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r4/README.md b/86.2m-g039-ai-r4/README.md index 408c6273f48ac691e7f5eaf0959bad73fd6e717b..3dc0b357ddc9b1c3b6ec71e20c09a8514846e637 100644 --- a/86.2m-g039-ai-r4/README.md +++ b/86.2m-g039-ai-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-ai-r8/README.md b/86.2m-g039-ai-r8/README.md index a7bdbf0b7cea2bcd242e681b49e891e0ccac3b25..c6d09a96ca3884d32f2b709db06450f8b77b529e 100644 --- a/86.2m-g039-ai-r8/README.md +++ b/86.2m-g039-ai-r8/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-control/README.md b/86.2m-g039-control/README.md index 6bf464bbe1a357ccaad732bf21b43390af2d89a8..b40a221c2c9a3c3954475241db47d7548d8700f4 100644 --- a/86.2m-g039-control/README.md +++ b/86.2m-g039-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-human-r0.025/README.md b/86.2m-g039-human-r0.025/README.md index 84a92fabc170f3004933d9d4955fbf1816c1ca17..56884f45567748adde09d353a00348ea3e28575a 100644 --- a/86.2m-g039-human-r0.025/README.md +++ b/86.2m-g039-human-r0.025/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-human-r0.05/README.md b/86.2m-g039-human-r0.05/README.md index 629f3a2d5190f03bc91d014c952525c428de3f2a..a87903dc74c32bf8c056d757195dba609429017f 100644 --- a/86.2m-g039-human-r0.05/README.md +++ b/86.2m-g039-human-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g039-human-r4/README.md b/86.2m-g039-human-r4/README.md index 9cbaad16a3cfa92f2adf600a9aa36f593c334eb6..6d8b663d586177dc84f87bcad012e37c842cdd24 100644 --- a/86.2m-g039-human-r4/README.md +++ b/86.2m-g039-human-r4/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g040-ai-r0.05/README.md b/86.2m-g040-ai-r0.05/README.md index 164eb6363c3e2b06859df33810d05b7ab0f9873b..59ee9d25bcab398d7efd016c08a6b13dfc16f272 100644 --- a/86.2m-g040-ai-r0.05/README.md +++ b/86.2m-g040-ai-r0.05/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g040-ai-r0.1/README.md b/86.2m-g040-ai-r0.1/README.md index 30b92cff3b96dc8ba4a536b29be63b3607c2c117..f19cc6df966160867c7540ef77c0dfa752e9622e 100644 --- a/86.2m-g040-ai-r0.1/README.md +++ b/86.2m-g040-ai-r0.1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g040-ai-r0.25/README.md b/86.2m-g040-ai-r0.25/README.md index 5a33ff046daa37bd5a7aecbec981c87c190dcd38..2dbfb2766f2d4a1ff11097b0f8466f056dffc6cc 100644 --- a/86.2m-g040-ai-r0.25/README.md +++ b/86.2m-g040-ai-r0.25/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g040-ai-r0.5/README.md b/86.2m-g040-ai-r0.5/README.md index 37f92ea00edc58bad427b80ceb0d1924cf85a886..c9d71452329107835ac7c575cda797ee61685e43 100644 --- a/86.2m-g040-ai-r0.5/README.md +++ b/86.2m-g040-ai-r0.5/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g040-ai-r1/README.md b/86.2m-g040-ai-r1/README.md index 8b0328aae2ccc9848871b1b1b9cf79c60a629dd9..baad71fdcfc3257d265d7ee03798cad563e606eb 100644 --- a/86.2m-g040-ai-r1/README.md +++ b/86.2m-g040-ai-r1/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g040-ai-r2.08/README.md b/86.2m-g040-ai-r2.08/README.md index 162af5bb14b45fd8aeeef1a78f7bb0e166326f2e..60bf732363e34d9515ca17b7677a45764150718f 100644 --- a/86.2m-g040-ai-r2.08/README.md +++ b/86.2m-g040-ai-r2.08/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g040-ai-r4.22/README.md b/86.2m-g040-ai-r4.22/README.md index 5ea655f5beccde97256063b1fcde1f4a035f839b..be734a6c6f1d5a4cc71315c7b59452146d62115b 100644 --- a/86.2m-g040-ai-r4.22/README.md +++ b/86.2m-g040-ai-r4.22/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g040-control/README.md b/86.2m-g040-control/README.md index 0de6d73cd7e8ddf5238844f1179a0c69f01904a5..eeaceae67e3cea0209a145d6ff5884e7a42742a8 100644 --- a/86.2m-g040-control/README.md +++ b/86.2m-g040-control/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295}, diff --git a/86.2m-g041-ai-r2.08/README.md b/86.2m-g041-ai-r2.08/README.md index 9ec597b8fd0c79c004f962bf4ffd3862863c736e..824282bf7af8035d91c9780ad090816e45e0b5ff 100644 --- a/86.2m-g041-ai-r2.08/README.md +++ b/86.2m-g041-ai-r2.08/README.md @@ -18,7 +18,7 @@ tags: > **License: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/).** The WildAI code, models and dataset are released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license: you may use and adapt them for non-commercial purposes, with attribution, and must share adaptations under the same license. -A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of 844 models that measure how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. +A 86.2M decoder-only language model trained from scratch for [*How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text*](https://arxiv.org/abs/2609.40295), one of the paper's models measuring how wild AI-generated web text in pretraining data changes a model's loss. It is a research artifact for studying scaling laws, not an assistant: it has had no instruction tuning or safety training. ## Training data @@ -95,7 +95,7 @@ The nanochat GPT recipe: rotary embeddings, RMSNorm without weights (also on que ## Citation ```bibtex -@article{russell2026aitoken, +@article{russell2026wildai, title = {How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text}, author = {Russell, Jenna and Glickenhaus, Ben and Thai, Katherine and Wieting, John and Iyyer, Mohit and Spero, Max and Emi, Bradley}, journal = {arXiv preprint arXiv:2609.40295},