Add prominent GitHub code link
Browse files
README.md
CHANGED
|
@@ -7,6 +7,8 @@ tags:
|
|
| 7 |
|
| 8 |
# Diffusion tool caller 150M — long pretrain
|
| 9 |
|
|
|
|
|
|
|
| 10 |
This is an inference-only checkpoint of a custom **149.7M-parameter masked diffusion language model** after approximately **57.0B input tokens**. The count includes the original ~3B pretrain tokens and two passes over a 27B-token corpus; it does not mean 57B unique tokens.
|
| 11 |
|
| 12 |
The continued-pretraining mixture targeted 45% FineWeb-Edu, 20% DCLM, 25% Stack-Edu Python/TypeScript/JavaScript, and 10% Cosmopedia-v2. Maximum context length is 2048. FP32 RoPE calculations and the original masked diffusion objective were retained.
|
|
|
|
| 7 |
|
| 8 |
# Diffusion tool caller 150M — long pretrain
|
| 9 |
|
| 10 |
+
**Code and reproduction:** [github.com/chameleon-lizard/diffusion-toolcaller](https://github.com/chameleon-lizard/diffusion-toolcaller)
|
| 11 |
+
|
| 12 |
This is an inference-only checkpoint of a custom **149.7M-parameter masked diffusion language model** after approximately **57.0B input tokens**. The count includes the original ~3B pretrain tokens and two passes over a 27B-token corpus; it does not mean 57B unique tokens.
|
| 13 |
|
| 14 |
The continued-pretraining mixture targeted 45% FineWeb-Edu, 20% DCLM, 25% Stack-Edu Python/TypeScript/JavaScript, and 10% Cosmopedia-v2. Maximum context length is 2048. FP32 RoPE calculations and the original masked diffusion objective were retained.
|