| --- |
| tags: |
| - diffusion |
| - masked-language-modeling |
| - pytorch |
| --- |
| |
| # Diffusion tool caller 150M — long pretrain |
|
|
| **Code and reproduction:** [github.com/chameleon-lizard/diffusion-toolcaller](https://github.com/chameleon-lizard/diffusion-toolcaller) |
|
|
| This is an inference-only checkpoint of a custom **149.7M-parameter masked diffusion language model** after approximately **57.0B input tokens**. The count includes the original ~3B pretrain tokens and two passes over a 27B-token corpus; it does not mean 57B unique tokens. |
|
|
| The continued-pretraining mixture targeted 45% FineWeb-Edu, 20% DCLM, 25% Stack-Edu Python/TypeScript/JavaScript, and 10% Cosmopedia-v2. Maximum context length is 2048. FP32 RoPE calculations and the original masked diffusion objective were retained. |
|
|
| This checkpoint has **no tool router or slot heads**. It is not an autoregressive chat model or a `transformers.AutoModel` package. Load it with [`hub.py` in the code repository](https://github.com/chameleon-lizard/diffusion-toolcaller): |
|
|
| ```python |
| from hub import load_hub_model |
| model, _, config = load_hub_model("path/to/downloaded/model") |
| model.eval() |
| ``` |
|
|
| Files are `model.safetensors`, `config.json`, and the exact training `tokenizer.json` (SHA-256 `6c8aaa9a542084f2457eab775d4eeb51f92a70c0fd9de28d5edb0ddec3c08d30`). Optimizer and RNG state, absolute paths, and raw training data were removed. The tokenizer's upstream license has not been independently established here. |
|
|
| See the [tool-calling posttrain](https://huggingface.co/chameleon-lizard/buckwheat-diffusion-toolcaller) for an evaluated checkpoint. This is a research release, not a tool-execution system. |
|
|
