File size: 1,647 Bytes
f757d4c af94d3c f757d4c 8d27a57 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | ---
tags:
- diffusion
- masked-language-modeling
- pytorch
---
# Diffusion tool caller 150M — long pretrain
**Code and reproduction:** [github.com/chameleon-lizard/diffusion-toolcaller](https://github.com/chameleon-lizard/diffusion-toolcaller)
This is an inference-only checkpoint of a custom **149.7M-parameter masked diffusion language model** after approximately **57.0B input tokens**. The count includes the original ~3B pretrain tokens and two passes over a 27B-token corpus; it does not mean 57B unique tokens.
The continued-pretraining mixture targeted 45% FineWeb-Edu, 20% DCLM, 25% Stack-Edu Python/TypeScript/JavaScript, and 10% Cosmopedia-v2. Maximum context length is 2048. FP32 RoPE calculations and the original masked diffusion objective were retained.
This checkpoint has **no tool router or slot heads**. It is not an autoregressive chat model or a `transformers.AutoModel` package. Load it with [`hub.py` in the code repository](https://github.com/chameleon-lizard/diffusion-toolcaller):
```python
from hub import load_hub_model
model, _, config = load_hub_model("path/to/downloaded/model")
model.eval()
```
Files are `model.safetensors`, `config.json`, and the exact training `tokenizer.json` (SHA-256 `6c8aaa9a542084f2457eab775d4eeb51f92a70c0fd9de28d5edb0ddec3c08d30`). Optimizer and RNG state, absolute paths, and raw training data were removed. The tokenizer's upstream license has not been independently established here.
See the [tool-calling posttrain](https://huggingface.co/chameleon-lizard/buckwheat-diffusion-toolcaller) for an evaluated checkpoint. This is a research release, not a tool-execution system.
|