Tiny dummy models
Collection
Randomly initialized tiny models for debugging/testing purpose • 225 items • Updated • 8
This is a randomly initialized tiny model based on thinkingmachines/Inkling-2-Small for testing and debugging.
| Feature | Source | Tiny |
|---|---|---|
| Checkpoint tensor elements (including packed weights/scales) | 8.69B BF16, 565.33K F32, 128.42B F8_E4M3, 247.67B U8 | 63.01M BF16, 5.90K F32, 30.08M F8_E4M3, 15.04M U8 |
| Safetensors weight files (including MTP) | 393.48 GB (25 files) | 171.22 MB (2 files) |
| Decoder layers / attention residual block | 48 / 6 | 12 / 6 |
| Attention cycle | 5 SWA + 1 global | Same; two complete cycles |
| Routed experts / experts per token | 384 / 8 | 96 / 8 |
| Expert precision | 40 NVFP4 + 8 MXFP8 layers | 6 NVFP4 + 6 MXFP8 layers |
| Vocabulary / tied embeddings | 201,024 / false | 201,024 / false |
| Hidden / expert width | 4,608 / 2,304 | 128 / 128 |
| Q heads / KV heads / head dimension | 48 / 8 / 128 | 8 / 4 / 128 |
| Vision width / layers | 1,024 / 23 | 32 / 4 |
| MTP layers / block size / Markov rank | 5 / 8 / 256 | 5 / 8 / 8 |
| MTP feature taps | 5,11,17,23,29,35,41,48 | 5,12 (final norm sentinel) |
| File path | Size |
|---|---|
| model.safetensors | 162.2MB |
| mtp.safetensors | 9.0MB |
Use SGLang with Inkling 2 support. No model-specific package is required.
SGLANG_BUILD_RUST_EXTS=none pip install 'git+https://github.com/sgl-project/sglang.git@f726172d760dea737c7f1e797c9f32efe14d1685#subdirectory=python'
sglang serve --model-path tiny-random/inkling-2 --tp 2 --ep 2 \
--dtype bfloat16 --attention-backend fa4 --enable-multimodal \
--context-length 2048 --mem-fraction-static 0.2 \
--disable-custom-all-reduce \
--skip-server-warmup \
--cuda-graph-config '{"decode": {"bs": [16]}, "prefill": {"bs": [128, 2048]}}' \
--reasoning-parser inkling --tool-call-parser inkling2
This repo is generated by GPT-6.1-Sol
Base model
thinkingmachines/Inkling-2-Small