|
Download README.md from tiny-random/inkling-2: direct link, hf CLI and curl.
- Browser
- Download file 2.35 kB
-
https://huggingface.co/tiny-random/inkling-2/resolve/main/README.md
- Command line
-
hf download hf://tiny-random/inkling-2/README.md
-
curl -L -o README.md https://huggingface.co/tiny-random/inkling-2/resolve/main/README.md
2.35 kB
| library_name: sglang | |
| pipeline_tag: image-text-to-text | |
| base_model: | |
| - thinkingmachines/Inkling-2-Small | |
| This is a randomly initialized tiny model based on [thinkingmachines/Inkling-2-Small](https://huggingface.co/thinkingmachines/Inkling-2-Small) for testing and debugging. | |
| | Feature | Source | Tiny | | |
| |---|---|---| | |
| | Checkpoint tensor elements (including packed weights/scales) | 8.69B BF16, 565.33K F32, 128.42B F8_E4M3, 247.67B U8 | 63.01M BF16, 5.90K F32, 30.08M F8_E4M3, 15.04M U8 | | |
| | Safetensors weight files (including MTP) | 393.48 GB (25 files) | 171.22 MB (2 files) | | |
| | Decoder layers / attention residual block | 48 / 6 | 12 / 6 | | |
| | Attention cycle | 5 SWA + 1 global | Same; two complete cycles | | |
| | Routed experts / experts per token | 384 / 8 | 96 / 8 | | |
| | Expert precision | 40 NVFP4 + 8 MXFP8 layers | 6 NVFP4 + 6 MXFP8 layers | | |
| | Vocabulary / tied embeddings | 201,024 / false | 201,024 / false | | |
| | Hidden / expert width | 4,608 / 2,304 | 128 / 128 | | |
| | Q heads / KV heads / head dimension | 48 / 8 / 128 | 8 / 4 / 128 | | |
| | Vision width / layers | 1,024 / 23 | 32 / 4 | | |
| | MTP layers / block size / Markov rank | 5 / 8 / 256 | 5 / 8 / 8 | | |
| | MTP feature taps | 5,11,17,23,29,35,41,48 | 5,12 (final norm sentinel) | | |
| | File path | Size | | |
| |------|------| | |
| | model.safetensors | 162.2MB | | |
| | mtp.safetensors | 9.0MB | | |
| ### Example usage | |
| #### SGLang serving (✅ verified) | |
| Use SGLang with Inkling 2 support. No model-specific package is required. | |
| ```bash | |
| SGLANG_BUILD_RUST_EXTS=none pip install 'git+https://github.com/sgl-project/sglang.git@f726172d760dea737c7f1e797c9f32efe14d1685#subdirectory=python' | |
| sglang serve --model-path tiny-random/inkling-2 --tp 2 --ep 2 \ | |
| --dtype bfloat16 --attention-backend fa4 --enable-multimodal \ | |
| --context-length 2048 --mem-fraction-static 0.2 \ | |
| --disable-custom-all-reduce \ | |
| --skip-server-warmup \ | |
| --cuda-graph-config '{"decode": {"bs": [16]}, "prefill": {"bs": [128, 2048]}}' \ | |
| --reasoning-parser inkling --tool-call-parser inkling2 | |
| ``` | |
| ### Creating this repo | |
| - Code: [.meta/create_model.py](.meta/create_model.py) | |
| - Model structure: [.meta/model.log](.meta/model.log) | |
| - Environment: [.meta/env.json](.meta/env.json) | |
| ### Change log | |
| - 2026-11-14: Preserve source mixed NVFP4/MXFP8 expert precision, with 96 experts and independent token tables. | |
| *This repo is generated by GPT-6.1-Sol* |