This is a randomly initialized tiny model based on thinkingmachines/Inkling-2-Small for testing and debugging.

Feature Source Tiny
Checkpoint tensor elements (including packed weights/scales) 8.69B BF16, 565.33K F32, 128.42B F8_E4M3, 247.67B U8 63.01M BF16, 5.90K F32, 30.08M F8_E4M3, 15.04M U8
Safetensors weight files (including MTP) 393.48 GB (25 files) 171.22 MB (2 files)
Decoder layers / attention residual block 48 / 6 12 / 6
Attention cycle 5 SWA + 1 global Same; two complete cycles
Routed experts / experts per token 384 / 8 96 / 8
Expert precision 40 NVFP4 + 8 MXFP8 layers 6 NVFP4 + 6 MXFP8 layers
Vocabulary / tied embeddings 201,024 / false 201,024 / false
Hidden / expert width 4,608 / 2,304 128 / 128
Q heads / KV heads / head dimension 48 / 8 / 128 8 / 4 / 128
Vision width / layers 1,024 / 23 32 / 4
MTP layers / block size / Markov rank 5 / 8 / 256 5 / 8 / 8
MTP feature taps 5,11,17,23,29,35,41,48 5,12 (final norm sentinel)
File path Size
model.safetensors 162.2MB
mtp.safetensors 9.0MB

Example usage

SGLang serving (✅ verified)

Use SGLang with Inkling 2 support. No model-specific package is required.

SGLANG_BUILD_RUST_EXTS=none pip install 'git+https://github.com/sgl-project/sglang.git@f726172d760dea737c7f1e797c9f32efe14d1685#subdirectory=python'
sglang serve --model-path tiny-random/inkling-2 --tp 2 --ep 2 \
  --dtype bfloat16 --attention-backend fa4 --enable-multimodal \
  --context-length 2048 --mem-fraction-static 0.2 \
  --disable-custom-all-reduce \
  --skip-server-warmup \
  --cuda-graph-config '{"decode": {"bs": [16]}, "prefill": {"bs": [128, 2048]}}' \
  --reasoning-parser inkling --tool-call-parser inkling2

Creating this repo

Change log

  • 2026-11-14: Preserve source mixed NVFP4/MXFP8 expert precision, with 96 experts and independent token tables.

This repo is generated by GPT-6.1-Sol

Downloads last month
74
Safetensors
Model size
0.1B params
Tensor type
F32
·
BF16
·
F8_E4M3
·
U8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for yujiepan/inkling-2-tiny-random

Finetuned
(6)
this model

Collection including yujiepan/inkling-2-tiny-random