README / README.md
ominn's picture
Update the calibration list
2d4477f verified
|
Raw History Blame Contribute Delete
8.83 kB
metadata
title: README
emoji: 📐
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false

SketchSSM calibrations

SketchSSM speeds up the decode of hybrid linear-attention models: the recurrent state is updated exactly once per window, and the other decode steps read a compact per-request sketch of it. Each repository here holds the calibration file (calibration.pt) for one model checkpoint; it contains no model weights.

vllm serve <base model> --sketchssm SketchSSM/<calibration repo> \
  --sketchssm-mean-rank 8 --mamba-ssm-cache-dtype float32

Use the calibration of the exact checkpoint (precision included) you serve. To calibrate another model, see the calibration guide.

Mamba-2 (NemotronH)

Collection

Calibration Base checkpoint Weights
SketchSSM/Nemotron-3-Nano-30B-A3B-Base-BF16 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16 BF16
SketchSSM/Nemotron-3-Nano-30B-A3B-BF16 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 BF16
SketchSSM/Nemotron-3-Nano-30B-A3B-FP8 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 FP8
SketchSSM/Nemotron-3-Nano-30B-A3B-NVFP4 nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 NVFP4
SketchSSM/Nemotron-3-Nano-4B-BF16 nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 BF16
SketchSSM/Nemotron-3-Nano-4B-FP8 nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8 FP8
SketchSSM/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8 nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-FP8 FP8
SketchSSM/Nemotron-3-Super-NVFP4 nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 NVFP4
SketchSSM/Nemotron-3.5-Lightning-30B-A3B-Base-BF16 nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Base-BF16 BF16
SketchSSM/Nemotron-3.5-Lightning-30B-A3B-BF16 nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 BF16
SketchSSM/Nemotron-Cascade-2-30B-A3B-BF16 nvidia/Nemotron-Cascade-2-30B-A3B BF16
SketchSSM/Nemotron-H-4B-Base-8K-BF16 nvidia/Nemotron-H-4B-Base-8K BF16
SketchSSM/Nemotron-Labs-3-Elastic-30B-A3B-BF16 nvidia/NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-BF16 BF16
SketchSSM/Nemotron-Labs-3-Elastic-30B-A3B-FP8 nvidia/NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-FP8 FP8
SketchSSM/Nemotron-Nano-12B-v2-Base-BF16 nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base BF16
SketchSSM/Nemotron-Nano-12B-v2-BF16 nvidia/NVIDIA-Nemotron-Nano-12B-v2 BF16
SketchSSM/Nemotron-Nano-12B-v2-VL-FP8 nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-FP8 FP8
SketchSSM/Nemotron-Nano-12B-v2-VL-NVFP4-QAD nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-NVFP4-QAD NVFP4
SketchSSM/Nemotron-Nano-9B-v2-Base-BF16 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Base BF16
SketchSSM/Nemotron-Nano-9B-v2-BF16 nvidia/NVIDIA-Nemotron-Nano-9B-v2 BF16
SketchSSM/Nemotron-Nano-9B-v2-FP8 nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8 FP8
SketchSSM/Nemotron-Nano-9B-v2-Japanese-BF16 nvidia/NVIDIA-Nemotron-Nano-9B-v2-Japanese BF16
SketchSSM/Nemotron-Nano-9B-v2-NVFP4 nvidia/NVIDIA-Nemotron-Nano-9B-v2-NVFP4 NVFP4

Gated DeltaNet

Collection

KDA

Collection

Calibration Base checkpoint Weights
SketchSSM/GLM-5.3-Flash-NVFP4 RedHatAI/GLM-5.3-Flash-NVFP4 NVFP4