Wicked-Stuff-31B-Final NVFP4 (W4A4)

NVFP4 quantization of ReadyArt/Wicked-Stuff-31B-Final (Gemma 4 31B).

  • Scheme: NVFP4 W4A4 (FP4 weights and activations, group 16) on all Linear layers, made with llm-compressor QuantizationModifier. Vision, audio, embeddings and lm_head stay BF16.
  • Calibration: 128 samples Γ— up to 4096 tokens of multi-turn roleplay from Gryphe/Sonnet3.5-Charcard-Roleplay, rendered with the model's own chat template.
  • Hardware: NVFP4 runs natively on Blackwell (RTX 5090, RTX PRO 6000, B200). On Hopper use the FP8 build instead.
vllm serve tacodevs/Wicked-Stuff-31B-Final-NVFP4-W4A4 --max-model-len 32768 --kv-cache-dtype fp8 \
  --reasoning-parser gemma4 --enable-auto-tool-choice --tool-call-parser gemma4

Thinking: pass chat_template_kwargs: {"enable_thinking": true}.

Downloads last month
47
Safetensors
Model size
20B params
Tensor type
BF16
Β·
U8
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for tacodevs/Wicked-Stuff-31B-Final-NVFP4-W4A4

Quantized
(8)
this model