Wicked-Stuff-31B-Final NVFP4 (W4A4)
NVFP4 quantization of ReadyArt/Wicked-Stuff-31B-Final (Gemma 4 31B).
- Scheme: NVFP4 W4A4 (FP4 weights and activations, group 16) on all
Linearlayers, made with llm-compressorQuantizationModifier. Vision, audio, embeddings andlm_headstay BF16. - Calibration: 128 samples Γ up to 4096 tokens of multi-turn roleplay from
Gryphe/Sonnet3.5-Charcard-Roleplay, rendered with the model's own chat template. - Hardware: NVFP4 runs natively on Blackwell (RTX 5090, RTX PRO 6000, B200). On Hopper use the FP8 build instead.
vllm serve tacodevs/Wicked-Stuff-31B-Final-NVFP4-W4A4 --max-model-len 32768 --kv-cache-dtype fp8 \
--reasoning-parser gemma4 --enable-auto-tool-choice --tool-call-parser gemma4
Thinking: pass chat_template_kwargs: {"enable_thinking": true}.
- Downloads last month
- 47
Inference Providers NEW
This model isn't deployed by any Inference Provider. π Ask for provider support
Model tree for tacodevs/Wicked-Stuff-31B-Final-NVFP4-W4A4
Base model
ReadyArt/gemma-4-31B-it-scotoma-2 Finetuned
ReadyArt/Wicked-Stuff-31B-Final