Wicked-Stuff-31B-Final NVFP4 (W4A4, GPTQ, roleplay-calibrated)

GPTQ NVFP4 of ReadyArt/Wicked-Stuff-31B-Final: FP4 weights and activations (group 16) on all text Linear layers. Vision, audio, embeddings and lm_head stay BF16. Built with llm-compressor 0.13 GPTQModifier(scheme="NVFP4").

Calibration: 256 samples of up to 8192 tokens of multi-turn roleplay (prompt plus reply).

Drift vs BF16

Measured on 50 held-out chats (23.7k reply tokens), served by vLLM 0.27.1 with fp8 KV:

Build Top-1 agreement Approx KL
This build 90.2% 0.061
Public NVFP4 (Gryphe calibration, RTN) 87.7% 0.102
FP8-Dynamic 94.9% 0.016
Downloads last month
46
Safetensors
Model size
18B params
Tensor type
BF16
Β·
U8
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for tacodevs/Wicked-Stuff-31B-Final-NVFP4-GPTQ-Max

Quantized
(8)
this model