|
Download README.md from tacodevs/Wicked-Stuff-31B-Final-FP8-Dynamic: direct link, hf CLI and curl.
- Browser
- Download file 904 Bytes
-
https://huggingface.co/tacodevs/Wicked-Stuff-31B-Final-FP8-Dynamic/resolve/main/README.md
- Command line
-
hf download hf://tacodevs/Wicked-Stuff-31B-Final-FP8-Dynamic/README.md
-
curl -L -o README.md https://huggingface.co/tacodevs/Wicked-Stuff-31B-Final-FP8-Dynamic/resolve/main/README.md
904 Bytes
| license: apache-2.0 | |
| base_model: | |
| - ReadyArt/Wicked-Stuff-31B-Final | |
| base_model_relation: quantized | |
| tags: | |
| - fp8 | |
| - llm-compressor | |
| - compressed-tensors | |
| - gemma4 | |
| - roleplay | |
| # Wicked-Stuff-31B-Final FP8-Dynamic (W8A8) | |
| FP8 quantization of [ReadyArt/Wicked-Stuff-31B-Final](https://huggingface.co/ReadyArt/Wicked-Stuff-31B-Final) (Gemma 4 31B). | |
| - **Scheme:** `FP8_DYNAMIC` W8A8: per-channel FP8 weights, dynamic per-token FP8 activations, on all `Linear` layers (llm-compressor). Vision, audio, embeddings and `lm_head` stay BF16. No calibration data needed. | |
| - **Hardware:** Hopper (H100/H200) and Blackwell; Ada (RTX 4090/L40S) also has FP8. | |
| ```bash | |
| vllm serve tacodevs/Wicked-Stuff-31B-Final-FP8-Dynamic --max-model-len 32768 --kv-cache-dtype fp8 \ | |
| --reasoning-parser gemma4 --enable-auto-tool-choice --tool-call-parser gemma4 | |
| ``` | |
| Thinking: pass `chat_template_kwargs: {"enable_thinking": true}`. | |