Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -14,14 +14,15 @@ tags:
14
  - fp4
15
  ---
16
 
 
17
  # Model Overview
18
 
19
  ## Description:
20
  The NVIDIA Qwen3.8-Flash-Next-FP4-Local model is the quantized version of Alibaba's Qwen3.8-Flash-Next model, which is an auto-regressive language model that uses an optimized transformer architecture. Qwen3.8-Flash-Next is a causal language model with a vision encoder, hybrid attention (Gated DeltaNet and Qwen Sparse Attention), Mixture-of-Experts, gated residual streams, and n-gram embeddings. For more information, please check [here](https://huggingface.co/Qwen/Qwen3.8-Flash-Next). The NVIDIA Qwen3.8-Flash-Next-FP4-Local model is quantized with [Model Optimizer](https://github.com/NVIDIA/Model-Optimizer) using [quantization-aware distillation (QAD)](https://developer.nvidia.com/blog/how-quantization-aware-training-enables-low-precision-accuracy-recovery/).
21
 
22
  The checkpoint occupies approximately **96 GiB on disk** and is best suited for local AI hardware. It has been validated with vLLM on the following hardware: <br>
23
- * **NVIDIA RTX PRO 6000 Blackwell** — single GPU, with PLE (n-gram embedding table) offload. <br>
24
- * **NVIDIA DGX Spark** — with or without PLE offload. <br>
25
 
26
  This model is ready for commercial or non-commercial use. <br>
27
 
 
14
  - fp4
15
  ---
16
 
17
+
18
  # Model Overview
19
 
20
  ## Description:
21
  The NVIDIA Qwen3.8-Flash-Next-FP4-Local model is the quantized version of Alibaba's Qwen3.8-Flash-Next model, which is an auto-regressive language model that uses an optimized transformer architecture. Qwen3.8-Flash-Next is a causal language model with a vision encoder, hybrid attention (Gated DeltaNet and Qwen Sparse Attention), Mixture-of-Experts, gated residual streams, and n-gram embeddings. For more information, please check [here](https://huggingface.co/Qwen/Qwen3.8-Flash-Next). The NVIDIA Qwen3.8-Flash-Next-FP4-Local model is quantized with [Model Optimizer](https://github.com/NVIDIA/Model-Optimizer) using [quantization-aware distillation (QAD)](https://developer.nvidia.com/blog/how-quantization-aware-training-enables-low-precision-accuracy-recovery/).
22
 
23
  The checkpoint occupies approximately **96 GiB on disk** and is best suited for local AI hardware. It has been validated with vLLM on the following hardware: <br>
24
+ * **NVIDIA RTX PRO 6000 Blackwell** — single GPU, with PLE (n-gram embedding table) offload. 850K tokens of FP8 KV Cache <br>
25
+ * **NVIDIA DGX Spark** — Can run with or without PLE offload. 1M tokens of FP8 KV Cache with PLE offload <br>
26
 
27
  This model is ready for commercial or non-commercial use. <br>
28