bupalinyu commited on
Commit
33fb783
·
verified ·
1 Parent(s): 941946d

docs: compare the expert pool at FP4 on both sides

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -405,7 +405,7 @@ and how much memory a machine needs:
405
  |---|---:|---:|---:|
406
  | Routed expert parameters | 120.8 B | **84.6 B** (−30%) | **60.4 B** (−50%) |
407
  | Expert parameters per token | 2.36 B | **1.65 B** | **1.18 B** |
408
- | Expert pool on disk · Q3_K builds | 46.84 GiB | **37.32 GiB** | **26.66 GiB** |
409
  | Decode · 16 GiB VRAM + 20 GiB RAM | 18.5 tok/s | **36.8 tok/s** | **62.4 tok/s** |
410
 
411
  On that budget, decode runs 2.0× Qwen's on the 145B and 3.4× on the 120B. Quality holds through the
 
405
  |---|---:|---:|---:|
406
  | Routed expert parameters | 120.8 B | **84.6 B** (−30%) | **60.4 B** (−50%) |
407
  | Expert parameters per token | 2.36 B | **1.65 B** | **1.18 B** |
408
+ | Expert pool on disk · NVFP4 builds | 63.2 GiB | **44.3 GiB** | **31.6 GiB** |
409
  | Decode · 16 GiB VRAM + 20 GiB RAM | 18.5 tok/s | **36.8 tok/s** | **62.4 tok/s** |
410
 
411
  On that budget, decode runs 2.0× Qwen's on the 145B and 3.4× on the 120B. Quality holds through the