Add official 324-task benchmark comparison table
Browse files
README.md
CHANGED
|
@@ -14,3 +14,17 @@ tags:
|
|
| 14 |
# prod-jev-9b
|
| 15 |
|
| 16 |
Production ready decision model quantized in NVFP4.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
# prod-jev-9b
|
| 15 |
|
| 16 |
Production ready decision model quantized in NVFP4.
|
| 17 |
+
|
| 18 |
+
| Metric / Dimension | Unquantized BF16 | NVFP4 Weights + FP8 KV | Everything NVFP4 (Weights + KV) |
|
| 19 |
+
| :--- | :---: | :---: | :---: |
|
| 20 |
+
| Model Weight Precision | bfloat16 (16-bit) | nvfp4 (4-bit ModelOpt) | nvfp4 (4-bit ModelOpt) |
|
| 21 |
+
| KV Cache Precision | bfloat16 (16-bit) | fp8 (e4m3fn) | nvfp4 (4-bit Micro-Scaled) |
|
| 22 |
+
| Model Size on Disk / RAM | 16.71 GiB | 7.48 GiB | 7.48 GiB |
|
| 23 |
+
| Model Weight VRAM Footprint | ~17.8 GiB | ~7.5 GiB | ~7.5 GiB |
|
| 24 |
+
| KV Cache Capacity (GB10 / 122 GiB) | 2.06M tokens | 3.64M tokens | 5.09M tokens |
|
| 25 |
+
| Max Concurrency (16K Context) | 125.5× | 222.0× | 310.8× |
|
| 26 |
+
| Decision Index Accuracy (324 Tasks) | 80.56% (261/324) | 76.23% (247/324) | 78.70% (255/324) |
|
| 27 |
+
| Average Decision Latency | 90.76 ms | 49.60 ms | 45.53 ms |
|
| 28 |
+
| Total Benchmark Wall Time (324 Tasks) | 29.41 s | 16.07 s | 14.75 s |
|
| 29 |
+
| Decisions per Second (Batch Evaluation) | 11.0 req/s | 20.2 req/s | 22.0 req/s |
|
| 30 |
+
| Recommended Operational Profile | High-precision reference | 8-bit infrastructure fallback | Production SOTA (Recommended) |
|