Card: measured start-up time (packaged install on the RTX 5090)
Browse files
README.md
CHANGED
|
@@ -99,7 +99,7 @@ The 2B answered 123 of the 231 items itself. Per-item results are in [validation
|
|
| 99 |
| Calibration set (1,000 requests, up to 12K tokens) | 37.5 ms | 139.0 ms | 673.6 ms |
|
| 100 |
| JevBench public items (231, one question each) | 29.0 ms | | 477.4 ms |
|
| 101 |
|
| 102 |
-
<sub>torch 2.14.1+cu130, transformers 5.18.0, flash-linear-attention 0.5.2, jevk5 0.3.3; the models are loaded and their CUDA graphs recorded before timing (start-up
|
| 103 |
|
| 104 |
## Memory
|
| 105 |
|
|
|
|
| 99 |
| Calibration set (1,000 requests, up to 12K tokens) | 37.5 ms | 139.0 ms | 673.6 ms |
|
| 100 |
| JevBench public items (231, one question each) | 29.0 ms | | 477.4 ms |
|
| 101 |
|
| 102 |
+
<sub>torch 2.14.1+cu130, transformers 5.18.0, flash-linear-attention 0.5.2, jevk5 0.3.3; the models are loaded and their CUDA graphs recorded before timing (start-up 47 s with the weights cached, including the SHA256SUMS check of JevK5-9B). A 25,275-token request with two questions took 2.4 s.</sub>
|
| 103 |
|
| 104 |
## Memory
|
| 105 |
|