Model card: final-weight latency
Browse files
README.md
CHANGED
|
@@ -43,13 +43,13 @@ Hardware: one NVIDIA RTX PRO 6000 (96 GB). Both parts are loaded in FP8 (block-1
|
|
| 43 |
| Held-out decision sets, vs KnowLine-31B-Gen1 | +8.88 points | [+7.24, +10.45] | Our unpublished held-out sets (3,000 items; 60% same-skill, 40% new-domain composite), FP8 engine, paired whole-case bootstrap, 2 Nov 2026 |
|
| 44 |
| Held-out decision sets, vs d3-ultra | +1.53 points | [+0.51, +2.60] | same |
|
| 45 |
| Public Decision Index 0.3 items, native index | 72.73 | [69.31, 75.44] | Fixed 1,000-item sample of the public suite, official scoring formula, BF16 parts; KnowLine 67.53, d3-ultra 69.87 on the same sample |
|
| 46 |
-
| Latency median / mean / p80 |
|
| 47 |
|
| 48 |
The choice between two candidate ensembles for this release used both the held-out sets and the same 1,000 public items shown above, so these numbers are not an independent confirmation after that choice.
|
| 49 |
|
| 50 |
On our held-out sets d3-ultra scores above KnowLine, while the public HF Decision Index ranks KnowLine first by its full score, which
|
| 51 |
includes private test sets we cannot measure. The numbers above therefore say how REAX 1.3 compares to its two bases on our data, not where
|
| 52 |
-
it would rank. A complete
|
| 53 |
|
| 54 |
## Limitations and licences
|
| 55 |
|
|
|
|
| 43 |
| Held-out decision sets, vs KnowLine-31B-Gen1 | +8.88 points | [+7.24, +10.45] | Our unpublished held-out sets (3,000 items; 60% same-skill, 40% new-domain composite), FP8 engine, paired whole-case bootstrap, 2 Nov 2026 |
|
| 44 |
| Held-out decision sets, vs d3-ultra | +1.53 points | [+0.51, +2.60] | same |
|
| 45 |
| Public Decision Index 0.3 items, native index | 72.73 | [69.31, 75.44] | Fixed 1,000-item sample of the public suite, official scoring formula, BF16 parts; KnowLine 67.53, d3-ultra 69.87 on the same sample |
|
| 46 |
+
| Latency median / mean / p80 | 545 / 809 / 706 ms | — | Final REAX 1.3 weights, one RTX PRO 6000, FP8, one request at a time, 300 real-length requests including long ones (p95 1,159 ms, max 23.6 s), 2 Nov 2026 |
|
| 47 |
|
| 48 |
The choice between two candidate ensembles for this release used both the held-out sets and the same 1,000 public items shown above, so these numbers are not an independent confirmation after that choice.
|
| 49 |
|
| 50 |
On our held-out sets d3-ultra scores above KnowLine, while the public HF Decision Index ranks KnowLine first by its full score, which
|
| 51 |
includes private test sets we cannot measure. The numbers above therefore say how REAX 1.3 compares to its two bases on our data, not where
|
| 52 |
+
it would rank. A complete run of the full public suite is in progress.
|
| 53 |
|
| 54 |
## Limitations and licences
|
| 55 |
|