decisionmodels commited on
Commit
7b696b9
·
verified ·
1 Parent(s): 1b72af5

Model card: final-weight latency

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -43,13 +43,13 @@ Hardware: one NVIDIA RTX PRO 6000 (96 GB). Both parts are loaded in FP8 (block-1
43
  | Held-out decision sets, vs KnowLine-31B-Gen1 | +8.88 points | [+7.24, +10.45] | Our unpublished held-out sets (3,000 items; 60% same-skill, 40% new-domain composite), FP8 engine, paired whole-case bootstrap, 2 Nov 2026 |
44
  | Held-out decision sets, vs d3-ultra | +1.53 points | [+0.51, +2.60] | same |
45
  | Public Decision Index 0.3 items, native index | 72.73 | [69.31, 75.44] | Fixed 1,000-item sample of the public suite, official scoring formula, BF16 parts; KnowLine 67.53, d3-ultra 69.87 on the same sample |
46
- | Latency median / mean / p80 | 546 / 815 / 706 ms | — | One RTX PRO 6000, one request at a time, 300 real-length requests including long ones; measured with same-architecture stand-in weights |
47
 
48
  The choice between two candidate ensembles for this release used both the held-out sets and the same 1,000 public items shown above, so these numbers are not an independent confirmation after that choice.
49
 
50
  On our held-out sets d3-ultra scores above KnowLine, while the public HF Decision Index ranks KnowLine first by its full score, which
51
  includes private test sets we cannot measure. The numbers above therefore say how REAX 1.3 compares to its two bases on our data, not where
52
- it would rank. A complete public-suite run and a final-weight latency measurement are in progress.
53
 
54
  ## Limitations and licences
55
 
 
43
  | Held-out decision sets, vs KnowLine-31B-Gen1 | +8.88 points | [+7.24, +10.45] | Our unpublished held-out sets (3,000 items; 60% same-skill, 40% new-domain composite), FP8 engine, paired whole-case bootstrap, 2 Nov 2026 |
44
  | Held-out decision sets, vs d3-ultra | +1.53 points | [+0.51, +2.60] | same |
45
  | Public Decision Index 0.3 items, native index | 72.73 | [69.31, 75.44] | Fixed 1,000-item sample of the public suite, official scoring formula, BF16 parts; KnowLine 67.53, d3-ultra 69.87 on the same sample |
46
+ | Latency median / mean / p80 | 545 / 809 / 706 ms | — | Final REAX 1.3 weights, one RTX PRO 6000, FP8, one request at a time, 300 real-length requests including long ones (p95 1,159 ms, max 23.6 s), 2 Nov 2026 |
47
 
48
  The choice between two candidate ensembles for this release used both the held-out sets and the same 1,000 public items shown above, so these numbers are not an independent confirmation after that choice.
49
 
50
  On our held-out sets d3-ultra scores above KnowLine, while the public HF Decision Index ranks KnowLine first by its full score, which
51
  includes private test sets we cannot measure. The numbers above therefore say how REAX 1.3 compares to its two bases on our data, not where
52
+ it would rank. A complete run of the full public suite is in progress.
53
 
54
  ## Limitations and licences
55