tmy100000001 commited on
Commit
a7eb01d
·
verified ·
1 Parent(s): 18701ca

card: fine-grid gate analysis (F0.5 optimum 0.95, plateau 0.89-0.98, 0.9 retained)

Browse files
Files changed (1) hide show
  1. README.md +8 -7
README.md CHANGED
@@ -96,13 +96,14 @@ pre-molecular papers and gene-free abstracts that the pipeline is designed to re
96
  "all" denominator understates recall by construction. Corpus fire rate = share of the 87,600
97
  abstracts with at least one gate candidate scoring ≥ 0.9.
98
 
99
- Why the gate is 0.9 (`revision/crossencoder_gate_analysis.csv`): F1 is maximised at 0.5 on both
100
- the test set and the frozen benchmark, but this stage is a precision filter ahead of the LLM.
101
- Raising the gate from 0.5 to 0.9 lifts test precision 0.921 → 0.938 and benchmark precision
102
- 0.788 → 0.813 while cutting false fires by 19%, for 1.6 points of in-scope recall; beyond 0.9
103
- the marginal recall lost per avoided false fire rises to 2.7× (0.95), 3.6× (0.975) and 7.8×
104
- (0.99) the 0.5–0.9 slope, and the precision-weighted F0.5 is flat over 0.9–0.975 within seed
105
- noise. Test-set P/R at 0.9: 0.938 / 0.900 (3-seed means).
 
106
 
107
  Top-k coverage of the true entry (test positives): within each abstract's gene-gate
108
  candidates (the deployment universe, median 2 candidates) top-1 = 0.797 ± 0.013; against the
 
96
  "all" denominator understates recall by construction. Corpus fire rate = share of the 87,600
97
  abstracts with at least one gate candidate scoring ≥ 0.9.
98
 
99
+ Why the gate is 0.9 (`revision/crossencoder_gate_analysis_fine.csv`, 0.01-step scan): F1 is
100
+ maximised at the lowest threshold on both the test set and the frozen benchmark, but this
101
+ stage is a precision filter ahead of the LLM. The precision-weighted benchmark F0.5 rises
102
+ from 0.772 (0.5) to 0.782 (0.9) and peaks at 0.789 at 0.95 (on all three seeds); every
103
+ threshold from 0.89 to 0.98 is within one seed-sd of that maximum. 0.9 is retained as the
104
+ recall-favouring edge of the plateau (0.9 -> 0.95 would trade 1.9 points of in-scope recall
105
+ and 6 points of test recall for +1.6 points of benchmark precision and 13% fewer false
106
+ fires). Test-set P/R at 0.9: 0.938 / 0.900 (3-seed means).
107
 
108
  Top-k coverage of the true entry (test positives): within each abstract's gene-gate
109
  candidates (the deployment universe, median 2 candidates) top-1 = 0.797 ± 0.013; against the