File size: 1,367 Bytes
b63535d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | ---
license: apache-2.0
pipeline_tag: text-classification
tags:
- data-use
- evidence-classification
- modernbert
---
# datause-tier-clf
Fine-tune of `answerdotai/ModernBERT-large` classifying data-use mention spans (marked
`>>> mention <<<` in context) into evidence tiers, v3.
Trained on 62k luna-judged spans
(`rafmacalaba/data-use-evidence-classifier-data`; val+holdout dual-pass
+ train-top single-pass verdicts), decontaminated against the 573
human-adjudicated gold controls used for evaluation below.
## Labels
- `tier1_evidential` — data used as warrant/analytical consumption
- `tier2_declaration` — named/consulted without analytical use
- `tier3_nonmention` — promissory/logframe/bibliographic/caveat/concept
- `junk` — malformed extraction
## Input format
`prev[<=220 chars] >>> mention <<< next[<=280 chars]`
## Training
- base model: `answerdotai/ModernBERT-large`
- epochs: 3, lr: 2e-05, batch: 32 x1
- max_length: 512, precision: bf16
## Benchmark (573 gold controls, human tiers)
- 4-class accuracy: 0.5079, macro-F1: 0.5662
- per-class F1: tier1_evidential=0.545, tier2_declaration=0.413, tier3_nonmention=0.580, junk=0.727
- binary T1-vs-rest @0.5: P=0.5641 R=0.5033 F1=0.5320
- best threshold 0.20 (F1=0.5743)
## Silver consensus (n=3,999 clean dual-pass, held out of training)
- 4-class accuracy: 0.9002, macro-F1: 0.8355
|