| --- |
| license: apache-2.0 |
| pipeline_tag: text-classification |
| tags: |
| - data-use |
| - evidence-classification |
| - modernbert |
| --- |
| |
| # datause-tier-clf |
|
|
| Fine-tune of `answerdotai/ModernBERT-large` classifying data-use mention spans (marked |
| `>>> mention <<<` in context) into evidence tiers, v3. |
|
|
| Trained on 62k luna-judged spans |
| (`rafmacalaba/data-use-evidence-classifier-data`; val+holdout dual-pass |
| + train-top single-pass verdicts), decontaminated against the 573 |
| human-adjudicated gold controls used for evaluation below. |
|
|
| ## Labels |
| - `tier1_evidential` — data used as warrant/analytical consumption |
| - `tier2_declaration` — named/consulted without analytical use |
| - `tier3_nonmention` — promissory/logframe/bibliographic/caveat/concept |
| - `junk` — malformed extraction |
|
|
| ## Input format |
| `prev[<=220 chars] >>> mention <<< next[<=280 chars]` |
|
|
| ## Training |
| - base model: `answerdotai/ModernBERT-large` |
| - epochs: 3, lr: 2e-05, batch: 32 x1 |
| - max_length: 512, precision: bf16 |
| |
| ## Benchmark (573 gold controls, human tiers) |
| - 4-class accuracy: 0.5079, macro-F1: 0.5662 |
| - per-class F1: tier1_evidential=0.545, tier2_declaration=0.413, tier3_nonmention=0.580, junk=0.727 |
| - binary T1-vs-rest @0.5: P=0.5641 R=0.5033 F1=0.5320 |
| - best threshold 0.20 (F1=0.5743) |
|
|
| ## Silver consensus (n=3,999 clean dual-pass, held out of training) |
| - 4-class accuracy: 0.9002, macro-F1: 0.8355 |
|
|
|
|
