| --- |
| library_name: bibr |
| base_model: sentence-transformers/all-MiniLM-L6-v2 |
| language: en |
| tags: |
| - text-classification |
| - scholarly-documents |
| --- |
| |
| # bibr section classifier |
|
|
| Two-head classifier that bibr uses to type section headings it cannot resolve |
| with its alias lookup. It predicts one of 16 canonical section types and |
| whether the heading is top level. Encoder: all-MiniLM-L6-v2 (22M parameters), |
| mean pooling, input template v3 (heading, position bucket, neighbouring |
| headings, start of the body). |
|
|
| This revision retrains the same architecture and template on a larger, |
| re-labelled corpus. It is a drop-in replacement for the September 2026 |
| release: same files, same label order, same runtime cost. |
|
|
| ## Training data |
|
|
| - `section_v5b_leakfiltered`: section headings from PMC Open Access JATS |
| papers (English), paper-grouped train/val/test split. Three papers removed |
| by a later leak re-check are excluded. |
| - Train subsample of at most 3,000 rows per class: 45,979 rows from 6,946 papers. |
| Epoch chosen on the v5b validation split only. |
| - No bibr-validation gold, blind evaluation, or held-out benchmark paper is in |
| the training or validation data (matched by DOI, PMCID and title). |
|
|
| ## Evaluation |
|
|
| Canonical-type macro-F1 / accuracy (model-routed rows: macro-F1). "Model-routed" is the subset bibr |
| actually sends to this model (headings the alias lookup misses), scored on |
| bibr's section gold. |
|
|
| | set | previous release | this revision | |
| |---|---|---| |
| | v5b test (English JATS, 9,507 headings) | 0.589 / 0.398 | 0.966 / 0.946 | |
| | gold, English headings (9,501) | 0.656 / 0.572 | 0.798 / 0.784 | |
| | gold, non-English headings (1,524) | 0.213 / 0.295 | 0.265 / 0.344 | |
| | gold, model-routed English (3,472) | 0.419 | 0.676 | |
| | gold, model-routed non-English (1,240) | 0.168 | 0.217 | |
|
|
| Part of the gain on gold comes from shared labelling conventions: the |
| training data and the gold both label subsections by their parent type and |
| structured-abstract parts as abstract. |
|
|
| Non-English headings are zero-shot: the training data is English only. |
|
|
| ## Files |
|
|
| `model.safetensors`, `tokenizer.json`, `tokenizer_config.json`, |
| `label_classes.json`, `inference_config.json` (template version 3), and an |
| ONNX Runtime bundle in `onnx/` used by bibr's core install. |
|
|
