| --- |
| pipeline_tag: token-classification |
| tags: |
| - sequence-labelling |
| - crf |
| - wapiti |
| - delft |
| - bilstm-crf |
| - grobid |
| - scientific-document-parsing |
| - document-segmentation |
| - reference-extraction |
| license: apache-2.0 |
| --- |
| |
| # ScienceBeam sequence models |
|
|
| Sequence-labelling models for the GROBID-style extraction pipeline used by |
| [sciencebeam-parser](https://github.com/elifesciences/sciencebeam-parser), trained on GROBID's |
| corpus plus training data generated from open-access preprint and journal corpora. |
|
|
| These are drop-in replacements for individual GROBID models, on the same feature layout, in the two |
| formats the parser already loads: `model.wapiti.gz` for a wapiti CRF, and a PyTorch model directory |
| for a delft `CustomBidLSTM_CRF`. |
|
|
| The wapiti models predict one label fewer than GROBID's shipped models, which also predict |
| `I-<other>`, marking the first token of a run of unlabelled text. That label is absent from the |
| models rather than from the corpus, and background text is labelled `<other>` throughout instead. |
| The delft models label background `O` under IOB2, which is delft's representation of the same |
| thing. Each model's label set is listed in its `training.yml`. |
|
|
| The segmentation models predict one label GROBID 0.9.0's does not: `<review>`, for the peer reviews |
| a preprint server publishes alongside the article, which `sciencebeam-parser` maps to |
| `div[@type="review"]`. A consumer that swaps one of these in without handling it will be given a |
| region it has no case for. |
|
|
| They are selected per model by a parser profile, not as a set, so a profile can take one of these |
| and leave the rest of the pipeline on GROBID's own models. |
|
|
| ## Artifacts |
|
|
| | path | model | engine | trained on | |
| | --- | --- | --- | --- | |
| | `reference-segmenter/wapiti/2026-09-08_wapiti_grobid_090_scielo_preprints_ore` | reference segmenter | wapiti | GROBID 0.9.0 corpus + generated `scielo_preprints-jats` and `ore` | |
| | `citation/wapiti/2026-09-08_wapiti_grobid_090_scielo_preprints_ore` | citation | wapiti | GROBID 0.9.0 corpus + generated `scielo_preprints-jats` and `ore` | |
| | `reference-segmenter/delft/2026-09-18_CustomBidLSTM_CRF_grobid_090_scielo_preprints_ore` | reference segmenter | delft | GROBID 0.9.0 corpus + generated `scielo_preprints-jats` and `ore` | |
| | `citation/delft/2026-09-18_CustomBidLSTM_CRF_grobid_090_scielo_preprints_ore` | citation | delft | GROBID 0.9.0 corpus + generated `scielo_preprints-jats` and `ore` | |
| | `segmentation/wapiti/2026-10-06_wapiti_grobid_090_scielo_preprints_ore` | segmentation | wapiti | GROBID 0.9.0 corpus + generated `scielo_preprints-jats` and `ore` | |
| | `segmentation/delft/2026-10-06_CustomBidLSTM_CRF_grobid_090_scielo_preprints_ore_b50_flex_t4` | segmentation | delft | GROBID 0.9.0 corpus + generated `scielo_preprints-jats` and `ore` | |
|
|
| The two engines are trained on the same documents, so a difference between them is the engine |
| rather than the data. |
|
|
| Each directory holds a `training.yml` recording what the model was trained on, with which |
| hyperparameters, the feature layout it expects, and the checksum of every file beside it. A wapiti |
| directory holds `model.wapiti.gz`; a delft directory holds the model's own layout — |
| `config.json`, `preprocessor.json` and `model_weights.pt`. |
|
|
| Paths are `<model>/<engine>/<artifact>`, so everything available for one pipeline model is listed |
| together whichever engine produced it. |
|
|
| The artifact name is the training run: `<date>_<engine>_<base>_<training set>`, with a trailing |
| suffix where the same training set was run more than once and the runs differ in something worth |
| telling apart — `_b50_flex_t4` names batch size 50 and a T4. The same name is used wherever the |
| artifact appears — a parser config, a benchmark run record, these tables — so they refer to each |
| other without a lookup. |
|
|
| ### Controls |
|
|
| | path | model | trained on | |
| | --- | --- | --- | |
| | `reference-segmenter/wapiti/2026-08-04_wapiti_grobid_090_grobid_only` | reference segmenter | GROBID 0.9.0 corpus alone | |
| | `citation/wapiti/2026-08-04_wapiti_grobid_090_grobid_only` | citation | GROBID 0.9.0 corpus alone | |
|
|
| These are GROBID 0.9.0's corpus alone through the same pipeline at the same settings as the models |
| above. They exist so that a difference measured against one of those is attributable to the added |
| training data rather than to the pipeline. They are not recommended for serving: GROBID's own |
| published models are trained on that corpus and are the ones to use. |
|
|
| ## Use |
|
|
| In a `sciencebeam-parser` config, as a `sequence_model_profiles` entry: |
|
|
| ```yaml |
| sequence_model_profiles: |
| wapiti_scielo_preprints_ore: |
| extends: grobid_crf_0_9_0 |
| reference_segmenter: |
| path: 'https://huggingface.co/elifepathways/sciencebeam-sequence-models/resolve/<commit-sha>/reference-segmenter/wapiti/2026-09-08_wapiti_grobid_090_scielo_preprints_ore' |
| engine: 'wapiti' |
| citation: |
| path: 'https://huggingface.co/elifepathways/sciencebeam-sequence-models/resolve/<commit-sha>/citation/wapiti/2026-09-08_wapiti_grobid_090_scielo_preprints_ore' |
| engine: 'wapiti' |
| ``` |
|
|
| The path is a directory; the parser appends `model.wapiti.gz`. A delft model is selected the same |
| way, by its own directory and `engine: 'delft'`: |
|
|
| ```yaml |
| sequence_model_profiles: |
| delft_scielo_preprints_ore: |
| extends: grobid_crf_0_9_0 |
| reference_segmenter: |
| path: 'https://huggingface.co/elifepathways/sciencebeam-sequence-models/resolve/<commit-sha>/reference-segmenter/delft/2026-09-18_CustomBidLSTM_CRF_grobid_090_scielo_preprints_ore' |
| engine: 'delft' |
| citation: |
| path: 'https://huggingface.co/elifepathways/sciencebeam-sequence-models/resolve/<commit-sha>/citation/delft/2026-09-18_CustomBidLSTM_CRF_grobid_090_scielo_preprints_ore' |
| engine: 'delft' |
| ``` |
|
|
| Segmentation is selected by its own key, and the path is again the directory: |
|
|
| ```yaml |
| sequence_model_profiles: |
| wapiti_seg_scielo_preprints_ore: |
| extends: grobid_crf_0_9_0 |
| segmentation: |
| path: 'https://huggingface.co/elifepathways/sciencebeam-sequence-models/resolve/<commit-sha>/segmentation/wapiti/2026-10-06_wapiti_grobid_090_scielo_preprints_ore' |
| engine: 'wapiti' |
| ``` |
|
|
| The delft segmentation model was trained over a 2000-token window at a stride of 1800, and a |
| segmentation sequence is a whole document rather than one reference, so serving it at a shorter |
| window truncates documents. `training.yml` records the window each model was trained at. |
|
|
| Serving a delft model needs the parser's `delft` extra, and `sciencebeam-trainer-delft` at 1.0.3 or |
| above. This is a floor rather than a recommendation: these models were trained with the pytorch-crf |
| implementation, which earlier versions of the architecture cannot build — they hardcode delft's |
| ported `ChainCRF`, whose weights go by other names — and those versions also run the model over |
| padded positions, which changes what it predicts. |
|
|
| Pin a commit sha rather than `main`, so a change to what a config serves is a visible diff. A |
| `HEAD` request against any `resolve/main` URL here returns the current one as `X-Repo-Commit`. |
|
|
| These models were selected against GROBID 0.9.0 on an internal benchmark of open-access corpora. |
| How much they help depends on the corpus and on what is being scored, so it is worth measuring on |
| your own documents rather than taking a number from ours. |
|
|
| ## Training data |
|
|
| Two kinds of input, listed per model in `training.yml`: |
|
|
| - **GROBID 0.9.0's own training data** for that model — the only input the `0.9.0` version applies |
| to, and the source of the feature layout, label set and CRF template the models share. |
| - **Training data generated by `sciencebeam-parser`** from the PDF and JATS of open-access |
| articles, from two corpora: |
| - [SciELO Preprints](https://preprints.scielo.org/) (DOI prefix `10.1590`), CC-BY 4.0 as recorded |
| per article; |
| - [Open Research Europe](https://open-research-europe.ec.europa.eu/) (DOI prefix `10.12688`), |
| which publishes under CC-BY 4.0 as platform policy. |
| |
| The generated training data is not published, and neither is the collection of source articles it |
| was generated from: both are derived from those corpora and redistribution is governed by their |
| terms. So `training.yml` identifies each generated input by the path it was written to rather than |
| by a name that resolves. Retraining from scratch needs that generation step rerunning rather than a |
| download; `training.yml` records the inputs and hyperparameters it would need. |
|
|
| ## Attribution |
|
|
| Derived from [GROBID](https://github.com/kermitt2/grobid) (Apache-2.0) — its model architecture, |
| feature layout, label sets, CRF templates and training corpus — with additional training data |
| generated from the CC-BY 4.0 articles of SciELO Preprints and Open Research Europe, as named above. |
| Modified in that the models are retrained on that combined corpus; they are not GROBID's published |
| models and are not endorsed by the GROBID project. |
|
|
