OpenMed

OpenMed GLiNER2.5 ProceduresTests Small v1

OpenMed on Hugging Face OpenMed SDK on GitHub pip install openmed License Apache-2.0

OpenMed's open model for finding procedures and tests in biomedical text. It returns diagnostic, laboratory and therapeutic procedure mentions in English biomedical text, labeled by type, with exact character offsets and a confidence score. Trained and evaluated by OpenMed. Apache-2.0, 73.9M parameters, runs on CPU.

Developed by OpenMed
Task Procedure mention extraction, three labels: diagnostic procedure, laboratory procedure, therapeutic or preventive procedure (the prompts it was trained with)
Output text, start, end, label, score; text[start:end] is the exact span
Parameters 73.9M
Default threshold 0.4
Language English
License Apache-2.0

About OpenMed

OpenMed builds open models and open-source tools for clinical and biomedical text that run on your own hardware.

Quick start

pip install -U "openmed[gliner2]"  # OpenMed 3.2 or later
from openmed import OpenMedConfig, analyze_text

text = (
    "Patients underwent screening colonoscopy, and polyps "
    "larger than 10 mm were removed by endoscopic mucosal "
    "resection."
)
result = analyze_text(
    text,
    model_id="OpenMed/GLiNER2.5-ProceduresTests-Small-v1",
    labels=[
        "diagnostic procedure",
        "laboratory procedure",
        "therapeutic or preventive procedure",
    ],
    threshold=0.05,
    confidence_threshold=0.4,
    max_length=150,
    config=OpenMedConfig(use_medical_tokenizer=False),
)
for e in result.entities:
    print(e.start, e.end, e.text, f"[{e.label}]", f"{e.confidence:.2f}")
19 40 screening colonoscopy [PROCEDURE] 0.88
87 115 endoscopic mucosal resection [therapeutic or preventive procedure] 0.95

The settings

  • labels holds the three prompts the model was trained with, one per label. Keep them as written and pass all of them: the evaluation below ran with all three together.
  • Each span comes back with the prompt that found it as its label, except diagnostic procedure, which the SDK reports under its standard label PROCEDURE. laboratory procedure and therapeutic or preventive procedure come back as written.
  • threshold=0.05 lets the model propose spans down to 0.05, and confidence_threshold=0.4 keeps those scoring 0.4 or more. This is the decoding behind the evaluation below. Raise confidence_threshold for fewer, surer spans.
  • max_length=150 makes analyze_text group sentences into pieces of at most 600 characters, the window size used in training and evaluation. It only matters for long texts.
  • use_medical_tokenizer=False keeps the model's own span boundaries.

Long abstracts, many abstracts, GPU

  • Pass a whole abstract: analyze_text splits it into sentences and returns offsets into the original text.
  • The model loads once and stays cached, so call analyze_text in a loop for many texts.
  • On a GPU, pass config=OpenMedConfig(device="cuda", use_medical_tokenizer=False).

Evaluation

MedMentions, 200 held-out abstracts: PubMed abstracts with 1,017 human-annotated procedure mentions (216 diagnostic, 189 laboratory and 612 therapeutic or preventive; three of the seven semantic types of the UMLS Procedures semantic group, each kept as its own label). The model was trained on 1,000 other MedMentions abstracts, both thresholds were chosen on 200 more, and these 200 were scored once at the end, with the model, thresholds and scoring code frozen first. A prediction counts only if its character span and its label both match exactly. Scores pool all labels (micro-average). All abstracts come from the official MedMentions training partition; the official test set was not used, so these numbers are not comparable to published MedMentions results.

Model Precision Recall F1
OpenMed GLiNER2.5 ProceduresTests Small v1 (this model), threshold 0.4 50.21 46.02 48.02
Before OpenMed fine-tuning: fastino/gliner2.5-small-v1, threshold 0.3 9.31 19.37 12.57

OpenMed's fine-tuning added +35.45 F1 points over all labels (95% CI +31.33 to +39.62, paired bootstrap over the 200 abstracts, 2,000 resamples). Each model runs at its own best threshold, chosen on the calibration abstracts before testing. At the same threshold, 0.5, the scores are 47.56 for this model and 11.45 before fine-tuning.

By label

Label Prompt Mentions Precision Recall F1 before fine-tuning F1 F1 change (95% CI)
Diagnostic_Procedure diagnostic procedure 216 46.63 44.91 8.26 45.75 +37.49 (+29.24 to +46.33)
Laboratory_Procedure laboratory procedure 189 32.67 25.93 11.00 28.91 +17.90 (+12.41 to +23.92)
Therapeutic_Procedure therapeutic or preventive procedure 612 56.10 52.61 18.73 54.30 +35.57 (+29.61 to +41.84)

Laboratory procedures are the weak label: 28.91 F1, with recall 25.93. On these abstracts the model finds about one laboratory procedure mention in four, and 67% of the spans it labels laboratory do not match an annotated laboratory procedure exactly. Review its laboratory spans before relying on them. All three labels improved with fine-tuning: each label's 95% interval is above zero.

The OpenMed GLiNER2.5 Small models

Same base model, same MedMentions source, each scored on its own held-out abstracts. ProceduresTests has three labels and its F1 pools them; the others have one label each.

Model Finds Prompt Default threshold F1 before fine-tuning F1
OpenMed GLiNER2.5 DrugChemical Small v1 drugs and chemicals chemical or drug 0.4 45.09 67.43
OpenMed GLiNER2.5 Anatomy Small v1 anatomy anatomy 0.3 25.71 61.85
OpenMed GLiNER2.5 ProceduresTests Small v1 (this model) diagnostic, laboratory and therapeutic procedures diagnostic procedure, laboratory procedure, therapeutic or preventive procedure 0.4 12.57 48.02

Examples

Real outputs of this model through the OpenMed SDK, with the quick-start settings (default threshold 0.4), on CPU. The sentences are synthetic, written in the style of PubMed abstracts for this card. Extracted spans are in bold with their label (diagnostic, laboratory, therapeutic) and score.

  1. Screening and removal: Patients underwent screening colonoscopydiagnostic 0.88, and polyps larger than 10 mm were removed by endoscopic mucosal resectiontherapeutic 0.95.
  2. Laboratory panel: Complete blood countlaboratory 0.45, serum creatininelaboratory 0.58 and liver function testsdiagnostic 0.55 were obtained at baseline and every four weeks.
  3. Molecular testing: SARS-CoV-2 infection was confirmed by reverse transcription PCRlaboratory 0.53 of nasopharyngeal swabslaboratory 0.52.
  4. Cardiac workup: Transthoracic echocardiographydiagnostic 0.99 and cardiac magnetic resonance imagingdiagnostic 1.00 were performed before percutaneous coronary interventiontherapeutic 0.98.
  5. Breast cancer treatment: After neoadjuvant chemotherapytherapeutic 0.85, patients underwent breast-conserving surgerytherapeutic 0.71 followed by whole-breast radiotherapytherapeutic 0.88.
  6. Pathology: Immunohistochemistry for HER2 and Ki-67 was performed on core needle biopsy specimens. (nothing extracted)
  7. Prevention: Influenza vaccination coverage rose from 41% to 63% after the reminder program. (nothing extracted)
  8. Dialysis unit: Patients on maintenance hemodialysis had blood cultureslaboratory 0.92 drawn at each febrile episode.
  9. Lung function: Spirometry and a six-minute walk test were repeated 12 months after lung transplantationtherapeutic 0.93.
  10. No procedures: Participants reported their sleep duration, dietary habits and daily screen time in an online survey. (nothing extracted)

A longer abstract (829 characters, 11 spans) in one analyze_text call:

BACKGROUND: Early detection of colorectal cancer relies on screening, but adherence to colonoscopydiagnostic remains low.
METHODS: We enrolled 1,200 adults aged 50 to 75 years. Participants completed a fecal immunochemical testlaboratory, and those with a positive result were referred for colonoscopydiagnostic. Biopsy specimens were examined by histopathology, and KRAS mutation status was determined by polymerase chain reactionlaboratory. Hemoglobinlaboratory and serum ferritinlaboratory were measured in all participants.
RESULTS: Colonoscopydiagnostic detected advanced adenomas in 84 participants, all of whom underwent polypectomytherapeutic. Eleven patients with invasive cancer were treated with laparoscopic colectomytherapeutic, and six received adjuvant chemotherapytherapeutic.
CONCLUSIONS: A stepwise screening program combining stool testing and colonoscopy increased detection and allowed early surgical treatmenttherapeutic.

Good to know

  • One call returns all three types: example 4 gives Transthoracic echocardiography and cardiac magnetic resonance imaging as diagnostic and percutaneous coronary intervention as therapeutic; example 8 gives blood cultures as laboratory.
  • Laboratory procedures are where it misses most, as in the evaluation: example 6 returns nothing for Immunohistochemistry, example 2 labels liver function tests diagnostic, and the longer abstract misses stool testing. It also returns nasopharyngeal swabs, the specimen, as a laboratory procedure in example 3.
  • It misses procedures of the other types too: core needle biopsy (example 6), Influenza vaccination (example 7), maintenance hemodialysis (example 8), Spirometry and the six-minute walk test (example 9).
  • Span edges can be off by a word: blood count without Complete (example 2), and early surgical treatment in the longer abstract.
  • Example 10, with no procedures, returns nothing.

Limitations

  • Trained and tested on English PubMed abstracts (MedMentions). It has not been validated on hospital notes or other clinical text; the examples above are synthetic.
  • Laboratory procedures are the weak label: 28.91 F1 and 25.93 recall on the held-out abstracts, so most laboratory procedure mentions are missed.
  • It covers three of the seven types of the UMLS Procedures group. Educational activities, health care activities, molecular biology research techniques and research activities are not labeled.
  • It finds procedure mentions only. It does not extract test results or values, billing codes or doses, and does not say whether a procedure was done, planned or refused.
  • No normalization: spans are not mapped to CPT, LOINC, SNOMED CT or UMLS codes.
  • Scores rank spans; they are not calibrated probabilities.
  • The base model's pretraining data is not public, so whether it saw these abstracts is unknown.
  • It finds text spans. It does not give medical advice and is not a medical device.

Built on

OpenMed fine-tuned this model from fastino/gliner2.5-small-v1 by Fastino (Apache-2.0), built with the open-source GLiNER2 library. Training and evaluation data: the MedMentions corpus of PubMed abstracts annotated with UMLS concepts by Mohan and Li (2019, CC0); mentions of three types of the UMLS Procedures semantic group (Diagnostic Procedure, Laboratory Procedure and Therapeutic or Preventive Procedure) were kept as three labels. Thank you to Fastino and to the MedMentions authors for releasing their work openly.

This model is released under Apache-2.0. No corpus text, UMLS concept IDs or terminology files are included in this repository.

Citation

@misc{panahi2025openmedner,
  title={OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers for Biomedical NER Across 12 Public Datasets},
  author={Maziyar Panahi},
  year={2025},
  eprint={2508.01630},
  archivePrefix={arXiv}
}

@inproceedings{mohan2019medmentions,
  title={MedMentions: A Large Biomedical Corpus Annotated with {UMLS} Concepts},
  author={Mohan, Sunil and Li, Donghui},
  booktitle={Proceedings of the 2019 Conference on Automated Knowledge Base Construction (AKBC)},
  year={2019}
}

@misc{zaratiana2025gliner2,
  title={GLiNER2: An Efficient Multi-Task Information Extraction System with Schema-Driven Interface},
  author={Urchade Zaratiana and Gil Pasternak and Oliver Boyd and George Hurn-Maloney and Ash Lewis},
  year={2025},
  eprint={2507.18546},
  archivePrefix={arXiv}
}

OpenMed · Hugging Face · GitHub · PyPI · X

Downloads last month
8
Safetensors
Model size
73.9M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for OpenMed/GLiNER2.5-ProceduresTests-Small-v1

Finetuned
(6)
this model

Dataset used to train OpenMed/GLiNER2.5-ProceduresTests-Small-v1

Papers for OpenMed/GLiNER2.5-ProceduresTests-Small-v1

Evaluation results

  • F1 (exact span, micro-averaged over the three labels, threshold 0.4) on MedMentions (200 held-out abstracts)
    self-reported
    48.020
  • F1 Diagnostic_Procedure (exact span, threshold 0.4) on MedMentions (200 held-out abstracts)
    self-reported
    45.750
  • F1 Laboratory_Procedure (exact span, threshold 0.4) on MedMentions (200 held-out abstracts)
    self-reported
    28.910
  • F1 Therapeutic_Procedure (exact span, threshold 0.4) on MedMentions (200 held-out abstracts)
    self-reported
    54.300