OrthophoNER

Model description

This model is a CamemBERT-based token classifier for French biomedical and clinical named entity recognition, specialized in the entities relevant to Speech and Language Therapy.

• Base model: camembert-base
• Architecture: CamembertForTokenClassification (CamemBERT encoder + linear token classifier)
• Output: 82 logits, one per label
• Label scheme: BILOU (B-, I-, L-, U-, for Beginning, Inside, Last, Unit) over 20 entity types, plus O and O-pad
• Classification type: multi-label. The original model applies a sigmoid to each logit and uses a threshold of 0.5. The Hugging Face model returns raw logits; apply torch.sigmoid and threshold at 0.5 to reproduce the original predictions.
• Tokenizer: camembert-base (SentencePiece, fast tokenizer)

O-pad is the padding label used during training and can be ignored for real tokens.

Labels

The model is trained to annotate 20 entity types, covering the following 14 entity types corresponding to UMLS semantic types

Label	Entity type
ACTI	Activity
ANAT	Anatomy
CHEM	Chemical
CONC	Concept
DEVI	Device
DISO	Disorder
GEOG	Geographic Entity
LIVB	Living Being
OBJC	Object
OCCU	Occupation
ORGA	Organization
PHEN	Phenomenon
PHYS	Physiology
PROC	Procedure

and 6 additional entity types:

Label	Entity type
date	date
dose	dose
duree	duration
frequence	frequence
moment	moment of the day
valeur	value

Each entity type has four BILOU labels, for example B-ANAT, I-ANAT, L-ANAT, U-ANAT, plus the two non-entity labels O and O-pad.

Intended uses

• French biomedical and clinical named entity recognition for Speech and Language Therapy.
• Information extraction and research on French medical text.
• Fine-tuning or evaluation on domain-specific data.

This model is not a medical device. Do not use it for clinical decision-making without independent validation.

Limitations

• The model is multi-label: a token can receive several labels at the same time. Use a threshold appropriate to your application (the original pipeline uses 0.5).
• Predictions are BILOU tags. They must be decoded into entity spans, and nested or overlapping entities may require application-specific post-processing.
• The model was trained on a specific corpus and may not generalise to other domains, text genres or annotation guidelines.
• CamemBERT is case-sensitive; preserve the original casing when tokenizing.

Usage

Install the dependencies:

pip install transformers torch

Load the model and tokenizer:

from transformers import AutoModelForTokenClassification, AutoTokenizer

model_id = "cea-list-ia/orthophoner"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)
model.eval()

Predict labels for each word

CamemBERT tokenizes words into SentencePiece subwords. The training pipeline aggregates subword predictions back to words, so the example below does the same: it keeps the prediction of the first subword of each word.

import torch

SPECIAL_LABELS = {"O", "O-pad"}


def predict_words(text, threshold=0.5):
    inputs = tokenizer(text, return_tensors="pt", return_offsets_mapping=True)
    offsets = inputs.pop("offset_mapping")[0]

    with torch.no_grad():
        logits = model(**inputs).logits[0]

    probabilities = torch.sigmoid(logits)
    predictions = (probabilities >= threshold).cpu()
    tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])

    words = []
    for token, offset, row in zip(tokens, offsets, predictions):
        if token in tokenizer.all_special_tokens:
            continue

        if token.startswith("▁") or not words:
            words.append(
                {
                    "text": token.replace("▁", ""),
                    "start": int(offset[0]),
                    "end": int(offset[1]),
                    "labels": [],
                }
            )
        else:
            words[-1]["text"] += token.replace("▁", "")
            words[-1]["end"] = int(offset[1])

        for index in row.nonzero(as_tuple=False).flatten().tolist():
            label = model.config.id2label[index]
            if label not in SPECIAL_LABELS and label not in words[-1]["labels"]:
                words[-1]["labels"].append(label)

    return words


text = "Le patient présente une douleur thoracique depuis deux jours."
for word in predict_words(text):
    print(f"{word['text']:<15} {word['labels'] or ['O']}")

Example output (may vary):

Le              ['O']
patient         ['U-LIVB']
présente        ['O']
une             ['O']
douleur         ['B-DISO', 'U-DISO']
thoracique      ['U-ANAT', 'L-DISO']
depuis          ['B-duree']
deux            ['I-duree']
jours.          ['L-duree']

Decode BILOU labels into entity spans

The following helper decodes the word-level BILOU labels into character spans. It keeps one BILOU tag per entity type and word, with priority B > I > L > U. This is a simple flat decoder; adapt it if you need nested or overlapping entities.

def decode_bilou(words):
    spans = []
    open_spans = {}
    priority = {"B": 0, "I": 1, "L": 2, "U": 3}

    for i, word in enumerate(words):
        tags = {}
        for tag in word["labels"]:
            prefix, separator, label = tag.partition("-")
            if not separator:
                continue
            if label not in tags or priority[prefix] < priority[tags[label]]:
                tags[label] = prefix

        # Close spans that are not continued by the current word.
        for label in list(open_spans):
            if tags.get(label) not in {"I", "L"}:
                start = open_spans.pop(label)
                spans.append((words[start]["start"], words[i - 1]["end"], label))

        for label, prefix in tags.items():
            if prefix == "U":
                spans.append((word["start"], word["end"], label))
            elif prefix == "B":
                if label in open_spans:
                    start = open_spans.pop(label)
                    spans.append((words[start]["start"], words[i - 1]["end"], label))
                open_spans[label] = i
            elif prefix == "I":
                if label not in open_spans:
                    open_spans[label] = i
            elif prefix == "L":
                if label in open_spans:
                    start = open_spans.pop(label)
                    spans.append((words[start]["start"], word["end"], label))
                else:
                    spans.append((word["start"], word["end"], label))

    for label, start in open_spans.items():
        spans.append((words[start]["start"], words[-1]["end"], label))

    return spans


text = (
    "Le patient présente une douleur thoracique depuis deux jours "
    "et prend de l'aspirine 100 mg."
)
words = predict_words(text)
for start, end, label in decode_bilou(words):
    print(f"{label:8s} {text[start:end]!r} ({start}:{end})")

Example output (may vary):

LIVB     'patient' (3:10)
ANAT     'thoracique' (32:42)
DISO     'douleur thoracique' (24:42)
duree    'depuis deux jours' (43:60)
CHEM     "l'aspirine" (73:83)

Training data

The label mapping matches some of the QUAERO tag set and some additional entities (20 entity types with the BILOU scheme). A first model was trained based on QUAERO and more than 400 annotated Speech and Language Therapy Assessment Reports (pre-annotated automatically and manually corrected). Since this first model is trained on sensitive data, we cannot provide it directly. Instead, we used this first model to automatically annotate the OrthoCorpus dataset and trained this Orthophoner model using both QUAERO and this annotated OrthoCorpus.