README.md

---
license: cc-by-4.0
library_name: transformers
pipeline_tag: text-generation
base_model: google/flan-t5-base
widget:
  - text: "cod: fever | age: unknown | sex: unknown"
    example_title: "Cause text with missing metadata"
  - text: "cod: pneumonia | age: 65 | sex: female"
    example_title: "Cause text with age and sex"
language:
  - en
  - nl
  - da
  - es
tags:
  - icd
  - historical-demography
  - digital-humanities
  - multi-label
  - text2text-generation
---

# codLLM-base-deploy

A fine-tuned **FLAN-T5-base** model that maps historical free-text causes of death to **ICD10h codes**.
This is the deployment checkpoint from our June 2026 bachelor's thesis at the Technical University of Denmark (DTU).
The main model series for the forthcoming publication will be released separately.

[Thesis (PDF)][thesis] · [Source code][code] · [codLLM organization](https://huggingface.co/codllm)

| Property | Description |
|---|---|
| Task | Sequence-to-sequence, multi-label ICD10h coding |
| Base model | [google/flan-t5-base](https://huggingface.co/google/flan-t5-base) |
| Training input | Cause-of-death text, age, and sex |
| Output | One or more comma-separated ICD10h codes |
| Intended use | Historical mortality research with expert review |
| Authors | Lucas E. B. Mortensen and Elias Dahl Lunøe |

## Quick start

Install the inference dependencies; Transformers 5.1.0 is the version recorded with this checkpoint:

```bash
uv add "transformers==5.1.0" torch sentencepiece
```

```python
import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_id = "codllm/codllm-base-deploy"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
model.eval()

text = "cod: fever | age: unknown | sex: unknown"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=32,
        do_sample=False,
        num_beams=1,
    )

prediction = tokenizer.decode(outputs[0], skip_special_tokens=True)
codes = [code.strip() for code in prediction.split(",") if code.strip()]
print(codes)
```

This is a minimal CPU example with an illustrative input. It does not compute uncertainty or reproduce the thesis evaluation.
The explicit generation limit overrides the bundled `generation_config.json` default of `max_length=20`.
The training pipeline raises its target budget when needed for longer multi-code targets; check for truncation in such cases.

## Input and output

Use the deployment training format, with all three fields in this order:

```text
cod: fever | age: unknown | sex: unknown
```

- Preserve the `cod: `, `age: `, and `sex: ` prefixes and the ` | ` separators.
- Use the literal `unknown` for missing or unmappable metadata; keep the field in the input.
- Follow the project's [input preprocessing](https://github.com/murrodroid/codllm/blob/main/src/codllm/input/transform.py)
  for age and sex normalization. The quick start assumes text has already been formatted.
- COD-only inputs were studied in separate experiments. Their bare-string format should not be treated as
  interchangeable with this checkpoint's metadata input or as reproducing its reported scores.

Predictions are comma-separated code strings. Evaluation treats codes as an unordered set.
Use the included [ICD10h 2024 masterlist](./ICD10h_Masterlist_2024.tsv) for code descriptions and validity checks;
free generation does not enforce membership in the masterlist.

## Training and data

The thesis describes 1,525,048 historical records from Amsterdam, Copenhagen, Madrid, Belgium, and Ipswich,
plus English reference strings. Raw archival records are subject to a data-use agreement and are not distributed here.
The model uses masterlist pretraining followed by historical fine-tuning.

The project's [reviewed language inventory][languages] identifies Dutch (including Flemish/Belgian Dutch), Danish,
Spanish, and English. The Belgian dataset owner's clarification supersedes the thesis's French/Dutch description.
Historical Latin expressions can occur within records; this is not a separately evaluated Latin capability.

<details>
<summary>Deployment training recipe</summary>

Settings below are reported in the thesis, Sections 6.1–6.2 and Table 6.4.

| Setting | Value |
|---|---|
| Historical split | Random record-level 90/5/5; data seed 333 |
| Input fields | COD + age + sex |
| Rare-code upsampling floor | 200 |
| Synthetic multi-COD ratio | 0.30 |
| Base perturbation rate | 0.50 |
| Learning rate / schedule | 2.5e-5 / constant with warmup |
| Warmup ratio | 0.10 |
| Effective batch size | 256 |
| Maximum fine-tuning epochs | 80 |
| Early-stopping patience | 10 epochs |
| Checkpoint selection | Highest validation macro-F1 |
| Source / target token budgets | 256 / 32 by default; target budget raised when necessary |

These are the thesis recipe settings. Current publication experiment specifications describe later work.

</details>

## Evaluation

The following are **thesis-reported results for codLLM-base-deploy**, Table 7.1, on historical records with
English reference strings excluded. They have not been recomputed for this card update.

| Metric | Full historical test | Unseen-string slice |
|---|---:|---:|
| Records | 76,249 | 5,862 |
| Exact-match accuracy | 0.985 ± 0.001 | 0.843 ± 0.009 |
| Jaccard | 0.989 ± 0.001 | 0.884 ± 0.007 |
| Macro-F1 | 0.749 | 0.625 |
| Micro-F1 | 0.987 ± 0.001 | 0.902 ± 0.006 |

**Split context matters:** 92.19% of historical test records share their exact cause text with a training record
(thesis Table 5.3). The unseen-string slice excludes cause strings appearing verbatim in training or validation.
It is a subset of the same record-level split, not an independently grouped or source-held-out test.
Rare causes and near-duplicate wording affect its interpretation.

The ± values are 95% cluster-bootstrap half-widths over distinct input strings, using 5,000 resamples.
They describe a fixed model and test set, not variation across training seeds. Macro-F1 is reported without an interval.

The thesis's leave-one-source-out results use **five separately trained holdout checkpoints** (Section 7.4).
They are not transfer scores for this uploaded checkpoint and do not establish cross-lingual performance.

## Uncertainty and limitations

The thesis ranks predictions by mean token entropy to study selective prediction.
On the unseen-string slice, retaining the most confident 80% raises exact-match accuracy from 0.843 to 0.925
(Figure 7.3). This is a rank-based result, not a calibrated probability or a universal entropy threshold.
Evaluate any acceptance rule on representative target data before using it to defer records for review.

- Performance depends on the archive's wording, label conventions, and representation in training.
- Rare codes and unfamiliar cause strings are harder; a correct broad category can still have an incorrect sub-code.
- Confident errors occur, and generated codes should be checked against the reference vocabulary.
- Training labels come from human-coded registers and can contain errors. The thesis does not establish performance
  across repeated training seeds or arbitrary new archives.

Use the model as support for historical coding and expert review. Clinical diagnosis and contemporary medical coding
were not evaluated.

## Citation

**Please credit Lucas E. B. Mortensen and Elias Dahl Lunøe when using this model, and cite the thesis in research publications.**
Include the model name and revision used. [CITATION.cff](./CITATION.cff) provides machine-readable citation metadata.
This citation request is not an additional license condition.

Lucas E. B. Mortensen and Elias Dahl Lunøe (2026).
*Automated Historical Cause-of-Death Coding Using Fine-Tuned Language Models.*
Bachelor's thesis, DTU Compute, Technical University of Denmark. [Read the thesis][thesis].

<details>
<summary>BibTeX</summary>

```bibtex
@thesis{mortensen_lunoe_2026_codllm,
  author = {Mortensen, Lucas E. B. and Lunøe, Elias Dahl},
  title = {Automated Historical Cause-of-Death Coding Using Fine-Tuned Language Models},
  institution = {Technical University of Denmark},
  type = {Bachelor's thesis},
  year = {2026},
  month = jun,
  url = {https://huggingface.co/spaces/codllm/README/resolve/main/codLLM_Thesis.pdf}
}
```

</details>

## License and attribution

The codLLM model contributions and this model card are offered under **[CC BY 4.0](./LICENSE)**.
Commercial and non-commercial use, modification, and redistribution are permitted under the applicable terms.
When sharing licensed material or adaptations, credit **Lucas E. B. Mortensen and Elias Dahl Lunøe**, link to this
model and the license, retain supplied notices, and indicate changes as required by CC BY 4.0.

This checkpoint modifies Google's [FLAN-T5-base](https://huggingface.co/google/flan-t5-base) through codLLM training.
The upstream **[Apache-2.0](./LICENSE-APACHE)** terms and notices remain applicable; the CC BY declaration does not
replace them. These are licenses for different contributions, not a choice between two licenses for the whole model.
The [project software][code] uses Apache-2.0.

See [NOTICE](./NOTICE) and [LICENSING.md](./LICENSING.md) for scope, attribution, and third-party exclusions.
Raw archival data and ICD10h reference materials retain their own terms.
Earlier model revisions remain available under their original Apache-2.0 terms; this update does not revoke prior grants.
The standard licenses do not create a universal citation obligation merely for running the model or publishing outputs.

[thesis]: https://huggingface.co/spaces/codllm/README/resolve/main/codLLM_Thesis.pdf
[code]: https://github.com/murrodroid/codllm
[languages]: https://github.com/murrodroid/codllm/blob/deploy/data/curation/source_languages.toml