---
license: apache-2.0
library_name: pytorch
pipeline_tag: text-classification
tags:
- language-identification
- onnx
- byte-level
datasets:
- baobabtech/millid-dataset
metrics:
- f1
- accuracy
model-index:
- name: millid-tiny-v1.5
  results:
  - task:
      type: text-classification
      name: Language identification
    dataset:
      name: MillID test split (the full build)
      type: baobabtech/millid-dataset
      split: test
    metrics:
    - type: f1
      name: Macro F1
      value: 0.9554
    - type: accuracy
      name: Top-1 accuracy
      value: 0.9587
  - task:
      type: text-classification
      name: Language identification
    dataset:
      name: UDHR-LID, the languages MillID knows
      type: baobabtech/millid-dataset
      config: udhr
      split: test
    metrics:
    - type: f1
      name: Macro F1
      value: 0.8764
    - type: accuracy
      name: Top-1 accuracy
      value: 0.8492
---

# MillID tiny v1.5: MultiLingual Language IDentifier

A byte-level, encoder-only transformer that names the language of a passage, from a table cell to a paragraph: **1,641 language-script labels** (ISO 639-3 + script, e.g. `eng_Latn`, `amh_Ethi`). 2.9M parameters, trained from scratch. Apache-2.0.

## The MillID models

Three sizes, this one in bold. Macro F1 on `test` and `udhr`, the sets below:

| Model | Parameters | ONNX | Test F1 | UDHR F1 | Runs on |
|---|---|---|---|---|---|
| **MillID tiny v1.5** | 2.9M | 11 MB | 0.9554 | 0.8764 | CPU, web page |
| [MillID medium v1](https://huggingface.co/baobabtech/millid-medium-v1) | 115M | 455 MB | 0.9576 | 0.8780 | GPU |
| [MillID large v1.5](https://huggingface.co/baobabtech/millid-large-v1.5) | 193M | 764 MB | 0.9646 | 0.8812 | GPU |

All three, language by language, against GlotLID and Meta's fastText: the [leaderboard](https://huggingface.co/baobabtech/millid). GlotLID, restricted to MillID's labels, scores 0.9755 and 0.8873.

## Use it

No server and no PyTorch: the model runs in your process from `onnx/model.onnx`, in Python or JavaScript, and the two give the same answers (one encoder, ported line for line and tested against the same fixtures in [baobab-tech/millid](https://github.com/baobab-tech/millid)).

```bash
pip install "millid[cpu] @ git+https://github.com/baobab-tech/millid.git"   # [gpu] for CUDA
```

```python
from millid import MillID

model = MillID.from_pretrained("baobabtech/millid-tiny-v1.5")
model.predict("Lave men ou ak savon anvan ou manje.", k=3)
# [Prediction(label='hat_Latn', language='Haitian Creole', probability=...), ...]
model.predict(["one text", "another"], k=3)  # one list of answers per text
model.identify_document(long_text)           # any length: its languages and segments
```

```bash
millid predict "Lave men ou ak savon anvan ou manje." --model baobabtech/millid-tiny-v1.5 -k 3
```

JavaScript, in Node or a browser (`js/` in the same repository; not on npm yet):

```js
import { MillID } from "@baobabtech/millid";   // "@baobabtech/millid/web" in a browser, with { ort }
const model = await MillID.fromPretrained("baobabtech/millid-tiny-v1.5");
await model.predict("Lave men ou ak savon anvan ou manje.", 3);
```

`millid.py` in this repository is the PyTorch training code, which reads `model.safetensors` to fine-tune or export.

The ONNX graph takes, for each UTF-8 byte of a passage cut to 255 bytes after a CLS token, `ids`, `script` and `charpos` (int64, `[batch, bytes]`), `ngrams` (int64, `[batch, bytes, 3]`) and `mask` (bool, `[batch, bytes]`), and returns `logits` (float32, `[batch, labels]`). `millid.encode_batch` and the JavaScript `encodeBatch` build them; script ids come from a frozen table (Unicode 15.0), so every runtime gives the same ones.

## Architecture

UTF-8 bytes (after Unicode NFKC), each embedded as byte + Unicode script + byte-in-character + hashed character 2-, 3- and 4-grams (4,096 buckets) -> depthwise convolution stem (kernel 5) -> strided convolution, 1 bytes per token -> 3 pre-norm blocks (RMSNorm, attention with RoPE, 3 heads, SwiGLU; d=192) -> masked mean pooling -> language head, each language's vector the sum of its own, its language's and its script's. Up to 256 bytes per passage. See `ARCHITECTURE.md`.

## Evaluation against GlotLID and fastText

`validation` chose the checkpoint; `test` is split by document from training; `udhr` is the Universal Declaration of Human Rights, labelled by its translators, over the languages MillID knows, and `udhr-all` over all of them. [GlotLID](https://huggingface.co/cis-lmu/glotlid) and Meta's [fastText language identifier](https://huggingface.co/facebook/fasttext-language-identification) (NLLB, 218 labels) are each restricted to MillID's labels, their best answer among them: the like-for-like comparison. fastText's answers are mapped onto MillID's spelling (Simplified and Traditional Han as `Hani`, `pes`/`prs` as `fas`, `est` as `ekk`). GlotLID's own labels came from training on data that includes UDHR and labelled FinePDFs and FineWeb-2, so its scores on `udhr` and on those sources are optimistic. fastText cannot answer most of the 1,600 languages here, so `test-common` and `udhr-common` score every system on the passages whose language all of them know: the head-to-head on equal terms.

Macro F1 on each set, every system on the same passages; GlotLID and fastText restricted to MillID's labels:

| Set | Languages | MillID | GlotLID | fastText |
|---|---|---|---|---|
| `test` | 1,621 | 0.9554 | 0.9755 | 0.0659 |
| `test-common` | 213 | 0.9513 | 0.9846 | 0.9173 |
| `udhr` | 358 | 0.8764 | 0.8873 | 0.3499 |
| `udhr-common` | 170 | 0.9513 | 0.9659 | 0.9329 |
| `udhr-all` | 429 | 0.7149 | 0.7245 | 0.2797 |

MillID on each set:

| Set | Top-1 | Top-3 | Top-5 | Macro P | Macro R | Macro F1 | Median F1 |
|---|---|---|---|---|---|---|---|
| `validation` | 0.9578 | 0.9889 | 0.9933 | 0.9539 | 0.9591 | 0.9551 | 0.9822 |
| `test` | 0.9587 | 0.9890 | 0.9932 | 0.9550 | 0.9588 | 0.9554 | 0.9825 |
| `test-common` | 0.9396 | 0.9857 | 0.9909 | 0.9709 | 0.9353 | 0.9513 | 0.9726 |
| `udhr` | 0.8492 | 0.9117 | 0.9257 | 0.9148 | 0.8736 | 0.8764 | 0.9907 |
| `udhr-common` | 0.9123 | 0.9606 | 0.9655 | 0.9794 | 0.9393 | 0.9513 | 1.0000 |
| `udhr-all` | 0.7274 | 0.7809 | 0.7928 | 0.7340 | 0.7290 | 0.7149 | 0.9649 |

Top-1 accuracy by passage length:

| Set | System | Short | Sentence | Paragraph |
|---|---|---|---|---|
| `validation` | MillID | 0.9312 | 0.9788 | 0.9850 |
| `test` | MillID | 0.9333 | 0.9797 | 0.9833 |
| `test` | GlotLID | 0.9674 | 0.9858 | 0.9912 |
| `test` | fastText | 0.2764 | 0.2506 | 0.1959 |
| `test-common` | MillID | 0.9125 | 0.9685 | 0.9725 |
| `test-common` | GlotLID | 0.9609 | 0.9885 | 0.9949 |
| `test-common` | fastText | 0.9125 | 0.9581 | 0.9655 |

### Per language: test

F1 language by language, GlotLID and fastText restricted to MillID's labels: against GlotLID, MillID is better on **103** languages, worse on **869** and within 0.01 on 649; against fastText, MillID is better on **1616** languages, worse on **0** and within 0.01 on 5. Every language, with precision, recall and the unrestricted scores, is in `reports/test/leaderboard.csv`.

The 20 languages with the most passages:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `fra_Latn` | French | 1652 | 0.929 | 0.928 | 0.808 |
| `eng_Latn` | English | 1444 | 0.917 | 0.859 | 0.648 |
| `spa_Latn` | Spanish | 1136 | 0.930 | 0.951 | 0.574 |
| `tam_Taml` | Tamil | 754 | 0.997 | 0.993 | 0.965 |
| `arb_Arab` | Arabic (Modern Standard) | 745 | 0.814 | 0.785 | 0.512 |
| `por_Latn` | Portuguese | 657 | 0.947 | 0.949 | 0.755 |
| `kan_Knda` | Kannada | 601 | 0.969 | 0.993 | 0.922 |
| `zul_Latn` | Zulu | 594 | 0.846 | 0.896 | 0.539 |
| `swh_Latn` | Swahili | 592 | 0.848 | 0.916 | 0.346 |
| `mar_Deva` | Marathi | 589 | 0.965 | 0.990 | 0.926 |
| `tso_Latn` | Tsonga | 581 | 0.965 | 0.956 | 0.425 |
| `hun_Latn` | Hungarian | 576 | 0.987 | 0.999 | 0.232 |
| `afr_Latn` | Afrikaans | 569 | 0.967 | 0.985 | 0.860 |
| `tel_Telu` | Telugu | 555 | 0.991 | 0.994 | 0.951 |
| `rus_Cyrl` | Russian | 528 | 0.932 | 0.949 | 0.462 |
| `nbl_Latn` | South Ndebele | 528 | 0.903 | 0.949 | 0.000 |
| `ory_Orya` | Odia (individual language) | 511 | 0.988 | 0.994 | 0.976 |
| `sot_Latn` | Southern Sotho | 491 | 0.966 | 0.978 | 0.660 |
| `nso_Latn` | Northern Sotho | 479 | 0.956 | 0.969 | 0.914 |
| `kua_Latn` | Kuanyama | 476 | 0.962 | 0.986 | 0.000 |

Where MillID gains most over GlotLID:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `lbj_Tibt` | Ladakhi | 66 | 0.943 | 0.000 | 0.000 |
| `ain_Latn` | Ainu | 1 | 0.667 | 0.000 | 0.000 |
| `ajp_Arab` | South Levantine Arabic | 28 | 0.946 | 0.581 | 0.000 |
| `und_Laoo` | und | 20 | 0.923 | 0.621 | 0.000 |
| `und_Thaa` | und | 29 | 0.881 | 0.651 | 0.000 |
| `sgh_Cyrl` | Shughni | 9 | 0.857 | 0.667 | 0.000 |
| `und_Geor` | und | 32 | 0.877 | 0.694 | 0.000 |
| `tuk_Arab` | Turkmen | 135 | 0.905 | 0.761 | 0.000 |
| `und_Syrc` | und | 43 | 0.966 | 0.822 | 0.000 |
| `gmh_Latn` | Middle High German | 177 | 0.922 | 0.786 | 0.000 |
| `orv_Cyrl` | Old Russian | 239 | 0.931 | 0.800 | 0.000 |
| `und_Kana` | und | 136 | 0.982 | 0.862 | 0.000 |
| `und_Yiii` | und | 38 | 1.000 | 0.884 | 0.000 |
| `neb_Latn` | Toura (Côte d'Ivoire) | 11 | 0.957 | 0.842 | 0.000 |
| `dag_Latn` | Dagbani | 149 | 0.767 | 0.660 | 0.000 |

Where MillID loses most to GlotLID:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `mus_Latn` | Muscogee | 2 | 0.333 | 1.000 | 0.000 |
| `ttj_Latn` | Tooro | 28 | 0.485 | 0.946 | 0.000 |
| `mbd_Latn` | Dibabawon Manobo | 14 | 0.596 | 1.000 | 0.000 |
| `knc_Arab` | Central Kanuri | 68 | 0.462 | 0.825 | 0.007 |
| `biu_Latn` | Biete | 7 | 0.667 | 1.000 | 0.000 |
| `dks_Latn` | Southeastern Dinka | 3 | 0.667 | 1.000 | 0.000 |
| `esi_Latn` | North Alaskan Inupiatun | 9 | 0.455 | 0.783 | 0.000 |
| `hrv_Latn` | Croatian | 168 | 0.622 | 0.944 | 0.178 |
| `plw_Latn` | Brooke's Point Palawano | 8 | 0.727 | 1.000 | 0.000 |
| `bos_Latn` | Bosnian | 198 | 0.649 | 0.902 | 0.181 |
| `xbr_Latn` | Kambera | 8 | 0.750 | 1.000 | 0.000 |
| `nyf_Latn` | Giryama | 10 | 0.645 | 0.870 | 0.000 |
| `ldi_Latn` | Laari | 62 | 0.777 | 0.992 | 0.000 |
| `hak_Hani` | Hakka Chinese | 29 | 0.621 | 0.824 | 0.000 |
| `ccp_Latn` | Chakma | 3 | 0.800 | 1.000 | 0.000 |

### Per language: udhr

F1 language by language, GlotLID and fastText restricted to MillID's labels: against GlotLID, MillID is better on **52** languages, worse on **105** and within 0.01 on 201; against fastText, MillID is better on **303** languages, worse on **15** and within 0.01 on 40. Every language, with precision, recall and the unrestricted scores, is in `reports/udhr/leaderboard.csv`.

The 20 languages with the most passages:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `roh_Latn` | Romansh | 399 | 0.994 | 0.999 | 0.946 |
| `oci_Latn` | Occitan | 398 | 0.277 | 0.376 | 0.435 |
| `cmn_Hani` | Chinese (Mandarin) | 319 | 0.892 | 0.868 | 0.468 |
| `krl_Latn` | Karelian | 183 | 0.000 | 0.011 | 0.000 |
| `qug_Latn` | Chimborazo Highland Quichua | 182 | 0.011 | 0.000 | 0.000 |
| `acu_Latn` | Achuar-Shiwiar | 180 | 0.828 | 0.854 | 0.000 |
| `hus_Latn` | Huastec | 175 | 0.977 | 0.986 | 0.000 |
| `hau_Latn` | Hausa | 155 | 0.990 | 1.000 | 0.876 |
| `auc_Latn` | Waorani | 154 | 0.000 | 0.000 | 0.000 |
| `kwi_Latn` | Awa-Cuaiquer | 145 | 0.880 | 0.877 | 0.000 |
| `jpn_Jpan` | Japanese | 140 | 1.000 | 1.000 | 0.944 |
| `snn_Latn` | Siona | 136 | 0.630 | 0.514 | 0.000 |
| `ktu_Latn` | Kituba (Democratic Republic of Congo) | 118 | 0.765 | 0.680 | 0.000 |
| `lat_Latn` | Latin | 118 | 0.996 | 0.992 | 0.000 |
| `hat_Latn` | Haitian Creole | 117 | 0.992 | 0.979 | 0.699 |
| `nya_Latn` | Nyanja | 116 | 0.991 | 1.000 | 0.806 |
| `kmb_Latn` | Kimbundu | 116 | 0.970 | 1.000 | 0.877 |
| `rmn_Latn` | Balkan Romani | 116 | 0.678 | 0.857 | 0.000 |
| `twi_Latn` | Twi | 116 | 0.983 | 0.996 | 0.396 |
| `chr_Cher` | Cherokee | 115 | 0.670 | 0.663 | 0.000 |

Where MillID gains most over GlotLID:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `mic_Latn` | Mi'kmaw | 59 | 0.992 | 0.427 | 0.000 |
| `tet_Latn` | Tetum | 58 | 0.638 | 0.167 | 0.000 |
| `fuf_Latn` | Pular | 58 | 0.906 | 0.483 | 0.000 |
| `sag_Latn` | Sango | 59 | 0.992 | 0.578 | 0.189 |
| `skr_Arab` | Saraiki | 63 | 0.867 | 0.500 | 0.000 |
| `qvh_Latn` | Huamalíes-Dos de Mayo Huánuco Quechua | 58 | 0.468 | 0.154 | 0.000 |
| `zgh_Tfng` | Standard Moroccan Tamazight | 60 | 0.750 | 0.462 | 0.000 |
| `lld_Latn` | Ladin | 59 | 0.909 | 0.731 | 0.000 |
| `tah_Latn` | Tahitian | 57 | 0.973 | 0.812 | 0.966 |
| `yue_Hani` | Cantonese | 49 | 0.734 | 0.580 | 0.111 |
| `ote_Latn` | Mezquital Otomi | 59 | 0.154 | 0.000 | 0.000 |
| `pnb_Arab` | Western Panjabi | 61 | 0.882 | 0.739 | 0.000 |
| `mxv_Latn` | Metlatónoc Mixtec | 58 | 0.129 | 0.000 | 0.000 |
| `lij_Latn` | Ligurian | 58 | 0.983 | 0.859 | 0.487 |
| `bum_Latn` | Bulu | 58 | 0.215 | 0.098 | 0.000 |

Where MillID loses most to GlotLID:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `gyr_Latn` | Guarayu | 49 | 0.537 | 0.990 | 0.000 |
| `shp_Latn` | Shipibo-Conibo | 62 | 0.469 | 0.912 | 0.000 |
| `dyu_Latn` | Dyula | 58 | 0.000 | 0.434 | 0.015 |
| `tbz_Latn` | Ditammari | 60 | 0.000 | 0.378 | 0.000 |
| `cbu_Latn` | Candoshi-Shapra | 57 | 0.500 | 0.874 | 0.000 |
| `idu_Latn` | Idoma | 51 | 0.603 | 0.938 | 0.000 |
| `ell_Grek` | Greek | 115 | 0.670 | 1.000 | 1.000 |
| `qvc_Latn` | Cajamarca Quechua | 58 | 0.648 | 0.906 | 0.000 |
| `cpu_Latn` | Pichis Ashéninka | 60 | 0.729 | 0.974 | 0.000 |
| `qvn_Latn` | North Junín Quechua | 57 | 0.219 | 0.460 | 0.000 |
| `hin_Deva` | Hindi | 59 | 0.723 | 0.957 | 0.975 |
| `hrv_Latn` | Croatian | 58 | 0.500 | 0.727 | 0.474 |
| `kbp_Latn` | Kabiyè | 59 | 0.781 | 1.000 | 0.389 |
| `cni_Latn` | Asháninka | 52 | 0.773 | 0.963 | 0.000 |
| `rmn_Latn` | Balkan Romani | 116 | 0.678 | 0.857 | 0.000 |

### Per language: test-common

F1 language by language, GlotLID and fastText restricted to MillID's labels: against GlotLID, MillID is better on **6** languages, worse on **154** and within 0.01 on 53; against fastText, MillID is better on **85** languages, worse on **51** and within 0.01 on 77. Every language, with precision, recall and the unrestricted scores, is in `reports/test-common/leaderboard.csv`.

The 20 languages with the most passages:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `fra_Latn` | French | 1652 | 0.943 | 0.955 | 0.963 |
| `eng_Latn` | English | 1444 | 0.932 | 0.877 | 0.857 |
| `spa_Latn` | Spanish | 1136 | 0.937 | 0.958 | 0.941 |
| `tam_Taml` | Tamil | 754 | 0.997 | 1.000 | 0.997 |
| `arb_Arab` | Arabic (Modern Standard) | 745 | 0.871 | 0.886 | 0.954 |
| `por_Latn` | Portuguese | 657 | 0.959 | 0.956 | 0.949 |
| `kan_Knda` | Kannada | 601 | 0.973 | 0.994 | 0.999 |
| `zul_Latn` | Zulu | 594 | 0.883 | 0.904 | 0.970 |
| `swh_Latn` | Swahili | 592 | 0.893 | 0.953 | 0.945 |
| `mar_Deva` | Marathi | 589 | 0.968 | 0.990 | 0.987 |
| `tso_Latn` | Tsonga | 581 | 0.979 | 0.956 | 0.986 |
| `hun_Latn` | Hungarian | 576 | 0.989 | 1.000 | 0.980 |
| `afr_Latn` | Afrikaans | 569 | 0.972 | 0.985 | 0.986 |
| `tel_Telu` | Telugu | 555 | 0.997 | 0.998 | 0.999 |
| `rus_Cyrl` | Russian | 528 | 0.945 | 0.985 | 0.927 |
| `ory_Orya` | Odia (individual language) | 511 | 0.994 | 1.000 | 0.996 |
| `sot_Latn` | Southern Sotho | 491 | 0.969 | 0.978 | 0.856 |
| `nso_Latn` | Northern Sotho | 479 | 0.958 | 0.970 | 0.967 |
| `asm_Beng` | Assamese | 467 | 0.997 | 0.999 | 0.997 |
| `ben_Beng` | Bangla | 466 | 0.988 | 0.997 | 0.989 |

Where MillID gains most over GlotLID:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `ewo_Latn` | Ewondo | 134 | 0.959 | 0.883 | 0.860 |
| `eng_Latn` | English | 1444 | 0.932 | 0.877 | 0.857 |
| `arb_Latn` | Standard Arabic | 163 | 0.942 | 0.913 | 0.852 |
| `tso_Latn` | Tsonga | 581 | 0.979 | 0.956 | 0.986 |
| `fuv_Latn` | Nigerian Fulfulde | 164 | 0.907 | 0.892 | 0.859 |
| `knc_Latn` | Central Kanuri | 293 | 0.967 | 0.952 | 0.981 |
| `quy_Latn` | Ayacucho Quechua | 271 | 0.940 | 0.933 | 0.979 |
| `kam_Latn` | Kamba | 272 | 0.980 | 0.974 | 0.863 |
| `bjn_Arab` | Banjar | 116 | 0.992 | 0.987 | 0.960 |
| `por_Latn` | Portuguese | 657 | 0.959 | 0.956 | 0.949 |
| `mlt_Latn` | Maltese | 291 | 0.995 | 0.993 | 0.961 |
| `hye_Armn` | Armenian | 178 | 1.000 | 1.000 | 0.977 |
| `khm_Khmr` | Khmer | 187 | 1.000 | 1.000 | 1.000 |
| `guj_Gujr` | Gujarati | 354 | 1.000 | 1.000 | 0.984 |
| `mal_Mlym` | Malayalam | 338 | 1.000 | 1.000 | 0.994 |

Where MillID loses most to GlotLID:

| Label | Language | Passages | MillID | GlotLID | fastText |
|---|---|---|---|---|---|
| `knc_Arab` | Central Kanuri | 68 | 0.522 | 0.860 | 0.029 |
| `hrv_Latn` | Croatian | 168 | 0.659 | 0.994 | 0.856 |
| `bos_Latn` | Bosnian | 198 | 0.685 | 1.000 | 0.897 |
| `tzm_Tfng` | Central Atlas Tamazight | 157 | 0.697 | 0.934 | 0.751 |
| `zsm_Latn` | Standard Malay | 244 | 0.789 | 0.998 | 0.908 |
| `taq_Tfng` | Tamasheq | 104 | 0.654 | 0.805 | 0.000 |
| `ind_Latn` | Indonesian | 335 | 0.842 | 0.991 | 0.887 |
| `kas_Deva` | Kashmiri | 47 | 0.557 | 0.703 | 0.587 |
| `cjk_Latn` | Chokwe | 24 | 0.857 | 1.000 | 0.807 |
| `nob_Latn` | Norwegian Bokmål | 265 | 0.821 | 0.959 | 0.894 |
| `kin_Latn` | Kinyarwanda | 210 | 0.849 | 0.981 | 0.854 |
| `mag_Deva` | Magahi | 97 | 0.890 | 1.000 | 0.772 |
| `lim_Latn` | Limburgish | 196 | 0.896 | 1.000 | 0.956 |
| `run_Latn` | Rundi | 291 | 0.883 | 0.985 | 0.855 |
| `taq_Latn` | Tamasheq | 106 | 0.902 | 1.000 | 0.838 |

Per-language precision, recall (sensitivity), F1 and specificity for every system, restricted and as it is, are in `reports/<set>/`, and the totals in `reports/summary.json`.

## Training

1,943,957 passages; the best validation macro F1 at step 106,000, after 0.6 hours on 1 GPU. Languages sampled in proportion to n^0.3; 25% of passages cut to a 20 to 80 character snippet; batch 512, AdamW at 0.001, cosine schedule, label smoothing 0.1.

The training data is the full MillID build. The public [baobabtech/millid-dataset](https://huggingface.co/datasets/baobabtech/millid-dataset) is that build without Save the Children's and Nal'ibali's text (1.6% of its passages), which is not ours to redistribute. Its splits are the same, so its `test` split is the one scored above, without those passages.

## Licence

The weights and the code are Apache-2.0. The training data carries its own licence row by row (see the [dataset card](https://huggingface.co/datasets/baobabtech/millid-dataset#licensing)), including text licensed for non-commercial use only (CC BY-NC, CC BY-NC-SA).