File size: 6,413 Bytes

---
license: mit
pipeline_tag: audio-classification
tags:
  - language-identification
  - spoken-language-identification
  - audio-classification
  - speech
  - multilingual
  - wav2vec2
  - mms
  - xls-r
  - pytorch-lightning
datasets:
  - google/fleurs
---
# BA-Lang - Spoken Language Identification checkpoints

Checkpoints for **spoken language identification (LID)**: a self-supervised
speech backbone, pooled with **MHFA**
(Multi-Head Factorized Attention) and scored with either a standard
cosine-softmax classifier or **BA-LR** (Binary Attribute Likelihood Ratio),
a Bernoulli classifier over a learned binary attribute embedding.

- Training code / recipe: [https://github.com/AMIAD-Research/BA-Lang]
- BA-LR / BA-Lang model: Y. Jelassi et al., *"Reconnaissance de la langue parlée explicable avec BA-LR"*, JEP 2026 —
  [isca-archive.org/jep_2026/jelassi26_jep.html](https://www.isca-archive.org/jep_2026/jelassi26_jep.html)


## Architecture

```
raw audio (16 kHz)
      │
      ▼
 SSL backbone (MMS-1B or wav2vec2-XLS-R)
      │
      ▼
 Multi-Head Factorized Attentive pooling (MHFA)
      │
      ▼
 classification head
   ├─ Linear, or
   └─ BA-Lang: binary encoder → per-class Bernoulli log-likelihood
      │
      ▼
 language logits
```



## Checkpoints

| File | Backbone | Corpus | Languages | Head | Step | Val. accuracy | Size |
|---|---|---|---|---|---|---|---|
| `MMS-Fleurs-step-010360-val_classif_acc-0.9168118238449097.ckpt` | MMS-1B | FLEURS | 102 | Linear | 10360 | 91.68% | 3.86 GB |
| `MMS-Fleurs-e2e-balang-stepstep-008856-val_classif_acc-0.9167537689208984.ckpt` | MMS-1B | FLEURS | 102 | BA-Lang (e2e) | 8856 | 91.68% | 3.87 GB |
| `MMS-Fleurs-e2e-balang-MIR-rho-0.95-step-032472-val_classif_acc-0.90935212373733...ckpt` | MMS-1B | FLEURS | 102 | BA-Lang (e2e) + Matryoshka-inspired regularization, ρ=0.95 | 32472 | 90.94% | 3.87 GB |
| `MMS-Fleurs-e2e-balang-ND-rho-0.857-step-047232-val_classif_acc-0.89306861162185...ckpt` | MMS-1B | FLEURS | 102 | BA-Lang (e2e) + Nested Dropout, ρ=0.857 | 47232 | 89.31% | 3.87 GB |
| `MMS-Tidylang-step-008533-val_classif_acc-0.9778640866279602.ckpt` | MMS-1B | Tidylang | 35 | Linear | 8533 | 97.79% | 3.86 GB |
| `MMS-Tidylang-e2e-balang-step-005565-val_classif_acc-0.9929152131080627.ckpt` | MMS-1B | Tidylang | 35 | BA-Lang (e2e) | 5565 | 99.29% | 3.87 GB |
| `XLSR-Fleurs-step-005904-val_classif_acc-0.8094159960746765.ckpt` | wav2vec2-XLS-R | FLEURS | 102 | Linear | 5904 | 80.94% | 1.27 GB |
| `XLSR-Fleurs-e2e-balang-step-047232-val_classif_acc-0.8120864033699036.ckpt` | wav2vec2-XLS-R | FLEURS | 102 | BA-Lang (e2e) | 47232 | 81.21% | 1.27 GB |
| `XLSR-Tidylang-step-008904-val_classif_acc-0.9662737846374512.ckpt` | wav2vec2-XLS-R | Tidylang | 35 | Linear | 8904 | 96.63% | 1.27 GB |
| `XLSR-Tidylang-e2e-balang-step-032648-val_classif_acc-0.9860262870788574...ckpt` | wav2vec2-XLS-R | Tidylang | 35 | BA-Lang (e2e) | 32648 | 98.60% | 1.27 GB |



## Datasets

Training is done separately on two corpora — pick the checkpoint matching
the languages you need.

- **[FLEURS](https://huggingface.co/datasets/google/fleurs)** (102
  languages, ISO 639-3): `afr, amh, ara, asm, ast, aze, bel, ben, bos, bul,
  cat, ceb, ces, ckb, cmn, cym, dan, deu, ell, eng, est, fas, fil, fin, fra,
  ful, gle, glg, guj, hau, heb, hin, hrv, hun, hye, ibo, ind, isl, ita, jav,
  jpn, kam, kan, kat, kaz, kea, khm, kir, kor, lao, lav, lin, lit, ltz, lug,
  luo, mal, mar, mkd, mlt, mon, mri, msa, mya, nep, nld, nob, nso, nya, oci,
  ori, orm, pan, pol, por, pus, ron, rus, slk, slv, sna, snd, som, spa, srp,
  swa, swe, tam, tel, tgk, tha, tur, ukr, umb, urd, uzb, vie, wol, xho, yor,
  yue, zul`.
- **Tidylang** (35 languages): `TidyVoiceX_ASV` (train/dev) and
  `TidyVoiceX2_ASV` (eval), from Mozilla Data Collective
  ([train/dev](https://mozilladatacollective.com/datasets/cmihtsewu023so207xot1iqqw),
  [eval](https://mozilladatacollective.com/datasets/cmkv32i5e02tumg07j79d3c35)):
  `ara, bak, bel, ben, bul, cat, chv, cym, deu, div, ell, eng, fas, fra, hin,
  hye, jpn, kat, lit, lug, mal, mar, nld, ori, pol, por, rus, tam, tha, tuk,
  tur, uig, uzb, yue, zho`.



```python
import yaml
lang_list = yaml.safe_load(open("configs/data/fleurs.yaml"))["language_list"]
# or configs/data/tidylang.yaml for the Tidylang checkpoints
```

## Quickstart — inference on your own audio

```bash
git clone https://github.com/AMIAD-Research/BA-Lang
cd BA-Lang
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

pip install huggingface_hub
python -c "
from huggingface_hub import hf_hub_download
hf_hub_download(
    repo_id='yjelassi/BA-Lang',
    filename='MMS-Fleurs-e2e-balang-stepstep-008856-val_classif_acc-0.9167537689208984.ckpt',
    local_dir='.',
)
"
```

Then, on a single audio file (no manifest needed — this loads the
checkpoint directly and runs one forward pass):

```python
import torch
import torchaudio
import yaml

from balr_lid.models.lid_model import LanguageIdentificationModel

CHECKPOINT = "MMS-Fleurs-e2e-balang-stepstep-008856-val_classif_acc-0.9167537689208984.ckpt"
LANGUAGES = yaml.safe_load(open("configs/data/fleurs.yaml"))["language_list"]

model = LanguageIdentificationModel.load_from_checkpoint(
    CHECKPOINT, map_location="cpu", weights_only=False
)
model.eval()

waveform, sr = torchaudio.load("my_audio.wav")
if waveform.shape[0] > 1:  
    waveform = waveform.mean(dim=0, keepdim=True)
if sr != 16000:
    waveform = torchaudio.functional.resample(waveform, sr, 16000)

with torch.no_grad():
    out = model(waveform) 
    probs = out["logits"].softmax(dim=-1)[0]

top5 = probs.topk(5)
for score, idx in zip(top5.values, top5.indices):
    print(f"{LANGUAGES[idx]}: {score.item():.3f}")
```



### Batch inference over a dataset

To evaluate on many files at once (accuracy, confusion matrix, embedding
dump), build a manifest CSV (see
[docs/data_manifest.md](https://github.com/AMIAD-Research/BA-Lang))
and use the repo's batch script instead:

```bash
python -m balr_lid.inference \
    --checkpoint MMS-Fleurs-e2e-balang-stepstep-008856-val_classif_acc-0.9167537689208984.ckpt \
    --manifest data/manifests/fleurs-test.csv \
    --audio-root /path/to/fleurs/audio \
    --language-list afr amh ara ... \
    --output predictions.csv
```

`--language-list` must match the checkpoint's training corpus, in order
(see [Datasets](#datasets) above).