shun3232/espnet3_lid_voxlingua107_mms_ecapa

ESPnet3 VoxLingua107 MMS–ECAPA LID

107-language speech classifier trained with ESPnet3 on the official VoxLingua107 training split (2,536,954 utterances; 6,628.46 hours). MMS and the classification components were fully fine-tuned: about 971 million trainable parameters, none frozen.

Training: 30 epochs, 1,000 category-sampled batches per epoch, accumulation over 2 batches, 15,000 optimizer updates. Each epoch samples from the corpus; it is not a complete traversal. The supplied checkpoint is the best individual validation-accuracy checkpoint, epoch 18 (step 9000). Training used BF16 mixed precision and FP32 matmul precision highest, on one H200 for about 8 hours 54 minutes.

Evaluation

ESPnet3 inference, FP32, variable-length batches. Reference languages absent from the 107-class inventory are excluded. Prediction candidates remain all 107 classes. Macro metrics average over reference languages actually present in each split. The selected model was chosen using VoxLingua107 dev; its dev score is not a held-out test result.

Split	Utterances	Accuracy (%)
voxlingua107_dev	1,609	93.78
fleurs_test	63,885	94.49
ml_superb2_dev	17,631	88.31
ml_superb2_dev_dialect	7,095	75.79
voxpopuli_test	16,991	88.42

VoxPopuli uses session-qualified IDs after correcting annotation-ID collisions.

Use

Requires the ESPnet3 esp2_lid implementation for ESPnet PR #6742. The tested implementation is commit 9b2f866c6. For older lid-based checkouts, use model revision d14bdccdea1f0126270c031069c84ba1988b9c96 instead. Tested with Python 3.12, PyTorch 2.9.1, transformers 5.8.0 and s3prl 0.4.18.

from huggingface_hub import snapshot_download
from espnet3.publication import InferenceModel
import soundfile as sf

folder = snapshot_download("shun3232/espnet3_lid_voxlingua107_mms_ecapa", max_workers=4)
model = InferenceModel.from_packed(folder, trust_user_code=True)
speech, rate = sf.read("speech_16k_mono.wav", dtype="float32")
assert rate == 16000 and speech.ndim == 1
result = model(speech)
print(result["hyp"])  # ISO 639-3 language code
embedding = result["embedding"]

The portable bundle defaults to CPU inference, contains the ordered class inventory, and includes the pinned MMS initialization files. Input is mono 16 kHz speech. Bundled recipe source requires trust_user_code=True. Raw training/evaluation audio is not included.

Sources and license

Based on Meta's MMS-1B, revision 0d2f7adb9903d98894d70ae11f7fbdfc8cb71a69, and the ESPnet VoxLingua107 LID recipe. MMS authors: Vineel Pratap and colleagues, Scaling Speech Technology to 1,000+ Languages (2023). The model weights are provided under CC BY-NC 4.0, following the MMS base model. ESPnet recipe source retains its Apache-2.0 license. See assets/mms-1b/README.md for upstream attribution.