CIIL/Anuvani-LID

---

Spoken Language Identification (23 languages)

Anuvani-LID tells you which language is spoken in a recording. It covers the 22 scheduled languages of India and Indian English (23 labels) and is built for the first decision of every multilingual speech pipeline: which recogniser to run, which way to translate, which language to answer in. It is a 115.6M-parameter Conformer trained end to end on 54,643 hours of Indian speech, and its scores are calibrated.

This repository comes with:

• The model in pure PyTorch: pipeline or AutoModel from transformers, whose only runtime dependencies are torch and transformers, plus the same network as one ONNX graph.
• triton_inference/, a serving kit for NVIDIA Triton Inference Server: TensorRT engines built on your GPU, a FastAPI tier with Silero VAD, an accuracy gate, from one GPU to many nodes, and a CPU-only path.
• mobile_inference/, an on-device kit: sherpa-onnx with model files, libraries and app code for Android, iOS and ARM Linux, so the model runs inside an app with no server.
• notebook.ipynb, a quick start that runs every example below in Google Colab.

Key capabilities:

• Accurate across conditions: 0.914 mean macro-F1 over 37 benchmark conditions, 5.2 points above Gemini 3.5 Flash, the strongest of the other systems scored on the same clips; 0.897 over all 23 languages, where no other system exceeds 0.713.
• Short clips: 24 points ahead of Gemini 3.5 Flash at one second.
• Calibrated: expected calibration error 0.008 after a single temperature, so the scores can be used as confidences; music and non-Indian languages are rejected with AUROC 0.89–0.91.
• Any length, mixed languages: classify_long_audio finds the speech with Silero VAD and returns a timeline and each language's share of the speech.
• Runs on what you have: NVIDIA GPUs behind Triton, CPUs through ONNX Runtime, phones through sherpa-onnx.
• Small: 115.6M parameters, 13 times fewer than Whisper-large-v3.
• Open source under Apache-2.0.

Model overview

Property	Value
Task	Spoken language identification: one language per recording, or per speech phrase with classify_long_audio
Languages	the 22 scheduled languages of India and Indian English (23 labels, listed below)
Architecture	80-dimensional log-Mel features → 4× convolutional subsampling → 17 Conformer blocks (d = 512, 8 heads, relative positions) → learned weighted sum of all 17 block outputs → attentive statistics pooling → 256-dimensional embedding → 23-way classifier
Parameters	115.6M
Training data	54,643 hours, 31.24M utterances from 18 published corpora; Indian English 13,039 h to Kashmiri 369 h
Training	the whole network fine-tuned from a self-supervised pretrained Conformer encoder; focal loss (γ = 2) with label smoothing 0.1; temperature-based language sampling; a confusability curriculum; 425k steps; the released weights average seven checkpoints
Input	16 kHz mono audio (the pipeline resamples files for you)
Output	23 logits; pipeline: labels with scores; classify_long_audio: overall language, time share per language and a timeline
Implementation	self-contained pure PyTorch (no NeMo, no hydra, no custom runtime), verified logit-faithful to the training checkpoint over the 102-condition benchmark
Runtimes	PyTorch with transformers; ONNX Runtime; TensorRT on Triton; sherpa-onnx on phones
Licence	Apache-2.0
Paper	arXiv 2610.20263 · Hugging Face paper page

Inference

opens a quick-start notebook (notebook.ipynb) that runs every example below in Google Colab.

pipeline (give it a file path; it resamples for you)

from transformers import pipeline
repo = "CIIL/Anuvani-LID"
clf = pipeline("audio-classification", model=repo, feature_extractor=repo, trust_remote_code=True)
# [{'label': 'hindi', 'score': 0.98}, ...]
print(clf("audio.wav")[:3])

This is a custom architecture, so trust_remote_code=True and feature_extractor=repo are required (a built-in transformers version, for the bare pipeline(model=repo), is in progress upstream). The pipeline decodes file paths with ffmpeg, which must be installed on the system.

No audio at hand? The repository ships five short FLEURS clips (samples/, CC-BY-4.0):

from huggingface_hub import hf_hub_download
print(clf(hf_hub_download(repo, "samples/hi_in.wav"))[:3])

AutoModel

import soundfile as sf, torch
from transformers import AutoModelForAudioClassification, AutoFeatureExtractor
repo = "CIIL/Anuvani-LID"
# native Wav2Vec2FeatureExtractor
fe = AutoFeatureExtractor.from_pretrained(repo)
model = AutoModelForAudioClassification.from_pretrained(repo, trust_remote_code=True).eval()
# 16 kHz mono
wav, sr = sf.read("audio.wav")
logits = model(**fe(wav, sampling_rate=sr, return_tensors="pt")).logits
print(model.config.id2label[logits.argmax(-1).item()])

Long audio (minutes to hours) — Silero VAD phrases, then a language timeline

Self-attention is O(T²): a single forward over a 10-minute file needs 20 GB of GPU transients and a 1-hour file is impossible (260 GB). For anything beyond ~2 minutes use the built-in helper. It finds the speech phrases with Silero VAD, merges phrases separated by short pauses into pieces of up to 30 s, classifies every piece (zero-padded to the smallest of the 2 / 4 / 8 / 15 / 30 s buckets, with its true length) in batches through the normal forward, and aggregates the pieces weighted by their duration, so silence, music and noise never vote:

import librosa
# any format/length
wav, _ = librosa.load("lecture_1hour.mp3", sr=16000, mono=True)
# needs: pip install silero-vad
out = model.classify_long_audio(wav)
# overall (duration-weighted over the speech pieces)
print(out["label"], out["score"])
# share of speech time per language
print(out["dominant_by_time"], out["language_time_share"])
# per phrase piece — surfaces code-switching
for p in out["timeline"]:
    print(f"{p['start']:7.1f}-{p['end']:7.1f}s  {p['label']}  {p['score']:.3f}")

For audio that mixes languages, read dominant_by_time and language_time_share: label is the single language that best explains all the speech together, and its score falls towards 0 when several languages are present (the production API reports the same way). The VAD settings are the ones the production API in triton_inference/ uses: threshold 0.5, minimum speech 250 ms, minimum silence 300 ms, speech padding 100 ms, maximum phrase 30 s; pieces shorter than 0.3 s are dropped and pauses up to 0.6 s are bridged. Knobs: vad_params={...} (any silero_vad.get_speech_timestamps keyword), vad_model= (a loaded Silero model), batch_size=16 (memory), and chunking="uniform" for fixed 30 s windows without a VAD (window_s, hop_s). This is an opt-in wrapper — the production forward(input_values, input_lengths) contract is untouched, so batched serving, ONNX export, torch.compile/TensorRT and every other optimization path are unaffected.

ONNX (no transformers / no torch needed — the most portable, longest-lived path)

The serving graph ships in this repository as triton_inference/assets/model/anuvani_lid.onnx (fp32, raw 16 kHz audio in, logits out). Checked against the PyTorch model on 26 golden clips: max |Δlogit| 1.5e-3 at native length, and at most 1.7e-3 with each clip zero-padded to every 2 / 4 / 8 / 15 / 30 s bucket that holds it, with no change of decision. Rows of a batch may be zero-padded to a common length: pass each row's true length in audio_length.

import onnxruntime as ort, soundfile as sf, numpy as np
from huggingface_hub import hf_hub_download
repo = "CIIL/Anuvani-LID"
sess = ort.InferenceSession(hf_hub_download(repo, "triton_inference/assets/model/anuvani_lid.onnx"))
labels = open(hf_hub_download(repo, "labels.txt")).read().split()
# 16 kHz mono
wav, _ = sf.read("audio.wav", dtype="float32")
logits = sess.run(["logits"], {"audio": wav[None], "audio_length": np.array([len(wav)], np.int64)})[0]
print(labels[int(logits.argmax())])

Requirements: pip install transformers torch soundfile (silero-vad for classify_long_audio, onnxruntime for the ONNX path, librosa for the long-audio example). No NeMo.

Production serving at scale: triton_inference/

triton_inference/ is the complete serving recipe for this model on NVIDIA Triton Inference Server, from a single GPU to many GPUs and nodes, plus a CPU-only path: TensorRT engines built on your GPU, the Triton model configurations, Dockerfiles, docker-compose files, the FastAPI client tier, the accuracy gate and the load tests. Start at triton_inference/README.md. It serves one GPU by default; two keys in deploy/compose/.env (written by ./lid build) scale a node to N GPUs (COMPOSE_PROFILES=gpus-<N>, one Triton container per GPU) with K model instances per GPU (LID_INSTANCES_PER_GPU=<K>, for GPUs with more SMs than one instance fills), and the client tier discovers the Tritons and their instance counts by itself (triton_inference/docs/05_scaling.md).

How audio reaches the model there: the client tier decodes each upload to 16 kHz mono, finds the speech phrases with Silero VAD (the settings above), merges and cuts them into pieces of up to 30 s, and sends every piece, zero-padded to its bucket with its true length, to Triton; the per-piece logits are aggregated by duration, as in classify_long_audio.

What is built in

• Engines built on your GPU: ./lid build turns one ONNX graph into TensorRT engines for five exact-length buckets (2, 4, 8, 15 and 30 s), with batch limits planned from your GPU's free memory (cards from 4 GB up), inside the Triton image that serves them.
• fp16 where it is safe: the Conformer runs in fp16 and the mel front end stays in fp32, as two engines in one Triton ensemble: 0.73 ms of engine time per 4-second item at batch 32, against 2.07 ms in fp32.
• Graph rewrites that let TensorRT use its fused attention kernels and sum the 17 block outputs incrementally: 9 % to 12 % less encoder time.
• Zero padding with the true length: every segment is padded to its bucket and sent with its real length. This changed 1.8 % of decisions against 2.7 % for repeating the audio to fill the bucket, and leaves macro-F1 unchanged.
• Batching tuned per bucket: queue delays of 2, 4, 8, 15 and 30 ms and batch caps of 64, 64, 64, 32 and 16. At 400 requests/s on one GPU this gave a p99 of 201.4 ms, against 669.8 ms with one flat 2 ms delay, and 165.0 ms with the graph rewrites.
• Memory sized to the batches served: building each bucket at its own batch cap cut the engines' activation memory from 15.6 GB to 6.4 GB.
• One Triton per GPU, NUMA-pinned, each with its own batchers: throughput grows linearly with the number of GPUs.
• A fast Silero VAD: 1.0 s of one CPU core per 10-minute file, against 1.9 s for the reference implementation.
• Admission control in the API tier (429 and 503 answers), so Triton never sheds load.
• An accuracy gate before any traffic: the 26 golden clips must give the certified decisions and at least 99.5 % of 918 benchmark clips must agree with the certified fp32 model.
• Tried and left out, with the evidence in triton_inference/docs/: INT8, CUDA graphs, a single mixed-precision engine, two model instances per GPU by default, and custom kernels (the GPU was already at its power limit).

Performance on H200 and B200

GPU	One GPU, mixed traffic, p99 ≤ 250 ms	8-GPU node, direct to Triton
NVIDIA H200	≈ 2,550 requests/s (≈ 22,000 audio-seconds per second)	≈ 20,400 requests/s (≈ 176,000 audio-seconds per second)
NVIDIA B200	≈ 5,800 requests/s (≈ 50,000 audio-seconds per second)	≈ 46,500 requests/s (≈ 401,000 audio-seconds per second)

Projected from the kit's measured results (triton_inference/docs/10_results.md) by each GPU's dense FP16 Tensor Core throughput, at the efficiency measured there; ./lid tune measures your GPU in minutes. Any NVIDIA GPU from compute capability 7.5 (Turing to Blackwell) is supported, because the engines are built on it. Through the API tier, file-level throughput is bounded by the CPU VAD (about 600 times real time per core in its fast mode), so size the API tier's cores with the GPUs (triton_inference/docs/05_scaling.md). On CPUs alone (ONNX Runtime, 24 cores) the kit serves about 52 audio-seconds per second: for development, CI and low-volume serving.

Quick start

hf download CIIL/Anuvani-LID --include "triton_inference/*" --local-dir Anuvani-LID
# downloads do not keep executable bits
cd Anuvani-LID/triton_inference && chmod +x lid
./lid doctor && ./lid prepare && ./lid build && ./lid up && ./lid gate
curl -F file=@speech.wav localhost:8080/v1/lid | python3 -m json.tool

On-device inference: mobile_inference/

mobile_inference/ runs Anuvani-LID inside an app, with no server. It is sherpa-onnx v1.13.8 with two patches that add the model and the API tier's long-audio procedure (Silero VAD phrases, pieces of up to 30 s, the duration-weighted decision and the same JSON), plus model files, prebuilt libraries, app code and accuracy gates. Start with int8 on the CPU everywhere.

Model file	Size	For
anuvani_lid.int8.onnx	169.0 MB	the default on every phone's CPU; any length
anuvani_lid.fp32.onnx	464.3 MB	the certified graph: reference results on any CPU
anuvani_lid.npu-15s.encoder.onnx with anuvani_lid.npu-15s.frontend.onnx	471.2 MB + 0.1 MB	NPUs, in fp16 on a fixed 15 s window: Core ML (iPhone and iPad, A12 and newer) and Qualcomm QNN (Snapdragon 8 Gen 1 and newer, Hexagon v69+)
anuvani_lid.npu-15s-nnapi.encoder.onnx	471.3 MB	Android NNAPI (Android 8.1 to 14), after measuring on the phone
silero_vad_6.2.1.onnx	2.3 MB	Silero VAD for long audio

• Platforms: Android 8.1 and newer (arm64-v8a, armeabi-v7a, x86_64, x86), iOS 15 and newer (a Swift package built on a Mac by ios/build-ios.sh), and Linux on 64-bit and 32-bit ARM; bindings for Kotlin, Java, Swift, Dart and Flutter, C# and .NET, C and C++.
• Checked against the certified model's decisions on 918 benchmark clips: int8 agrees on 99.24 % (macro-F1 0.8536 against 0.8552), the fp32 and NPU graphs on 100 %, and the 26 golden clips give the same language everywhere. On emulated ARM cores int8 agrees on 99.02 % to 99.24 %. The Android build ran on an Android 15 emulator, NNAPI included, and Qualcomm's emulated NPU ran the whole encoder on Hexagon v69 to v81 (v68 falls back to the CPU).
• Not yet measured: speed and battery on a physical phone, real NPU and NNAPI drivers, and the iOS build, which needs a Mac. ./mobile bench android measures a phone over USB.

hf download CIIL/Anuvani-LID --include "mobile_inference/*" --local-dir Anuvani-LID
# downloads drop executable bits
cd Anuvani-LID/mobile_inference && chmod +x mobile prebuilt/*/bin/*
# check the model files
(cd models && sha256sum -c SHA256SUMS)

Start at mobile_inference/README.md: which file for which phone, the Android, iOS and Flutter guides, and the gates.

Languages

assamese, bengali, bodo, dogri, english, gujarati, hindi, kannada, kashmiri, konkani, maithili, malayalam, manipuri, marathi, nepali, odia, punjabi, sanskrit, santali, sindhi, tamil, telugu, urdu (label id = alphabetical order; see config.json id2label).

Performance (held-out 102-condition benchmark suite, macro-F1)

Equal-weighted mean over all conditions: 0.8453.

slice	macro-F1
clean (6k balanced panel)	0.976
confusable language clusters	0.948
noise (MUSAN, SNR 0/5)	0.936
reverb	0.935
open-set panels	0.936
telephony (G.711)	0.923
plain cross-corpus panels	0.921
babble (competing speaker)	0.903
short-duration crops (0.25–3 s)	0.713 (the weak axis)
VAD-anchored short crops	0.603 (the hardest axis)

Reference points on individual panels: 10 s clean 0.985, 3 s 0.973, 1 s 0.827.

Notes & limitations

• Short utterances (<3 s) are the weak slice (0.25 s random crops are near-chance for any system); strong regional accents (e.g. Svarah/HiACC panels) are the other hard axis.
• Audio longer than ~2 minutes: use model.classify_long_audio(...) (see Long audio above) — a single forward is O(T²) in memory and a one-label answer over an hour is usually wrong anyway.
• Phrase chunking uses Silero VAD (MIT licence, © Silero Team): the silero-vad package here, and two Silero VAD v6 ONNX files in the Triton client tier (triton_inference/THIRD_PARTY_NOTICES.md).
• Responsible use: do not use LID output as a proxy for ethnicity, religion or nationality. Hindi and Urdu, in particular, share a spoken register, so a recording's label says nothing about its speaker's identity.
• License Apache-2.0. Please cite the paper (below).

Acknowledgement

Anuvani-LID was conceived and developed by Stephen Fernandes under the leadership and guidance of Dr. Narayan Kumar Choudhary. We thank Dr. Choudhary and the Central Institute of Indian Languages (CIIL) for their assistance, for the compute they provided and for their continued support.

Citation

@misc{fernandes2026anuvanilid,
  title={Anuvani-LID: Robust and Calibrated Spoken Language Identification for Indian Languages},
  author={Stephen Fernandes and Narayan Choudhary},
  year={2026},
  eprint={2610.20263},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2610.20263}
}