Upendar/videolens-aligner-te

Telugu forced aligner (ONNX) — wav2vec2 XLSR-53 Telugu

An ONNX conversion of the wav2vec2 XLSR-53 Telugu model by Anurag Singh (anuragshas/wav2vec2-large-xlsr-53-telugu), released under the Apache License 2.0, made for the forced word alignment in VideoLens. All credit for the model goes to its authors; this repository only repackages it.

Files

• model_fp16.onnx — the CTC model exported to ONNX (opset 17) and converted to half precision, with float32 input (input_values, 16 kHz mono, normalised to zero mean and unit variance) and float32 output (logits, one frame per 20 ms).
• vocab.json — the model's letters; the CTC blank is named [PAD].

Changes from the original

Converted from the original PyTorch weights with convert_w2v.py (included here; torch.onnx export, then onnxconverter_common.float16 with float32 inputs and outputs); the greedy reading of the ONNX file agrees with PyTorch on 99–100% of frames. The weights are otherwise unchanged.

Use in VideoLens

VideoLens uses this model as a forced aligner: it places each word of a transcript on the audio (Viterbi over the CTC output). Measured against Meta's MMS forced aligner on the same words: word starts a median 0.02 s apart, 90% within 0.18 s on clearly heard lines (FLEURS Telugu, read speech).

Licence

Apache License 2.0, as the original model — see LICENSE.