Tamil forced aligner (ONNX) — Vakyansh wav2vec2 Tamil
An ONNX conversion of the Vakyansh Tamil wav2vec2 model by Harveen Chadha / EkStep Foundation (Harveenchadha/vakyansh-wav2vec2-tamil-tam-250, upstream), released under the MIT licence, made for the forced word alignment in VideoLens. All credit for the model goes to its authors; this repository only repackages it.
Files
model_fp16.onnx— the CTC model exported to ONNX (opset 17) and converted to half precision, with float32 input (input_values, 16 kHz mono, normalised to zero mean and unit variance) and float32 output (logits, one frame per 20 ms).vocab.json— the model's letters; the CTC blank is named<blank>.
Changes from the original
Converted from the original PyTorch weights with convert_w2v.py (included here; torch.onnx export, then
onnxconverter_common.float16 with float32 inputs and outputs); the greedy reading of the
ONNX file agrees with PyTorch on 99–100% of frames. In vocab.json the token <s> (id 0, which this fairseq-trained model uses as its CTC blank) is renamed <blank>. The weights are otherwise unchanged.
Use in VideoLens
VideoLens uses this model as a forced aligner: it places each word of a transcript on the audio (Viterbi over the CTC output). Measured against Meta's MMS forced aligner on the same words: word starts a median 0.02 s apart, 90% within 0.04 s (FLEURS Tamil, read speech); 0.02 / 0.06 s on a film trailer with music.
Licence
MIT, as the original model — see LICENSE.