MixDirective/wav2vec2-base-960h-onnx

wav2vec2-base-960h (ONNX, int8)

An ONNX export of facebook/wav2vec2-base-960h at revision 22aad52d435eb6dbaf354bdad9b0da84ce7d6156, used by the Mix Directive desktop app for local, offline CTC forced alignment of English lyrics to vocal stems. The app runs it with ONNX Runtime on the CPU; no PyTorch is required.

Changes from the original

This is a modified version of the original model, redistributed under the same Apache-2.0 licence:

• Converted from PyTorch to ONNX (opset 17, dynamic sample/frame axes).
• Dynamically quantised to int8 on MatMul nodes only. The convolutional feature encoder stays float32 because ONNX Runtime's CPU provider has no ConvInteger kernel.

No retraining or fine-tuning was done. The weights come from the original checkpoint.

Files

File	Size	SHA-256
model.int8.onnx	121,954,282 bytes	02bcb2e690107934e4636ff883a2623f2f4a186530d42594296df87711789202
vocab.json	—	CTC vocabulary (unchanged from the original)
export_wav2vec2_ctc_onnx.py	—	Script that produced model.int8.onnx

Usage

• Input input_values: float32 [1, samples], mono, 16 kHz, normalised per utterance to zero mean and unit variance: (x - mean) / sqrt(var + 1e-7).
• Output logits: float32 [1, frames, 32], one frame per 20 ms. Apply log-softmax for CTC.
• Vocabulary: <pad> (id 0) is the CTC blank; | (id 4) is the word separator. Characters are upper-case A–Z and apostrophe.

Evaluation

On five songs with hand-corrected word timings (separated lead-vocal stems), forced alignment with this file placed 88.5% of word starts within 0.5 s, with a median error of 0.04 s. The full-precision export scored the same. This is a small, informal sample. The model was trained on read English speech (LibriSpeech), not singing.

Reproduce

Run the included export_wav2vec2_ctc_onnx.py (needs torch, transformers, onnx and onnxruntime). It loads the pinned revision with transformers, exports it with torch.onnx.export, quantises it with onnxruntime.quantization.quantize_dynamic(op_types_to_quantize=["MatMul"]), and checks the output against the PyTorch logits.

Attribution

Original model: facebook/wav2vec2-base-960h by Meta AI, licensed Apache-2.0.

Baevski, A., Zhou, H., Mohamed, A., & Auli, M. (2020). wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. NeurIPS 2020. arXiv:2006.11477

Licence

Apache License 2.0. See LICENSE.