---
license: cc-by-nc-4.0
language:
- ja
- zh
- en
tags:
- forced-alignment
- lyrics
- source-separation
- onnx
---

# PotLyrics lyric alignment models

ONNX models used by PotLyrics (a free, non-commercial desktop lyrics app) to align
line-level lyrics (LRC) to the vocals of a song, character by character, fully offline on the CPU.

Pipeline: `htdemucs.onnx` separates the vocals → lyrics are romanized (Japanese via MeCab + `ipadic`, Chinese via
pinyin) → `mms.onnx` produces CTC emissions and a Viterbi forced alignment gives the time of every character.

| File | What it is | Source | License |
|---|---|---|---|
| `htdemucs.onnx` | Hybrid Transformer Demucs (`htdemucs`), inner network only (STFT/iSTFT are done outside), MatMul weights dynamically quantized to int8 | [facebookresearch/demucs](https://github.com/facebookresearch/demucs) — Rouard, Massa, Défossez, *Hybrid Transformers for Music Source Separation* (ICASSP 2023) | MIT |
| `mms.onnx` | MMS forced-alignment model (`torchaudio.pipelines.MMS_FA`), wav2vec2 without the final log-softmax, MatMul weights quantized to int8 (per channel) | Pratap et al., *Scaling Speech Technology to 1,000+ Languages* (2023), via torchaudio | **CC-BY-NC 4.0** (non-commercial) |
| `mms_vocab.json` | Labels of `mms.onnx` (blank `-`, 26 letters, `'`, plus `*` for the unknown-token column added at runtime) | same as above | CC-BY-NC 4.0 |
| `ipadic.zip` | IPAdic dictionary compiled for MeCab (as shipped by the `ipadic` Python package) | [polm/ipadic-py](https://github.com/polm/ipadic-py), NAIST / ICOT | IPAdic license (permissive, see `ipadic/COPYING`) |
| `manifest.json` | File sizes and SHA-256, read by the app before downloading | — | — |

The repository as a whole is published under CC-BY-NC 4.0 because of the MMS weights. The conversion was done only
for use in PotLyrics; no fine-tuning was applied.

## Accuracy (for reference)

One Last Kiss (Hikaru Utada), aligned from a line-level LRC and compared with the hand-made word timing from the
AMLL TTML database: median error per character 67 ms, 82 % within 200 ms (the non-quantized PyTorch models give
66 ms / 83 %). About 100–130 s per 4-minute song with 8 CPU threads.