lyric-studio-models / README.md
SDIDSA's picture
Lyric Studio models
2e0a1ae verified
|
Raw History Blame Contribute Delete
1.75 kB
---
license: mit
tags:
- audio
- source-separation
- forced-alignment
- onnx
- tflite
---
# Lyric Studio models
The models the Lyric Studio Android app downloads on first use, to time song lyrics on the phone.
The app checks each file's size and SHA-256 before using it.
| File | What it does | Runs with | Based on |
|---|---|---|---|
| `vocals_mdx9482.onnx` | Isolates the vocals from a song (MDX-Net, 44.1 kHz stereo, 6-second chunks) | ONNX Runtime, CPU | UVR-MDX-NET `UVR_MDXNET_9482` from Ultimate Vocal Remover, via the MIT-licensed [Politrees/UVR_resources](https://huggingface.co/Politrees/UVR_resources) |
| `vocals_mdx9482.tflite` | The same model converted for the phone's GPU (input `[1, 4, 2048, 256]`, output `[1, 2048, 256, 4]`) | LiteRT GPU delegate | As above, converted with onnx2tf |
| `sync_int8.onnx` | Letter probabilities for each 20 ms of the isolated vocals, used to align the lyrics (8-bit) | ONNX Runtime, CPU | [facebook/wav2vec2-base-960h](https://huggingface.co/facebook/wav2vec2-base-960h) (Apache-2.0) |
| `sync_wav2vec2.tflite` | The same sync model at full precision with 16-bit weights, for the phone's GPU (input `[1, 480000]`, one normalised 30 s window; output `[1, 1499, 32]` logits) | LiteRT GPU delegate | As above, converted with LiteRT Torch. Its grouped positional convolution is split into plain ones, and its attention scores and norms are computed so that 16-bit arithmetic cannot overflow (same results at full precision) |
## Licences
- The vocal models keep the licence of their source, which the mirror above lists as MIT. Credit to the
Ultimate Vocal Remover project and the MDX-Net authors.
- The sync model is derived from `facebook/wav2vec2-base-960h`, released under Apache-2.0.