lyric-studio-models / README.md
SDIDSA's picture
Lyric Studio models
2e0a1ae verified
|
Raw History Blame Contribute Delete
1.75 kB
metadata
license: mit
tags:
  - audio
  - source-separation
  - forced-alignment
  - onnx
  - tflite

Lyric Studio models

The models the Lyric Studio Android app downloads on first use, to time song lyrics on the phone. The app checks each file's size and SHA-256 before using it.

File What it does Runs with Based on
vocals_mdx9482.onnx Isolates the vocals from a song (MDX-Net, 44.1 kHz stereo, 6-second chunks) ONNX Runtime, CPU UVR-MDX-NET UVR_MDXNET_9482 from Ultimate Vocal Remover, via the MIT-licensed Politrees/UVR_resources
vocals_mdx9482.tflite The same model converted for the phone's GPU (input [1, 4, 2048, 256], output [1, 2048, 256, 4]) LiteRT GPU delegate As above, converted with onnx2tf
sync_int8.onnx Letter probabilities for each 20 ms of the isolated vocals, used to align the lyrics (8-bit) ONNX Runtime, CPU facebook/wav2vec2-base-960h (Apache-2.0)
sync_wav2vec2.tflite The same sync model at full precision with 16-bit weights, for the phone's GPU (input [1, 480000], one normalised 30 s window; output [1, 1499, 32] logits) LiteRT GPU delegate As above, converted with LiteRT Torch. Its grouped positional convolution is split into plain ones, and its attention scores and norms are computed so that 16-bit arithmetic cannot overflow (same results at full precision)

Licences

  • The vocal models keep the licence of their source, which the mirror above lists as MIT. Credit to the Ultimate Vocal Remover project and the MDX-Net authors.
  • The sync model is derived from facebook/wav2vec2-base-960h, released under Apache-2.0.