vivi-ai-models

Model weights used by the Vivi Music desktop app's on-device AI helper (vivi-ai). The app uses them for karaoke (vocal removal), playing the vocal or instrumental track on its own, and syncing lyrics automatically. The files are converted copies of existing open models; mms_int8.bpk is an 8-bit quantized derivative of mms.bpk. No weights were retrained or fine-tuned.

File	Size	SHA-256	Original model	License
mdx_inst_hq3.bpk	66,770,192 B	a9252b735512ec527f8a09c0356f64dd723fd2d5526dd009b925a9448cfcbf7b	UVR-MDX-NET-Inst_HQ_3	MIT (UVR)
mms.bpk	1,261,917,308 B	ee287d471965876671bdcd137e080a3c618965fef0e921b2d6846d4947b86ea5	MahmoudAshraf/mms-300m-1130-forced-aligner (based on Meta MMS)	CC-BY-NC-4.0
mms_int8.bpk	505,566,076 B	8090b023c3e1a1931b47272046064414ba2e9337335d0219ee5653d6904474db	same as mms.bpk, 8-bit quantized	CC-BY-NC-4.0

The checksums are also listed in SHA256SUMS and manifest.json.

Changes made

Both models were converted from ONNX to the Burn record format (.bpk, burn 0.22) with burn-onnx, using vivi-ai convert <model.onnx> <out.bpk>. The conversion changes only the file format and tensor layout. The weights are numerically unchanged (except in mms_int8.bpk, see below).

• mdx_inst_hq3.bpk was converted from UVR-MDX-NET-Inst_HQ_3.onnx, distributed through the Ultimate Vocal Remover model repository (TRvlvr/model_repo).
• mms.bpk was converted from an ONNX export of MahmoudAshraf/mms-300m-1130-forced-aligner (wav2vec2 CTC, 16 kHz).
• mms_int8.bpk is mms.bpk with the weights of all linear layers quantized to 8 bits (symmetric int8, one fp32 scale per block of 8 weights, burn native quantization, vivi-ai quantize --in mms.bpk --out mms_int8.bpk --block 8). This changes the weights. Convolutions, layer norms and biases stay fp32. It is 40 % of the size and needs much less memory. On a 3:43 test song, 99 % of the word start times were within 20 ms of mms.bpk (as close as two mms.bpk runs on different GPUs).

Credits and licenses

UVR-MDX-NET-Inst_HQ_3

• Trained by the Ultimate Vocal Remover (UVR) team: Anjok07, aufr33 and contributors. See https://github.com/Anjok07/ultimatevocalremovergui and https://github.com/TRvlvr/model_repo.
• The architecture is MDX-Net (KUIELab, Kim et al., "KUIELab-MDX-Net: A Two-Stream Neural Network for Music Demixing", 2021).
• License: MIT, as for the UVR project. The UVR authors ask: "For all third-party application developers who wish to use our models, please honor the MIT license by providing credit to UVR and its developers." This card provides that credit.

MMS forced aligner

• The base model is Meta AI's Massively Multilingual Speech (MMS): Pratap et al., "Scaling Speech Technology to 1,000+ Languages", 2023. See https://github.com/facebookresearch/fairseq/tree/main/examples/mms.
• The forced-alignment checkpoint is MahmoudAshraf/mms-300m-1130-forced-aligner by Mahmoud Ashraf: https://huggingface.co/MahmoudAshraf/mms-300m-1130-forced-aligner.
• License: Creative Commons Attribution-NonCommercial 4.0 International (CC-BY-NC-4.0), https://creativecommons.org/licenses/by-nc/4.0/. You may redistribute it, including in modified form, provided that you give attribution, indicate the changes (see above), and do not use it for commercial purposes. mms_int8.bpk is a modified (quantized) version of this model and is distributed under the same license with the same attribution. mms.bpk and mms_int8.bpk may not be used commercially.

Usage

The Vivi Music desktop app downloads the files from https://huggingface.co/<repo>/resolve/main/<file>. It resumes interrupted downloads and checks every file against its SHA-256 before use. The repo id can be changed in Settings > AI features.