Instructions to use LattifAI/YuE2-3B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use LattifAI/YuE2-3B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf LattifAI/YuE2-3B-GGUF:Q6_K # Run inference directly in the terminal: llama cli -hf LattifAI/YuE2-3B-GGUF:Q6_K
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf LattifAI/YuE2-3B-GGUF:Q6_K # Run inference directly in the terminal: llama cli -hf LattifAI/YuE2-3B-GGUF:Q6_K
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf LattifAI/YuE2-3B-GGUF:Q6_K # Run inference directly in the terminal: ./build/bin/llama cli -hf LattifAI/YuE2-3B-GGUF:Q6_K
Use Docker
docker model run hf.co/LattifAI/YuE2-3B-GGUF:Q6_K
- Llama new
- LM Studio
- Jan
- Ollama
How to use LattifAI/YuE2-3B-GGUF with Ollama:
ollama run hf.co/LattifAI/YuE2-3B-GGUF:Q6_K
- Unsloth Desktop
- Docker Model Runner
How to use LattifAI/YuE2-3B-GGUF with Docker Model Runner:
docker model run hf.co/LattifAI/YuE2-3B-GGUF:Q6_K
- Lemonade
How to use LattifAI/YuE2-3B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull LattifAI/YuE2-3B-GGUF:Q6_K
Run and chat with the model
lemonade run user.YuE2-3B-GGUF-Q6_K
List all available models
lemonade list
- Atomic Chat
English | 简体中文
YuE2-3B-GGUF
Generate songs for free in EdgeSpeak
No purchase required. Song generation is free. Download EdgeSpeak Desktop and turn your lyrics into songs with vocals and accompaniment on your own computer.
Download EdgeSpeak for free · Usage documentation
- Open Broadcast in EdgeSpeak and select YuE2 3B, marked Free, in the model selector. Follow the prompts to download the model and audio decoder on first use.
- Enter complete lyrics in the Lyrics tab, with section labels such as
[Verse]and[Chorus]. Describe your musical style in the expression instructions. - Let score planning generate a melody and chords, or open the Score tab to view, edit or supply your own ABC notation. Advanced settings expose CFG and inference steps.
- Click Generate to create the song locally, then listen and export the audio. Output is 48 kHz stereo.
Use the desktop interface without writing code, or automate generation through the local OpenAI-compatible API below. Speed depends on hardware, song length and settings; measured performance and model fidelity results are listed below.
Free access does not change the model license: the weights remain under CC BY-NC 4.0. See the license and original-author attribution below.
Quantized GGUF weights of YuE2-3B and its audio decoder YuE2-Vae, prepared by LattifAI for local song generation in EdgeSpeak.
YuE2 turns lyrics into a full song with vocals and accompaniment. Before generating audio it can plan a score (melody, optionally with chords) that you can read, edit and feed back. All credit for the model goes to the YuE2 authors; this repository only changes the numeric format of the weights.
License: CC BY-NC 4.0, non-commercial use only. These files are quantized derivatives of the YuE2-3B and YuE2-Vae checkpoints and keep their original license. See License and attribution.
Files
| File | Source | Format | Size | SHA-256 |
|---|---|---|---|---|
YuE2-3B-Q5_K.gguf |
m-a-p/YuE2-3B | Mixed precision, Q5_K main | 2,572,593,664 bytes | 1b13f9bd41e158e64c71659c01e119e367c6e2bff9fce0f50a2defa206a270f2 |
YuE2-VAEDecoder-Q6_K.gguf |
m-a-p/YuE2-Vae (decoder only) | Mixed precision, Q6_K main | 59,699,264 bytes | 98b19a5d9a15a1296ca9003799ea06e3f2d005cccbb933e1eebbff38ed124583 |
You need both files. The main model (3.58 billion parameters, 28 layers) generates the score and audio latents; the decoder turns latents into 48 kHz stereo audio.
YuE2-3B-Q5_K.gguf: 394 of 628 tensors are quantized with calibration; the remaining tensors keep their original precision.YuE2-VAEDecoder-Q6_K.gguf: decoder weights only (the encoder is not included), quantized with calibration and a post-quantization refinement pass. Its source weights matchmodel.safetensorsinm-a-p/YuE2-Vae(SHA-256807ce9d5149fa27c5ad3e6582058469852e908f6c5acc8c8aa338e7ab7751346).
Runtime
These files use a YuE2-specific tensor layout. Loading them with upstream llama.cpp has not been verified. They are loaded by the EdgeSpeak local engine and have been tested on CUDA (NVIDIA) and Metal (Apple Silicon).
Use with EdgeSpeak
In the EdgeSpeak desktop app, open Broadcast, choose YuE2, write lyrics with section tags such as [Verse] and [Chorus], and click Generate. You can pick how the score is planned, or paste your own ABC score.
The same model is available from the local OpenAI-compatible API:
curl --fail-with-body "$EDGESPEAK_BASE_URL/audio/speech" \
-H "Authorization: Bearer $EDGESPEAK_API_KEY" \
-H "Content-Type: application/json" \
-H "buffered-speech: 1" \
-d '{"model":"m-a-p/YuE2-3B","voice":"builtin:auto","input":"[Verse]\nStreetlights hum along the way\n[Chorus]\nSing it back to me","response_format":"wav","music":{"planning_mode":"full"},"output_container":"zip"}' \
-o generation.zip
music.planning_mode is full (melody and chords, default), melody, or off. music.abc takes your own ABC score. With output_container: "zip" the response holds the audio, the score the model actually used, and metadata. See the API reference.
Quality
Measured against the original BF16 checkpoints on held-out data that was not used for calibration. Arrows show the better direction.
| File | Held-out data | KL divergence ↓ | Top-1 agreement ↑ | Latent cosine ↑ | Latent MSE ↓ | SNR ↑ | LSD ↓ |
|---|---|---|---|---|---|---|---|
YuE2-3B-Q5_K.gguf |
54 records, 29,905 positions | 0.00285 | 95.0% | 0.99968 | 0.000785 | – | – |
YuE2-VAEDecoder-Q6_K.gguf |
6 songs | – | – | – | – | 32.55 dB | 0.908 dB |
These are model-level fidelity metrics. They do not include a word error rate for sung lyrics or a listening test, and they do not measure musical quality.
Speed
Measured in EdgeSpeak with both files on one 22.3-second song (48 kHz stereo):
| Device | Generation time | Real-time factor |
|---|---|---|
| NVIDIA RTX 4090 | 9.09 s (median of 3 runs, model loading excluded) | 0.41 |
Peak GPU memory sampled during these runs was about 5.6 GiB. Apple Silicon (Metal) runs these files but is currently much slower than real time; optimization is in progress. Speed varies with song length, device and settings, so treat these numbers as a single reference point.
License and attribution
The YuE2-3B and YuE2-Vae checkpoint weights are licensed by their authors under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). The files in this repository are adaptations of those weights and are distributed under the same license.
- Original work: YuE2-3B (m-a-p/YuE2-3B) and YuE2-Vae (m-a-p/YuE2-Vae) by the Multimodal Art Projection (M-A-P) team.
- Changes made: weights converted to GGUF and quantized (Q5_K / Q6_K mixed precision); the decoder half of YuE2-Vae extracted on its own. No retraining or fine-tuning.
- Non-commercial: you may not use these files for commercial purposes. For commercial use, contact the YuE2 authors.
- Upstream creator permission: the upstream
MODEL_LICENSEadds a permission for individual creators to monetize outputs generated with YuE2. It does not cover commercial redistribution or sale of the weights. Whether it extends to these quantized files is determined by the upstream terms; read them before relying on it. - The upstream repositories carry separate notices and MIT license texts for the Oobleck / stable-audio-tools and SnakeBeta / BigVGAN code their inference implementation is derived from; see their
THIRD_PARTY_NOTICES.md. This repository contains weights only, no code.
This repository is not affiliated with or endorsed by the YuE2 authors. Please cite their work:
- YuE2 technical report: arXiv:2609.33757
- SheetSage2: arXiv:2610.05336
- Project: github.com/multimodal-art-projection/YuE
- Downloads last month
- 514
6-bit
Model tree for LattifAI/YuE2-3B-GGUF
Base model
m-a-p/YuE2-3B