Ruach Studio: a full studio on YuE2 (yue2.cpp), and what we measured on the way
Thank you for YuE2. We built a studio on it, through yue2.cpp and over IronWolve's YuE2 Kit:
- five rooms: Creator, Writer, Refiner, Librarian, LoRA Trainer;
- a score-first workflow;
- our own LoRA trainer.
It is open (AGPL): https://github.com/igrbible/Ruach_Studio
Along the way we measured a few things that may help anyone working with YuE2. All of it was measured on RTX 3090s,
by ear and by numbers.
Style prompts: short tags beat long descriptions. Write the style the way an SD/SDXL prompt is written: short
comma-separated tags, the most important first:- the language and the genre with its era;
- the defining instruments and the lead voice;
- the mood and the production;
- the tempo (BPM) last.
For example: "cyberpunk electro-pop, modern synthwave, aggressive bassline, energetic male vocals, dramatic
intro, electronic hip-hop beat, catchy synth hooks, 125 BPM". This gave a striking song where long prose
descriptions gave less.Guidance (CFG).
- 1.4–1.6 is the steady ground.
- 2.0 moves more (dynamics).
- 2.4 is the most that stays stable.
- From about 2.6, the vocals break and music tokens get cut.
- Exactly 1.0 means a single pass.
LoRA strengths.
- The music half's strengths, stacked: the score came out whole up to 2.25 together in every run, and broken from
2.5 in every run. - The sound half: two adapters at 1.0 each (2.0 together) brought a heavy bass; about 0.55 together sounded right.
- An adapter trained in Direct mode (no score) broke Full-mode scores in its late epochs, even at 0.5 on the music
half. In Full, give such an adapter the sound half only.
- The music half's strengths, stacked: the score came out whole up to 2.25 together in every run, and broken from
An instrumental-score adapter on a sung song. At 1.00 with words to sing, the score lost its vocal line: 1 note
in 127 vocal bars, the music looping. Keep it at 0.3–0.5 for a sung song.The metallic buzz is the VAE's frame seam.
- The 4–12 kHz envelope folded by 1920 samples shows a 20 % pattern peaking at the seam (5 %, noise, folded by
1900 or 1940). - It is the same with the standard, legacy and blended decoders, so changing the decoder does not cure it.
- Dividing the highs by the frame's profile (over 50 frames) does. About 80 % strength sounds best.
- The 4–12 kHz envelope folded by 1920 samples shows a 20 % pattern peaking at the seam (5 %, noise, folded by
Blending the two decoders is not "the best of both".
- Above 4 kHz the decoders differ in phase (correlation 0.3–0.15), and the mix dulls the top by 1–1.5 dB.
- We render with standard plus debuzz.
Instruments. YuE2 does not know the shofar (it gives another pipe or horn every time), the oboe or the gong.
"orchestral" before an instrument builds a whole orchestra around it, and the instrument is lost. Our cheat-sheet
has 200 instruments with the ear's verdict on each, and the A/B probes come with the studio.Long readings. A long text eats the context before a note sounds: three chapters (11.5k characters) took 5.7k
tokens of prefix. The model then stops early and skips. Feed long texts in parts.
If any of this is wrong on your side, we would like to know. The numbers are ours, not laws.
Where Ruach Studio stands (October 16), and what to expect
A short update for those following this thread.
Releases. Release candidates rc2 to rc8 are out; rc8 came on October 15 (GitHub, page). There will be no more RCs. Next is 2.0.0, and until then only bug fixes go in.
What changed in the studio:
- Twelve interface languages: English, Russian, Ukrainian, Belarusian, Greek, Spanish, Italian, French, Portuguese, German, Japanese and Chinese. The built-in guide is in all of them.
- The Artist room makes cover art on the same machine (Qwen-Image).
- LoRAs:
- Each adapter can follow a strength curve through the song: flat, an arch, or drawn by hand.
- A budget makes adapters that push on the same layers share them instead of piling up.
- A broken score stops the run before any audio is rendered.
- The Refiner:
- It splits a take into 6 or 8 stems and mixes them in a small DAW, which will be a preview in 2.0.0.
- It takes steps of your own as plugins.
- Content Credentials (C2PA) on downloads:
- Each file carries a signed record that it was made by AI (YuE2) in Ruach Studio, and which edits a person made after that.
- Signing can be switched off, and you can sign with your own certificate.
What we measured (rough numbers, RTX 3090s):
- Keys with C♭ or B♯ in their signature (A♯ minor, for example): YuE2 sang those notes natural, 9 to 17 % of the notes in our probes. The studio now writes such a score in the enharmonic key, with the same pitches (B♭ minor for A♯ minor).
- Merged weights: folding a LoRA into the weights gives the same audio as loading it at runtime, byte for byte, with the same seed and score.
- A trailer-style adapter, music half: at strength 1.0 it broke the ABC score in 9 runs out of 9 (no
K:line). At 0.5 to 0.75 every score held. - The same adapter's sound half on top of Real Audio raised the hiss, even at 0.32 to 0.4.
A merged checkpoint you can try: goldhub/YuE2-3B-Ruach_Studio-IRA-Merge.
- What it is: YuE2-3B with two of Mothersuperior's adapters folded in: Instrumental on the music half (0.75) and Real Audio v9 on the sound half (0.85).
- Formats: safetensors in m-a-p's layout, and GGUF (BF16, Q8_0, Q6_K) for yue2.cpp.
- Licence: CC BY-NC 4.0, like the base model.
- The card gives the method and our preliminary checks.
- In Ruach Studio: the studio downloads it and offers it beside m-a-p's weights.
- What it does best: sung songs. A pure instrumental can carry a little noise.
Next:
- 2.0.0, a polishing release, with the DAW as a preview.
- After that, text-to-speech readings (2.1 / 2.2) and a Mac build.
- Linux stays first.
Thanks to m-a-p for YuE2, to Serveurperso for yue2.cpp, and to the adapter authors whose work we build on.