dkaraush/omnivoice-lora-uk

OmniVoice LoRA — Ukrainian (uk)

Fine-tuned LoRA adapter for OmniVoice, adapted for Ukrainian speech synthesis and zero-shot voice cloning, with additional dictionary-guided stress training.

This LoRA is designed to help OmniVoice follow stress marks in Ukrainian text more accurately.

Training Details

• Base model: k2-fsa/OmniVoice (Qwen3-0.6B backbone)
• Method: LoRA (rank=16, alpha=32, dropout=0.05)
• Target modules: Language-model attention Q/K/V/O and MLP gate/up/down projections; audio embeddings and output heads also trained
• Training data: ~350 hours of Ukrainian speech samples with placed stress marks in text
• Steps: 20,150
• Precision: bf16
• Hardware: 4x RTX 6000 Pro Max-Q

Usage

import torch
import soundfile as sf
from omnivoice import OmniVoice
from peft import PeftModel

# Load the base model.
model = OmniVoice.from_pretrained(
    "k2-fsa/OmniVoice",
    device_map="cuda:0",
    dtype=torch.float16,
)

# Load the adapter on the full model, including its trained audio heads.
model = PeftModel.from_pretrained(model, "dkaraush/omnivoice-lora-uk")
model = model.merge_and_unload().eval()

# Generate
with torch.inference_mode():
    audio = model.generate(
        text="Ба́ндами, послу́хай, новачки́, сте́жити, мальо́вана, раді́й, тридцяти́, поме́рти, яку́, це́гла, ім’я́, кві́тами, ду́мки, думки́, а́тлас, атла́с, бра́ти, брати́, пла́чу, плачу́, ко́си, коси́, ру́ки, руки́, повтори́.",
        language="uk",
    )[0]

sf.write("ukrainian.wav", audio, 24000)