futhark-hugin / VERSIONS.md
codah80's picture
Hugin v0.8.1: B1 voice only, hugin-finetune (own voice), short model card
fd13602 verified
|
Raw History Blame Contribute Delete
7.2 kB

Futhark Hugin - versions

Packaged in this repository: v0.8-en (package version 0.8.1)

file from voice
weights/hugin-v0.8-b1.fp16.pt elev/runs/hugin-v0.8-en/ckpt_176000.pt w (B1), tag 1, front en_v2
weights/vocos-b1-40k.fp16.pt runs/vob1/vocos_40000.pt vocoder for w (unchanged since v0.5)
weights/hift-s3gopen.fp16.pt out/runs/s3gopen/s3gen_8000.pt (the mel2wav.* HiFT part) vocoder for own voices made with hugin-finetune (unchanged since v0.7)
weights/g2p.fp16.pt, weights/g2p_en.fp16.pt novel/g2p/g2p.pt, novel/g2p/g2p_en.pt Swedish / English G2P (unchanged)
weights/lex/langchar.json.gz novel/lex/langchar.pkl (same counts, stored as JSON, no pickle) English-vs-Swedish character model (new in v0.8)

Earlier model files (v0.7-b1, v0.5-b1) were packaged before and removed from the tree.

0.8.1 (2026-10-13): one voice + hugin-finetune

  • The public package ships one voice, B1 (w). The model weights are unchanged (v0.8-en).
  • New hugin-finetune (hugin/finetune/, extra [finetune]): prepare (KBLab wav2vec2 CTC forced alignment onto Hugin's own units), train (frozen-language adaptation of the B1 model, tag 2, mel GAN, resumable) and export (a voice file with the HiFT vocoder). Hugin.from_pretrained(voice="my_voice.pt"), python -m hugin --voice my_voice.pt and python -m hugin.server --voice my_voice.pt load such files. Guide: docs/FINETUNE.md.
  • README is now a short model card; the full documentation moved to docs/DETAILS.md.

v0.8-en (2026-10-13)

Model w (hugin-v0.8-en): Hugin v0.7-pos 152k fine-tuned to 176k (24k steps, LR 1.2e-4, n_src 2, about 49 min on an RTX 5090) on all v0.7 data plus about 18.7 h of new clips (12,157 shards: 7,675 English, 2,307 mixed Swedish/English, 1,835 Swedish, 340 own pure-English sentences) read by the in-house Munin FUT13 model (token model + Swedish S3Gen "s3gopen" + HiFT) in the B1 voice (reference clip from the VoxCPM2 B1 voice). Texts: 8,000 single sentences from the LibriTTS-R normalized transcripts (train-clean subsets), our own pure-English and mixed sentences (written for the project), and 2,000 Swedish sentences from the B1 text set; each take was checked with Whisper (letter CER <= 0.06 en / 0.10 mix / 0.05 sv, two takes, ~7.5 % rejected). Munin's own durations were used as targets (no forced alignment). The new clips use voice tag 1 (Munin-B1; the tag earlier used for Munin number clips), and the owner picked tag 1 for w ("tagg 1 låter bäst"): under tag 0 English is less robust. B1 validation mel loss stayed flat (no Swedish drift). Two new units (o, ¤; harmless).

Test bench (Whisper letter error, in-house bench, sv 10 / en 12 / mix 20 sentences): v0.7 w sv 0.4 % / en 8.8 % / mix 5.5 %, English expressions right 9/23 -> v0.8 tag 1 sv 0.4 % / en 0.0 % / mix 0.3 %, 23/23 (the Munin teacher itself: 19/23).

Text front-end ("en_v2", stored as info["front"] in the model file; older model files keep the v0.7 path):

  • English sentence detection (hugin/frontend/entags.py, copied from the Munin engine): balance_en_tags closes/reopens <en> per sentence and line; auto_en tags sentences that are mostly English; undecided sentences inherit the previous one's language.
  • normalize_en_spans: numbers, clock times, phone numbers, e-mail, years, ordinals, decimals, currency, % and acronyms inside <en> read in English.
  • English units v2: unstressed ER0 -> @ r\, words with only secondary stress get a primary stress.
  • Whole-English chunks (>= 80 % <en> words) skip the Swedish prominence step.
  • CMUdict-only words are read as English only if they look English (character 4-gram model langchar, or an English loanword list); English contractions are one word; unknown English compounds (cashback) read as English with English compound stress; kort = card in compounds (kortkunder, kortnummer) unless the next part means "short".
  • Homographs without Jev: Stanza part of speech picks the reading (tom, kort, fort, kör, ...), "ett kort." = a card.
  • Spelled letter/digit codes (N3A, 4G), USA read as a word, quote marks dropped, "sa till henne att ..." not a particle verb.
  • User dictionary: ! entries are forced (also over English readings); a tempo factor !1.3[...] is accepted and ignored by Hugin. weights/lex/user_lex.json updated to the production copy of 2026-10-13.
  • The warm-up for a chunk that follows one without a closing .!?… is now taken as in production (also across a line break, acronym marks kept).

Parity with the production API (2026-10-13): units fed to the model (warm-up + chunk units + duration floors) identical for all 178 chunks of 171 test texts (bench sv/en/mix, 100 held-out v0.8 sentences, 29 edge cases), on GPU and CPU; audio length identical, waveform correlation 0.998-0.99999 per sentence (fp16 weights; 1.0 with the fp32 checkpoints).

Original version notes (copied from the training machine, elev/runs/HUGIN_VERSIONS.md):

Steps count from the start of the whole student line (old Futhark base f2 = 56k).

  • v0.1-b1 (dir hugin-v0.1-b1, was b1ft): f2 56k fine-tuned on VoxCPM2 "B1" synthetic voice, 41 h, steps 56k-76k.
  • v0.2-b1 (hugin-v0.2-b1, was b1ft2): + short phrases + numbers (VoxCPM2), 80 Hz high-pass on data, steps 76k-88k.
  • v0.3-b1 (b1ft3 -> hugin-v0.3-b1 when done): numbers from Munin (FUT3S + s3gopen, B1 voice), VoxCPM2 clips with numerals removed, padding fix, steps 88k-100k. Vocoder for B1: runs/vob1 (Vocos on real B1 audio, init vo2).
  • v0.4-b1 (hugin-v0.4-b1): v0.3 100k + SOURCE TOKEN (elev.py n_src=2: 0 = VoxCPM2 B1, 1 = Munin-B1 number shards >= 1000; zero-init; inference uses 0) to stop the timbre switching to Munin on numbers. Steps 100k-112k. Same data as v0.3 (Munin shards x3).
  • v0.4 CRASHED at 102k before saving (eval collate bug after adding src) - dir removed; merged into v0.5.
  • v0.5-b1 (hugin-v0.5-b1), packaged until v0.7: v0.3 100k + source token (n_src=2) + regenerated greeting clips (best-of-8 by F0, x3) replacing 69 odd old short greetings; Munin numbers x3. Steps 100k-120k. Data /w/elev/teacher_b1_v5.
  • v0.6-en (hugin-v0.6-en): v0.5 + real English phoneme units (14 new tokens, embeddings grown from the nearest Swedish unit) trained on VoxCPM2 B1 English/mixed sentences; English G2P model for words outside CMUdict. Steps 120k-132k. (An earlier v0.6 attempt that also carried VoxCPM number spellings like "fyrtio en" into Hugin was rejected: Hugin already said numbers right.)
  • v0.7-b1 (hugin-v0.7-pos), packaged as voice w until v0.8: v0.6-en 132k + word prominence in the training text (Stanza Talbanken POS: function words de-accented, verb particles/stranded prepositions kept, minimum vowel floors on "full" words, particle-verb list). Steps 132k-152k. Front-end fixes in the same release: compound splitting, acronym marking + pause, de/dem -> "dom", "för" as a verb, letter+digit reading, typographic quotes, 300-char splits before conjunctions, paragraph pauses, word timestamps.
  • v0.8-en (hugin-v0.8-en) - PACKAGED HERE as voice w (tag 1): see the section above.