# Futhark Hugin - details The short overview is in the [README](../README.md); making your own voice is in [FINETUNE.md](FINETUNE.md). Futhark Hugin is a small Swedish text-to-speech model that also speaks English well, on its own and inside Swedish sentences. It has **15 M parameters**, outputs **24 kHz** audio and comes with one voice, **B1** (`w`). You can add your own voice with `hugin-finetune` ([FINETUNE.md](FINETUNE.md)). The packaged version is **v0.8-en**. | voice | name | model | voice tag | vocoder | |---|---|---|---|---| | `w` (default) | **B1**, woman (synthetic) | `hugin-v0.8-b1` | 1 | Vocos-B1 (8 M, fast) | | your own (`my_voice.pt`) | made with `hugin-finetune` | B1 model adapted to your recordings | 2 | HiFT from S3Gen (20 M, slower) | **New in v0.8-en:** English. Hugin learned it from the in-house "Munin" model (FUT13), which read LibriTTS-R transcripts and our own English and mixed Swedish/English sentences in the B1 voice. The text front-end now finds English sentences by itself (no `` tags needed), reads numbers, times, phone numbers and acronyms inside English text in English, and handles English contractions and compounds ("I'm", "cashback"). On the in-house test bench the letter error rate (Whisper) for voice `w` went from 8.8 % to 0.0 % on English and from 5.5 % to 0.3 % on mixed sentences, with Swedish unchanged (0.4 %). See [VERSIONS.md](../VERSIONS.md). Hugin runs non-autoregressively in a single pass: no sampling, no diffusion steps, no speaker encoder. The pipeline is ``` text -> English sentence detection -> Swedish/English normalisation -> sentence chunks -> phonemes (NST lexicon + G2P + English (CMUdict + English G2P) + homographs + user dictionary) -> word prominence (Stanza part-of-speech, optional) -> Hugin acoustic model (Transformer, durations at 25 Hz, voice tag) -> 80-bin mel at 50 Hz -> Vocos (voice w) or HiFT (own voices) -> 24 kHz ``` It runs many times faster than real time on a GPU and also on an ordinary CPU (see [Performance](#performance)). **Voice B1 is synthetic.** It is not a recording of a real person. It was created with the voice-design feature of [VoxCPM2](https://github.com/OpenBMB/VoxCPM) from the prompt *"A Swedish woman around fifty, warm and slightly low voice, calm audiobook narrator, clear studio recording"*. Hugin was then trained on about 41 h of Swedish speech that VoxCPM2 generated in that voice, plus number clips and (v0.8) about 19 h of English, mixed and Swedish sentences from the in-house "Munin" model reading in the same voice. Voice `w` uses the Munin-B1 voice tag (tag 1), which carries the English best. See [VERSIONS.md](../VERSIONS.md) and [ATTRIBUTION.md](../ATTRIBUTION.md). **Own voices** are made by *frozen-language adaptation*: the B1 model's language part (phoneme embeddings, encoder, duration predictor, encoder voice tags) is frozen and only the voice-carrying part (decoder, postnet, decoder voice tag) is trained on your recordings. So an own voice pronounces and phrases exactly like `w`; only the voice differs. See [FINETUNE.md](FINETUNE.md). ## Listen These samples were generated by this package (v0.8-en) **on CPU** (`Hugin.from_pretrained(device="cpu")`, 12-core x86 VM in Docker, prominence on, after one warm-up call), exactly as you get it after installing. On Hugging Face they can be played directly on the model page. | Sample | Voice | Text (as typed, the front-end normalises numbers, abbreviations and symbols) | Audio | Generated in (CPU) | |---|---|---|---|---| | [01_halsning.mp3](../samples/01_halsning.mp3) | w | Hej och välkommen! Jag heter Hugin, och jag läser svensk text högt. | 5.5 s | 0.22 s | | [02_siffror.mp3](../samples/02_siffror.mp3) | w | Tåget mot Göteborg avgår kl. 19:05 från spår 3, och biljetten kostar 245 kr. Ring 08-123 45 67 om du har frågor. | 14.0 s | 0.40 s | | [03_berattelse.mp3](../samples/03_berattelse.mp3) | w | Det var en tidig morgon i slutet av augusti. Dimman låg fortfarande kvar över sjön, och någonstans långt borta hördes en båtmotor. | 9.1 s | 0.25 s | | [04_nyhet.mp3](../samples/04_nyhet.mp3) | w | Under natten väntas kraftiga vindar längs kusten, och flera färjeturer kan bli inställda redan i morgon bitti. | 7.5 s | 0.21 s | | [05_uttal.mp3](../samples/05_uttal.mp3) | w | Sju sjösjuka sjömän sköljde skjortorna, medan Rolf ringde runt till rektorerna. | 5.8 s | 0.18 s | | [06_partikelverb.mp3](../samples/06_partikelverb.mp3) | w | Hon tog upp frågan på mötet, men chefen höll inte med. Det kan jag inte berätta eftersom det råder förundersökningssekretess. | 8.8 s | 0.30 s | | [07_engelska.mp3](../samples/07_engelska.mp3) | w | Could you send me the updated report before lunch tomorrow? The meeting starts at 10:30. | 5.4 s | 0.11 s | | [08_blandat.mp3](../samples/08_blandat.mp3) | w | Vi har en deadline på fredag, så jag bokar ett kort meeting med vårt product team. Alla kortkunder får cashback. | 7.9 s | 0.24 s | Regenerate them with `python tools/make_samples.py` (writes `samples/*.mp3`, needs `ffmpeg`). ## Install ```bash pip install -U huggingface_hub hf download codah80/futhark-hugin --local-dir futhark-hugin # or: git lfs install && git clone https://huggingface.co/codah80/futhark-hugin cd futhark-hugin pip install -e . # torch, numpy, scipy, soundfile, num2words, huggingface_hub pip install -e ".[server]" # optional: the small HTTP server (fastapi, uvicorn) pip install -e ".[finetune]" # optional: hugin-finetune, to make your own voice (torchaudio, transformers) pip install -e ".[prominence]" && python -c "import stanza; stanza.download('sv', processors='tokenize,pos')" # optional but recommended: word prominence (Stanza + its Swedish model, ~180 MB, one download) ``` **Prominence is optional.** With Stanza installed Hugin de-accents function words and keeps the stress on verb particles ("hon **tog** inte **upp** den"), which fixes most phrasing errors. Without Stanza it runs as before (the particle-verb list and the other rules still apply, but not the part-of-speech step). `prominence="auto"` (default) uses Stanza if it is installed, `prominence=False` / `--no-prominence` turns it off. Stanza looks for its model in `~/stanza_resources` or `$STANZA_RESOURCES_DIR`; nothing is fetched at runtime. The weights live in `weights/` (about 180 MB). An editable install of the downloaded repository finds them on its own. If the package is installed without them, `from_pretrained()` downloads `weights/` from the Hugging Face repository once (cached by huggingface_hub; set `HUGIN_REPO` to use another copy). To use a local folder instead, set `HUGIN_WEIGHTS=/path/to/weights` or pass `path=` to `from_pretrained`. ## Quick start ### Command line ```bash python -m hugin "Hej och välkommen!" -o hej.wav python -m hugin "Lite snabbare nu." -o snabb.wav --speed 1.15 python -m hugin "Det här är min röst." -o min.wav --voice my_voice.pt # your own voice (FINETUNE.md) python -m hugin "Hej och välkomna." -o hej.wav --words # also prints word timestamps (JSON) python -m hugin -f kapitel1.txt -o kapitel1.wav --device cpu python -m hugin --phonemes "Hon heter Malin och bor i Umeå." # show normalised text + phonemes, no audio ``` Options: `--voice w|FILE.pt`, `--words`, `--no-prominence`, `--speed` (>1 is faster), `--device cuda|cpu|cuda:1`, `--threads N`, `--weights DIR`, `--user-lex FILE`, `--swedish-english`, `--no-hpf`, `--jev`. Installing the package also adds a `hugin` command that behaves the same. ### Python ```python from hugin import Hugin import soundfile as sf tts = Hugin.from_pretrained() # weights/ next to the package; cuda if available audio, sr = tts.tts("Tåget mot Göteborg avgår kl. 19:05 från spår 3.") sf.write("tag.wav", audio, sr) # float32 numpy array, sr == 24000 audio, sr = tts.tts("Lite långsammare.", speed=0.9) print(tts.normalize("Kl. 08:15 kostar det ca 250 kr.")) # the text as it will be read print(tts.phonemes("Hej {\"tOn}!")) # phoneme units per chunk audio, sr, words = tts.tts("Hej och välkomna.", words=True) # words = [{"word": "Hej", "start": 0.02, "end": 0.31}, ...] seconds, from the model's own durations mine = Hugin.from_pretrained(voice="my_voice.pt") # your own voice file from hugin-finetune export ``` `Hugin.from_pretrained(path=None, device=None, voice="w", user_lex=None, english_us=True, jev=False, jev_key=None, threads=None, prominence="auto")` **Word timestamps** come straight from the duration predictor, so they are exact for the generated audio (no ASR). Times are in seconds from the start of the returned audio, including the pauses between chunks. ### HTTP server (optional) ```bash python -m hugin.server --port 7880 [--device cuda] [--voice my_voice.pt] curl -s -G --data-urlencode "text=Hej och välkomna." http://127.0.0.1:7880/tts -o hej.wav curl -s -X POST -H "content-type: application/json" -d '{"text": "Hej!", "speed": 1.1}' http://127.0.0.1:7880/tts -o hej.wav curl -s -G --data-urlencode "text=Hej och välkomna." -d format=json http://127.0.0.1:7880/tts # JSON: base64 wav + word timestamps ``` `model` is `w` (default) or the name of an own voice file given to the server with `--voice my_voice.pt` (repeatable; the name is the file name without `.pt`, here `my_voice`). The response has an `X-Model` header. All voices are loaded at start. The server has no authentication and binds to 127.0.0.1 by default. ## Text front-end ### Normalisation (`hugin/frontend/textfront.py`) - **Numbers** are read as separate lexicon words: `250` → "två hundra femtio". Decimals: `12,5` → "tolv komma fem". Thousands written with spaces work (`12 500`). Years: `1987` → "nitton hundra åttiosju", `2026` → "två tusen tjugosex". - **Dates and times:** `3 oktober` → "tredje oktober", `18:30` / `18.30` → "arton trettio", `08:15` → "noll åtta femton". - **Phone numbers** are read digit by digit with pauses between the groups (`08-123 45 67`). **Road names:** `E4:an` → "e fyran". **Ranges:** `1–2 dagar` → "en till två". **Scores** after a result word: `slutade 3–2` → "tre två". - **Abbreviations** such as `t.ex.`, `bl.a.`, `kl.`, `ca`, `dvs.`, `m.fl.`, `fr.o.m.`, `inkl.`, `kr` (about 70 in total). A capital letter is kept (`Kl.` → "Klockan"). - **Units** right after a number: `5 km`, `25 °C`, `3 kg`, `90 km/h`, `10 kWh`. - **Symbols:** `%`, `&`, `§`, `$5`, `€`, `£`, `+`, `−`, `×`, `÷`, `=`, `≈`, `±`, `½`, `#3`, `#hashtag`, `info@x.se` ("snabel-a"), `exempel.se` ("exempel punkt se"), version numbers (`6.1` → "sex punkt ett"). - **English text** (v0.8-en voices): sentences that are mostly English are detected and tagged `` automatically (English function words and words only in CMUdict vs Swedish ones; å/ä/ö means Swedish; a sentence that is undecided inherits the previous sentence's language, also across lines). Inside `` numbers, clock times (`10:30` → "ten thirty"), phone numbers, e-mail addresses, years, ordinals, decimals, `$`/`€`/`£`, `%` and acronyms (`MVP` → "em vee pee") are read in English. Tags spanning several sentences or lines are closed and reopened per sentence. - **Acronyms** are spelled out letter by letter (`SMHI` → "ess emm hå i") unless they are pronounced as words (`NATO`, `IKEA`, `USA`, ...). Mixed letter/digit codes are spelled (`N3A`, `4G`, `B2B`). Spelled letters keep their stress and get a short pause (~0.16 s) after the last letter. A letter next to digits is read as a letter: `F1` → "eff ett", `v2` → "version två", `3D` → "tre de". - **Quotes:** typographic quotes (`”“„«»’‘`) are treated like plain `"` and `'`; double quotes carry no sound and are dropped. - **Punctuation:** colons and semicolons become comma pauses, dashes and parentheses become pauses, and runs of repeated letters are collapsed (`neeeeej`). If the text has no closing punctuation, a full stop is added, because short inputs like "hej" get cut otherwise. - **Chunking:** each line is a paragraph (0.7 s pause after it). The text is split into sentences, and short sentences are joined into chunks of up to about 110 characters. A sentence longer than 300 characters is split at commas, else just before a conjunction (*och, men, som, att, eftersom, ...*), and at spaces only as a last resort. ### Pronunciation (`hugin/frontend/phonfront.py`) Words are looked up in this order: 1. Inline `{SAMPA}`: exact NST X-SAMPA for that spot, for example `Tre {"tOn} grus.`. Use `""` for accent 2, `"` for accent 1, `%` for secondary stress and `$` for syllable boundaries. 2. Inline `{=respelling}`: a Swedish respelling read through the lexicon/G2P, for example `{=jomms-hof}`. A hyphen marks a syllable boundary. 3. `...` (added automatically for English sentences, see above): the tagged words get an English reading, even if they also exist in Swedish (`machine learning`). Hugin has real English sounds (14 extra phoneme units such as /D/, /T/, /w/, /z/, /r\/; v0.8 "en_v2" also separates unstressed American *-er* from schwa). English words outside CMUdict are split into two CMUdict words when possible (*walkthrough*, with English compound stress) or go through a small English G2P model (`weights/g2p_en.fp16.pt`). Untagged words: English contractions (*I'm, don't*) stay one English word, an unknown word made of two English words is read as English (*cashback, smartwatch*) unless it also splits into two Swedish words and does not look English (*hundbur*). 4. Homographs (`lex/homographs.json`, 24 words such as *drog, kort, rest, planet, tom, oliver*). Offline rules: a capitalised word in mid-sentence is read as a name ("Oliver sa"), and "rest" after a form of *ha* is read as the verb. With Stanza installed, the part of speech then picks the reading for *tom/tomt, kort, fort, kör, driver, syntes, vikt, rosa, format, rest, bete, delta, genom, drog, oliver, dom, modern* ("ett kort" at the end of a phrase = a card). Without Stanza, or for words whose readings share a part of speech (*banan, planet*), the lexicon default applies, or a choice from the optional Jev API (below). 5. Person names whose name reading differs from the lexicon (SCB name statistics), when they are capitalised in mid-sentence ("pratade med Albin"). 6. Context rules, for example "tre ton" (weight) vs "en ton" (music). 7. **User dictionary** (`weights/lex/user_lex.json`, or your own file via `user_lex=` / `--user-lex`): ```json { "kantats": "[\"\"kan$tats]", "ddos": "didåsa" } ``` A value in `[...]` is exact SAMPA. Any other value is a respelling. A value starting with `!` is forced: it also wins over the English reading of words that CMUdict knows (otherwise old Swedish-sound respellings of English words such as `"of": "ov"` are skipped by the English-unit voices). `!1.3[...]` carries a tempo factor used by Munin; Hugin ignores the factor and uses the SAMPA. The file is re-read when it changes. The shipped `user_lex.json` is a copy of the production dictionary at release time. 8. The NST lexicon (about 815k word forms), then CMUdict for words outside the Swedish lexicon **only if they look English** (a character 4-gram model, `weights/lex/langchar.json.gz`, or a short loanword list; so *extra, mesta* stay Swedish) (American-style by default; `english_us=False` / `--swedish-english` reads them the way a Swede would), then **compound splitting** for long unknown words (lexicon parts with an optional linking *s*; accent 2 on the first part, secondary stress on the rest; *kort* in a compound means card, *kortkunder*, unless the next part makes it "short", *kortsiktig*), then the G2P model for everything else. **Prominence (`hugin/frontend/prominence.py`):** with Stanza, function words (pronouns, articles, prepositions, conjunctions, auxiliaries) lose their word accent, while verb particles and stranded prepositions keep it and get a minimum vowel length. A built-in particle-verb list covers the cases Stanza misses (*ta upp, få till, hålla med, ...*), *de/dem* are read "dom", and "för" as a verb ("han för protokollet") keeps its stress. **Jev (optional, off by default):** Jev is an external LLM service (api.typesafe.ai) that picks the meaning of a homograph from the sentence. It is only used when `jev=True` / `--jev` is set and a key is provided via `jev_key=` or the `HUGIN_JEV_KEY` environment variable. No key is shipped with the package. When Jev is off, homographs get the lexicon default, apart from the offline rules above. ### Synthesis details (`hugin/synth.py`) These details are the same as in the production API. - **Sentence-start warm-up:** the model sounds slightly "digital" in the first moments of a chunk. So every chunk is generated with "Okej." and a long (at least 480 ms) pause in front of it, and the audio is cut in the middle of that pause. A chunk that follows one without a closing `.!?…` (a long sentence split at a comma) gets that chunk's last 3 words as warm-up instead, and is cut 40 ms before its first sound. - **Voice tag:** the model has a learned source/voice tag (0 = VoxCPM2 B1 data, 1 = Munin-B1 data, 2 = an adapted own voice); `w` uses tag 1 (the tag that heard all the English), own voices tag 2. - **English sentences:** a chunk whose words are at least 80 % ``-tagged skips the Swedish prominence step (Munin taught English without it). The text front (`en_v2`) is stored in the model file (`info["front"]`), so older model files keep the v0.7 behaviour. - **Vowel clash:** a one-vowel word before a word that starts with a vowel ("fyller i inkomst") gets a minimum length and a short break. - **Edges:** each chunk gets 20 ms of silence, a 20 ms raised-cosine fade-in and a 10 ms fade-out, with 0.12 s of silence between chunks. - **High-pass filter:** the whole output goes through a 4th-order Butterworth high-pass at 80 Hz, because the B1 voice has sub-bass rumble. Turn it off with `hpf=False` / `--no-hpf`. - **Speed:** all durations are divided by `speed`. ## Performance **v0.8-en** has the same architecture and size as v0.7, so speed is unchanged. Measured on the Intel i7-14700K VM (12 vCPUs, default 6 threads, torch 2.11, prominence on, 55 s of mixed Swedish/English text, after warm-up): v0.8 voice `w` RTF 0.025 (~41× real time), v0.7 with the same code 0.026 (~39×). **v0.7 (measured on the NVIDIA GB10 CPU, 10 threads, after warm-up):** voice `w` ~40× real time with prominence, ~55× without (Stanza tagging costs about a quarter). Own voices use the HiFT vocoder, which is much heavier than Vocos: about 4–5× real time on CPU, so run them on a GPU when speed matters. Older figures for v0.5 (voice `w`, no prominence): Each figure is measured after a warm-up call, on long texts (51 s and 118 s of audio). RTF is synthesis time divided by audio duration, so lower is faster. | device | RTF | speed | |---|---|---| | RTX 4070 (CUDA, torch 2.11) | 0.002–0.004 | ~250–500× real time | | NVIDIA GB10 / DGX Spark CPU (aarch64, 20 cores, default 10 threads, torch 2.10) | 0.017 | ~58× real time | | Intel i7-14700K VM, 12 vCPUs, default 6 threads (torch 2.11) | 0.017 | ~60× real time | | same VM, all 12 threads while other jobs were running | 0.45 | thread oversubscription | By default Hugin uses half the CPU cores on CPU (`threads=` / `--threads` overrides this, and so does `OMP_NUM_THREADS`). With every core in use it can be many times slower. The first call takes longer (CUDA start-up, G2P loading, and the first homograph / English-detection lookups, up to about 1 s), and loading the model takes about 2 s. ## Files ``` hugin/ model.py (Elev acoustic model), vocos.py, hift.py, synth.py (class Hugin, VOICES), cli.py, server.py hugin/finetune/ hugin-finetune: prepare.py, align.py, audio.py, train.py, export.py, cli.py hugin/frontend/ textfront.py, phonfront.py, prominence.py, entags.py (English sentence detection), langchar.py, g2p.py, english.py, sampa.py weights/ hugin-v0.8-b1.fp16.pt vocos-b1-40k.fp16.pt hift-s3gopen.fp16.pt (vocoder for own voices) g2p.fp16.pt g2p_en.fp16.pt weights/lex/ lex_full.json.gz (NST) cmudict.dict.gz langchar.json.gz homographs.json names.json names_scb_alt.json user_lex.json tools/ export_weights.py (how the fp16 files were made from the training checkpoints), make_samples.py tests/ test_smoke.py, test_finetune.py docs/ DETAILS.md (this file), FINETUNE.md ``` The weights are stored as fp16, with tiny vectors kept in fp32, and are loaded back to fp32 at runtime. Durations, and so the audio length, are identical to the original fp32 checkpoints; the waveform correlates at 0.998–0.99999 per sentence for v0.8 (0.99999 for v0.7). Loading the fp32 checkpoints into this code gives the production output exactly (correlation 1.0). ## Tests ```bash pip install pytest && python -m pytest -q tests # HUGIN_DEVICE=cpu to force CPU ``` `tests/test_finetune.py` also runs `hugin-finetune prepare` end to end when transformers, torchaudio and the KBLab wav2vec2 model (in the Hugging Face cache) are available; otherwise that test is skipped. ## License The code and the model weights in this repository are released under the [Apache License 2.0](../LICENSE). The third-party data and models used to build Hugin keep their own licences; they are listed, with the required credits for CC BY sources, in [ATTRIBUTION.md](../ATTRIBUTION.md).