| # ComfyUI-OGG |
|
|
| [OGG TTS](https://huggingface.co/FashionFlora/ogg-tts) (0.8B, block diffusion, 8 languages, 24 emotion and vocal tags) as **self-contained** |
| ComfyUI nodes: the model, the Fish codec it speaks in, voice cloning, decode, an editor and Qwen3-ASR for transcripts. No other node pack is needed |
| (it can be installed next to ComfyUI-Drama3: ids and types differ). |
|
|
| ``` |
| Load OGG TTS ───────────────────────────────┐ |
| Load OGG Codec ─┬─► OGG Encode Voice ──► voice ─► OGG TTS Generate ─► codes ─► OGG Decode / OGG Editor ─► audio |
| Load Audio ─────┘ ▲ Qwen3-ASR Transcribe |
| ``` |
|
|
| ## Install |
|
|
| 1. `cd ComfyUI/custom_nodes && git clone <this repo>`, restart ComfyUI. Nothing to pip install (uses the `transformers` ComfyUI requires). Needs ComfyUI ≥ 0.40. |
| 2. Files: |
|
|
| | File | Folder | |
| |---|---| |
| | OGG TTS `model.safetensors` from [FashionFlora/ogg-tts](https://huggingface.co/FashionFlora/ogg-tts) (rename it, e.g. `ogg_tts.safetensors`) | `models/diffusion_models` | |
| | `drama3_codec.safetensors` from [Drama-3-ComfyUI](https://huggingface.co/PedroMarinhoDev/Drama-3-ComfyUI) | `models/vae` | |
| | `qwen3_asr_1.7b_bf16.safetensors` or `_0.6b_` from [Qwen3-ASR-ComfyUI](https://huggingface.co/PedroMarinhoDev/Qwen3-ASR-ComfyUI) (only for transcripts and the editor's word timing) | `models/asr` | |
|
|
| OmniVoice's `config.json` and tokenizer files go in this pack's `omnivoice/` folder; **Load OGG TTS** fetches them there on first use if they are missing (see `omnivoice/README.md`). |
| 3. Open **Templates → ComfyUI-OGG**: `ogg_text_to_speech`, `ogg_voice_clone`, `ogg_dialogue` and `ogg_voice_design`. |
|
|
| ## Nodes |
|
|
| | Node | | |
| |---|---| |
| | Load OGG TTS | any file of `models/diffusion_models` (names with "ogg" first); `dtype` float32 (as the authors run it) or bfloat16 (half the memory). | |
| | Load OGG Codec | `models/vae`, names with "codec" first. | |
| | OGG Encode Voice / Save / Load | a clip and its exact transcript → a voice; saved voices (output/ogg_voices) are a few KB and load faster. ComfyUI-Drama3 voice files load too. | |
| | OGG TTS Generate | text, language, optional voice → codes. `split_lines`: one take per line, each hearing the two lines before and after it. Inline tags: see the node's description. Advanced: `speaker` (a trained speaker, or `random`), `quality` (a trained recording), acoustic-decoder sampling. | |
| | OGG TTS Dialogue | a script `Name: [tag] text`, up to 6 named speakers, each with a voice (`name_N` + `voice_N`), **optional**: a speaker without one speaks in a trained identity of the model (`speaker_N`, or one fixed by the name). Modes: *line by line* (every line from its speaker's voice, hearing the neighbouring lines as context) and *one pass* (the whole scene and all voices at once: the model's dialogue mode, one take). | |
| | OGG Voice from Take | a generated take (one of its lines) → a voice: fixes a voice found by chance, to use for any text. | |
| | Save / Load OGG Codes | keep a take (output/ogg_codes) to decode or edit again without generating. | |
| | OGG Decode | codes → 44.1 kHz audio; takes joined with 0.3 s gaps. `shorten_long_pauses` (off by default) trims as Fish's Drama 3 Space does. | |
| | OGG Editor | decode and edit on a timeline: captions follow the voice, cut, add silence, name voices and set volume. | |
| | Load Qwen3-ASR / Transcribe (OGG) | speech to text in 30 languages; word timing for the editor with the ForcedAligner file. | |
| |
| Defaults follow the reference script: `steps` 6, `cfg` 1.5 (1–1.5 natural tempo, 2 faster), `temperature` 0.8. |
| |
| ## What was checked, and what was not |
| |
| Checked on CPU with ComfyUI replaced by stand-ins (`tests/`: `python test_pipeline.py`, `test_codec.py`, `test_dialogue.py`): all modules import; the codec's |
| real architecture (random weights) encodes 1 s to `[10, 22]` codes and decodes to the right length; the ported OGG model loads a state dict, and generation runs end to end |
| (no voice, voice, context lines, `cfg` 0); voice and codes save/load round-trip; Decode and the Editor's analysis and plan run on takes; script parsing, both dialogue modes (including the turn markers the model is given) and the missing-voice errors; all three |
| example workflows match the nodes' inputs one to one. |
|
|
| **Not checked: the real weights, a GPU, or a live ComfyUI** (so not the editor's web UI, nor audio quality). Most likely first-contact problems: |
| (a) the `transformers` version (KV-cache calls are the reference code's), (b) how ComfyUI's `ModelPatcher` moves a plain module, (c) the checkpoint's exact key set, |
| (d) the OmniVoice config/tokenizer files' layout. |
|
|
| The example voice-clone workflow has no clip: pick your own in **Load Audio**. |
|
|
| ## Shaping a voice |
|
|
| OGG takes **no description of a voice in words** (no "deep, tired, middle-aged"); it has no input for it. What does set the voice and its manner: |
|
|
| | Lever | What it is | |
| |---|---| |
| | a **reference clip** | the strongest: voice, manner and mood of the clip carry over | |
| | `speaker` | no clip: `spk_0` ... `spk_3156` (3157 anonymous trained speakers, no gender or age listed), `ru_x` ... (the unknown speaker of a language), or `random` (picked by the seed; the console prints which) | |
| | `quality` | `q0` ... `q598`: a book or a game the model heard, with its room and, to a degree, its manner of delivery | |
| | inline tags, `cfg`, `temperature` | delivery inside a line; cfg 2 is faster and more pointed | |
|
|
| To get a voice you like: **audition**, then **fix it**. Use the `ogg_voice_design` workflow: `speaker` is `random`, every run is another speaker, each run saves its take and its voice |
| (a clip of that take with its words) under matching numbers. Keep the good ones, and use them through Load OGG Voice in Generate or Dialogue; the speaker then stays the same for any text. |
|
|
| If the tags seem to have no effect: the checkpoint (120k of 400k steps) learned tags from labels of what was *clearly audible*, spans with tags were a minority, and in a quarter of the samples they were stripped, |
| so they are hints. A neutral reference clip pulls the delivery toward neutral; try the same line **without a voice** (`speaker` = `random`), in English, with `[whispering]` or `[shouting]` |
| (the easiest to hear), and a higher `cfg` (2-3). To see whether a tag changes anything at all, run the same seed with and without it: if the audio is identical, tell the pack's author. |
|
|
| Tags and take length: the model was trained on spans of **1-5 consecutive sentences, up to 25 s**, with a tag placed before the words it applies to (and again where it changes; at most 4 per sentence, at most 2 side by side), and in a quarter of the samples the tags were removed, so they are hints, not commands. `split_lines` on gives every line its own short take, which is what it learned; with it off a long text is one take far longer than any training span, and tags deep inside it are easily lost. Keep a take to a few sentences, a tag right before its words. Reference clips have no limit in this pack; the model's training prompts were up to about 8 s, so a much longer clip is outside what it saw. |
|
|
| Dialogue: *line by line* follows the authors' documented way (their `--before`/`--after`) and is the one to use. *one pass* is **experimental**: it follows their training code (`train_omnifish.dialogue_spans`: voices named `S1`, `S2`..., all text first, then the clips one after another, instruct `dialogue`), but their README does not mention it, and in that code dialogue scenes are trained only when a data-check file exists (`speaker_mismatch.parquet`), so the released weights may never have learned it. The advanced `dialogue_layout` switches to the context layout of the authors' inference script. Neither is tested on the real weights, and the editor has no per-line timing for one pass. |
|
|
| Not in v1: streaming, continuation. |
|
|
| ## License |
|
|
| GPL-3.0 (it includes code from ComfyUI-Drama3), with parts under Apache-2.0 and the Fish Audio Community License; see NOTICE. |
| Do not clone the voice of a real person without their consent. |
|
|
