Lite voice (Kokoro-82M): requirements, measured comparison, how to choose, install switch, credits; five Kokoro previews
Browse filesCard: optional lite voice section and requirements; Credits table adds Kokoro-82M, kokoro, mlx-audio, misaki, spaCy, espeak-ng and StyleTTS 2. previews/kokoro/: synthetic Kokoro-82M stock voices (bf_emma, bf_isabella, bm_george, af_heart, am_michael).
- .gitattributes +5 -0
- README.md +38 -3
- previews/kokoro/af_heart.wav +3 -0
- previews/kokoro/am_michael.wav +3 -0
- previews/kokoro/bf_emma.wav +3 -0
- previews/kokoro/bf_isabella.wav +3 -0
- previews/kokoro/bm_george.wav +3 -0
.gitattributes
CHANGED
|
@@ -53,3 +53,8 @@ media/ui-desktop-light.png filter=lfs diff=lfs merge=lfs -text
|
|
| 53 |
media/ui-desktop.png filter=lfs diff=lfs merge=lfs -text
|
| 54 |
media/ui-hermes-card.png filter=lfs diff=lfs merge=lfs -text
|
| 55 |
media/ui-phone-camera.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
media/ui-desktop.png filter=lfs diff=lfs merge=lfs -text
|
| 54 |
media/ui-hermes-card.png filter=lfs diff=lfs merge=lfs -text
|
| 55 |
media/ui-phone-camera.png filter=lfs diff=lfs merge=lfs -text
|
| 56 |
+
previews/kokoro/af_heart.wav filter=lfs diff=lfs merge=lfs -text
|
| 57 |
+
previews/kokoro/am_michael.wav filter=lfs diff=lfs merge=lfs -text
|
| 58 |
+
previews/kokoro/bf_emma.wav filter=lfs diff=lfs merge=lfs -text
|
| 59 |
+
previews/kokoro/bf_isabella.wav filter=lfs diff=lfs merge=lfs -text
|
| 60 |
+
previews/kokoro/bm_george.wav filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -12,13 +12,14 @@ tags:
|
|
| 12 |
- voices
|
| 13 |
- tensorfold
|
| 14 |
- fish-audio
|
|
|
|
| 15 |
---
|
| 16 |
|
| 17 |
# Keys Ghost Machine — audio/visual agent, powered by [TensorFold](https://github.com/ashhart/TensorFold)
|
| 18 |
|
| 19 |
<video controls autoplay muted loop playsinline width="320" poster="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s-poster.jpg" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s.mp4"></video>
|
| 20 |
|
| 21 |
-
The voices (Drama 3), the promo and screenshots for Boo.
|
| 22 |
|
| 23 |
Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the
|
| 24 |
audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent).
|
|
@@ -33,6 +34,11 @@ silicon (macOS 26, Metal) or about 21 GB on an NVIDIA DGX Spark (CUDA 13). The f
|
|
| 33 |
1.1.4 (the prebuilt builds in the GitHub release add NVFP4 pruned Drama 3, the voice-prefix cache and Cosmos3-Edge; the
|
| 34 |
DGX Spark's Drama 3 runs on plotarmordev's NVFP4 fork, because stock 1.1.4 refuses the pruned checkpoint).
|
| 35 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
## Install
|
| 37 |
|
| 38 |
The whole Boo stack, these voices included, installs with one command on a DGX Spark or an Apple silicon Mac:
|
|
@@ -42,7 +48,8 @@ curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-v
|
|
| 42 |
```
|
| 43 |
|
| 44 |
It checks memory and disk, fetches the TensorFold 1.1.4 builds, the models and these voices, and starts Boo; `kgm start`,
|
| 45 |
-
`kgm stop` and `kgm status` run it after that.
|
|
|
|
| 46 |
[One-shot install](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent#one-shot-install) section of the
|
| 47 |
GitHub README.
|
| 48 |
|
|
@@ -85,6 +92,26 @@ A task handed to Hermes: the card shows only the tool's name while it works, the
|
|
| 85 |
|
| 86 |
`previews/NAME.wav` is each voice saying "Hi, I am NAME. Tell me what you would like to make today."
|
| 87 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 88 |
## How they were made
|
| 89 |
|
| 90 |
Every voice is a take of Drama 3's own built-in voice, generated by the model; **no recorded person is in any of
|
|
@@ -99,6 +126,7 @@ codec into reference codes. Each file says so in its `source` field.
|
|
| 99 |
- `voices/spark/NAME.json`: `drama3-voice/1` (`[10, frames]` codes with `codes_sha256`), for
|
| 100 |
[plotarmordev's TensorFold](https://github.com/plotarmordev/TensorFold) CUDA build.
|
| 101 |
- `previews/NAME.wav`: a short sample of each.
|
|
|
|
| 102 |
|
| 103 |
## Use
|
| 104 |
|
|
@@ -118,7 +146,7 @@ a streaming client should ask again for the unspoken text, as Boo does.
|
|
| 118 |
|
| 119 |
The voice codes are derived from Drama 3's output, so they fall under the
|
| 120 |
[Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third
|
| 121 |
-
parties need a separate licence from Fish Audio.
|
| 122 |
|
| 123 |
Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues).
|
| 124 |
|
|
@@ -134,6 +162,13 @@ Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-
|
|
| 134 |
| Drama 3 | The voice (speech synthesis); the 14 named voices are takes of its built-in voice | Fish Audio | Fish Audio Community License | [Hugging Face](https://huggingface.co/fishaudio/drama-3) |
|
| 135 |
| drama-3-NVFP4-Pruned, NVFP4 CUDA fork and named voices | The 4-bit pruned Drama 3 checkpoint, the DGX Spark's Drama engine and `--voices` | plotarmordev | Fish Audio Community License (weights) · Apache-2.0 (engine) | [checkpoint](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned) · [fork](https://github.com/plotarmordev/TensorFold) |
|
| 136 |
| TensorFold | The engine for Bolt, Drama 3 and Cosmos3-Edge, on Metal and CUDA | Ash Hart | Apache-2.0 | [GitHub](https://github.com/ashhart/TensorFold) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 137 |
| Eyes of Kai | The continuity log: timestamped captions so Boo can answer about the past | tonyd2wild | MIT | [GitHub](https://github.com/tonyd2wild/eyes-of-kai) |
|
| 138 |
| Hermes Agent | The back agent with tools and skills, over ACP | Nous Research | see repository | [GitHub](https://github.com/NousResearch/hermes-agent) |
|
| 139 |
| qwen-audio-agent | Design inspiration: the frontstage/backstage split (a fast voice in front, a tool-using agent behind that reports back); no code from it is used | Qwen Audio team | Apache-2.0 | [GitHub](https://github.com/QwenAudio/qwen-audio-agent) |
|
|
|
|
| 12 |
- voices
|
| 13 |
- tensorfold
|
| 14 |
- fish-audio
|
| 15 |
+
- kokoro
|
| 16 |
---
|
| 17 |
|
| 18 |
# Keys Ghost Machine — audio/visual agent, powered by [TensorFold](https://github.com/ashhart/TensorFold)
|
| 19 |
|
| 20 |
<video controls autoplay muted loop playsinline width="320" poster="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s-poster.jpg" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s.mp4"></video>
|
| 21 |
|
| 22 |
+
The voices (Drama 3), the promo and screenshots for Boo, and previews of its optional lite voice (Kokoro-82M).
|
| 23 |
|
| 24 |
Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the
|
| 25 |
audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent).
|
|
|
|
| 34 |
1.1.4 (the prebuilt builds in the GitHub release add NVFP4 pruned Drama 3, the voice-prefix cache and Cosmos3-Edge; the
|
| 35 |
DGX Spark's Drama 3 runs on plotarmordev's NVFP4 fork, because stock 1.1.4 refuses the pruned checkpoint).
|
| 36 |
|
| 37 |
+
**Lite voice instead (optional):** [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) (hexgrad, Apache-2.0) in place
|
| 38 |
+
of Drama 3: about 0.7 GB on a Mac or 3.5 GB on a DGX Spark and 0.4 GB on disk, against Drama 3's 21 GB. With it the
|
| 39 |
+
whole stack needs about 52 GB on a Mac (the 96 GB minimum stays, Bolt sets it) or about 44 GB on a DGX Spark (about 48
|
| 40 |
+
GB free is enough). It gives up these 14 voices, cloning, Drama 3's expressiveness and languages beyond English.
|
| 41 |
+
|
| 42 |
## Install
|
| 43 |
|
| 44 |
The whole Boo stack, these voices included, installs with one command on a DGX Spark or an Apple silicon Mac:
|
|
|
|
| 48 |
```
|
| 49 |
|
| 50 |
It checks memory and disk, fetches the TensorFold 1.1.4 builds, the models and these voices, and starts Boo; `kgm start`,
|
| 51 |
+
`kgm stop` and `kgm status` run it after that. Add `bash -s -- --voice lite` for the lite voice (Kokoro-82M, no Drama 3
|
| 52 |
+
download) or `--voice both` for a Full/Lite switch in the page. Settings, the Docker alternative and uninstalling are in the
|
| 53 |
[One-shot install](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent#one-shot-install) section of the
|
| 54 |
GitHub README.
|
| 55 |
|
|
|
|
| 92 |
|
| 93 |
`previews/NAME.wav` is each voice saying "Hi, I am NAME. Tell me what you would like to make today."
|
| 94 |
|
| 95 |
+
## Lite voice (Kokoro-82M)
|
| 96 |
+
|
| 97 |
+
Boo's optional lite voice runs [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) behind the same API as Drama 3
|
| 98 |
+
(MLX on the Mac, PyTorch CUDA on the DGX Spark). Measured on the live stacks:
|
| 99 |
+
|
| 100 |
+
| | DGX Spark, lite | DGX Spark, full | Mac, lite | Mac, full |
|
| 101 |
+
|---|---|---|---|---|
|
| 102 |
+
| Voice-to-voice, median | **0.33 s** | 0.55–0.77 s | 0.56–0.64 s | 0.47–0.60 s |
|
| 103 |
+
| Voice engine memory in use | **≈ 3.5 GB** | ≈ 42 GB | **≈ 0.7 GB** | ≈ 10 GB |
|
| 104 |
+
| Whole stack in use | **≈ 62 GB** | ≈ 101 GB | **≈ 55 GB** | ≈ 64 GB |
|
| 105 |
+
|
| 106 |
+
Choose the full voice (Drama 3, these 14 voices) when memory allows; choose lite when memory or disk is short or for
|
| 107 |
+
the fastest replies on a DGX Spark. Lite speaks English only in Boo (American and British voices; Kokoro has no
|
| 108 |
+
German), with Kokoro's stock voices: `bf_emma` (British female, default), `bf_isabella`, `bm_george`, `af_heart`,
|
| 109 |
+
`am_michael` and more.
|
| 110 |
+
|
| 111 |
+
`previews/kokoro/NAME.wav` is a Kokoro voice saying "Hello, I am Boo, in the lite voice. I have queued three takes of
|
| 112 |
+
the fox shot, and the first one should be ready in about a minute." (synthetic, from Kokoro's stock voices): `bf_emma`,
|
| 113 |
+
`bf_isabella`, `bm_george`, `af_heart`, `am_michael`.
|
| 114 |
+
|
| 115 |
## How they were made
|
| 116 |
|
| 117 |
Every voice is a take of Drama 3's own built-in voice, generated by the model; **no recorded person is in any of
|
|
|
|
| 126 |
- `voices/spark/NAME.json`: `drama3-voice/1` (`[10, frames]` codes with `codes_sha256`), for
|
| 127 |
[plotarmordev's TensorFold](https://github.com/plotarmordev/TensorFold) CUDA build.
|
| 128 |
- `previews/NAME.wav`: a short sample of each.
|
| 129 |
+
- `previews/kokoro/NAME.wav`: samples of the lite voice (Kokoro-82M stock voices; Apache-2.0).
|
| 130 |
|
| 131 |
## Use
|
| 132 |
|
|
|
|
| 146 |
|
| 147 |
The voice codes are derived from Drama 3's output, so they fall under the
|
| 148 |
[Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third
|
| 149 |
+
parties need a separate licence from Fish Audio. The Kokoro previews are Kokoro-82M output (Apache-2.0).
|
| 150 |
|
| 151 |
Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues).
|
| 152 |
|
|
|
|
| 162 |
| Drama 3 | The voice (speech synthesis); the 14 named voices are takes of its built-in voice | Fish Audio | Fish Audio Community License | [Hugging Face](https://huggingface.co/fishaudio/drama-3) |
|
| 163 |
| drama-3-NVFP4-Pruned, NVFP4 CUDA fork and named voices | The 4-bit pruned Drama 3 checkpoint, the DGX Spark's Drama engine and `--voices` | plotarmordev | Fish Audio Community License (weights) · Apache-2.0 (engine) | [checkpoint](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned) · [fork](https://github.com/plotarmordev/TensorFold) |
|
| 164 |
| TensorFold | The engine for Bolt, Drama 3 and Cosmos3-Edge, on Metal and CUDA | Ash Hart | Apache-2.0 | [GitHub](https://github.com/ashhart/TensorFold) |
|
| 165 |
+
| Kokoro-82M | The lite voice (optional, instead of Drama 3) | hexgrad | Apache-2.0 | [Hugging Face](https://huggingface.co/hexgrad/Kokoro-82M) |
|
| 166 |
+
| kokoro | Runs Kokoro-82M on PyTorch (DGX Spark) | hexgrad | Apache-2.0 | [GitHub](https://github.com/hexgrad/kokoro) |
|
| 167 |
+
| mlx-audio | Runs Kokoro-82M on MLX (Mac); weights from `mlx-community/Kokoro-82M-bf16` | Prince Canuma (Blaizzy) | MIT | [GitHub](https://github.com/Blaizzy/mlx-audio) |
|
| 168 |
+
| misaki | Kokoro's text-to-phoneme step (G2P) | hexgrad | Apache-2.0 | [GitHub](https://github.com/hexgrad/misaki) |
|
| 169 |
+
| spaCy and en_core_web_sm | English tokenising and tagging inside misaki | Explosion | MIT | [GitHub](https://github.com/explosion/spaCy) |
|
| 170 |
+
| espeak-ng (through phonemizer-fork and espeakng-loader) | misaki's fallback pronunciation for words its dictionary lacks | espeak-ng contributors | GPL-3.0 | [GitHub](https://github.com/espeak-ng/espeak-ng) |
|
| 171 |
+
| StyleTTS 2 | The architecture Kokoro-82M is built on (with an iSTFTNet decoder) | Yinghao Aaron Li et al. | MIT | [GitHub](https://github.com/yl4579/StyleTTS2) |
|
| 172 |
| Eyes of Kai | The continuity log: timestamped captions so Boo can answer about the past | tonyd2wild | MIT | [GitHub](https://github.com/tonyd2wild/eyes-of-kai) |
|
| 173 |
| Hermes Agent | The back agent with tools and skills, over ACP | Nous Research | see repository | [GitHub](https://github.com/NousResearch/hermes-agent) |
|
| 174 |
| qwen-audio-agent | Design inspiration: the frontstage/backstage split (a fast voice in front, a tool-using agent behind that reports back); no code from it is used | Qwen Audio team | Apache-2.0 | [GitHub](https://github.com/QwenAudio/qwen-audio-agent) |
|
previews/kokoro/af_heart.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b22b79e463bedcd1e7d5fdb18084d2744328c12a08762819ab5c8289823e6621
|
| 3 |
+
size 352800
|
previews/kokoro/am_michael.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:54f40f3e00351c6d645c24c01293c46d9e2efed1db3c495ba3243df0f618edb3
|
| 3 |
+
size 378854
|
previews/kokoro/bf_emma.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e8cd6e798c9a08a6ae81731d3b1e52d57e6c7791907e30728bc93ab345b7a332
|
| 3 |
+
size 327600
|
previews/kokoro/bf_isabella.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7b047edfe06af791cae1f505597ca17bedf63b18124b2957962e5e0feb9bfea0
|
| 3 |
+
size 326950
|
previews/kokoro/bm_george.wav
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:58f5b590f3b6f6a6772b11de044f8099ecb9c73230472580184d7c6eaa59f93b
|
| 3 |
+
size 376332
|