drowzeys commited on
Commit
0091180
·
verified ·
1 Parent(s): 0354f80

Lite voice (Kokoro-82M): requirements, measured comparison, how to choose, install switch, credits; five Kokoro previews

Browse files

Card: optional lite voice section and requirements; Credits table adds Kokoro-82M, kokoro, mlx-audio, misaki, spaCy, espeak-ng and StyleTTS 2. previews/kokoro/: synthetic Kokoro-82M stock voices (bf_emma, bf_isabella, bm_george, af_heart, am_michael).

.gitattributes CHANGED
@@ -53,3 +53,8 @@ media/ui-desktop-light.png filter=lfs diff=lfs merge=lfs -text
53
  media/ui-desktop.png filter=lfs diff=lfs merge=lfs -text
54
  media/ui-hermes-card.png filter=lfs diff=lfs merge=lfs -text
55
  media/ui-phone-camera.png filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
53
  media/ui-desktop.png filter=lfs diff=lfs merge=lfs -text
54
  media/ui-hermes-card.png filter=lfs diff=lfs merge=lfs -text
55
  media/ui-phone-camera.png filter=lfs diff=lfs merge=lfs -text
56
+ previews/kokoro/af_heart.wav filter=lfs diff=lfs merge=lfs -text
57
+ previews/kokoro/am_michael.wav filter=lfs diff=lfs merge=lfs -text
58
+ previews/kokoro/bf_emma.wav filter=lfs diff=lfs merge=lfs -text
59
+ previews/kokoro/bf_isabella.wav filter=lfs diff=lfs merge=lfs -text
60
+ previews/kokoro/bm_george.wav filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -12,13 +12,14 @@ tags:
12
  - voices
13
  - tensorfold
14
  - fish-audio
 
15
  ---
16
 
17
  # Keys Ghost Machine — audio/visual agent, powered by [TensorFold](https://github.com/ashhart/TensorFold)
18
 
19
  <video controls autoplay muted loop playsinline width="320" poster="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s-poster.jpg" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s.mp4"></video>
20
 
21
- The voices (Drama 3), the promo and screenshots for Boo.
22
 
23
  Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the
24
  audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent).
@@ -33,6 +34,11 @@ silicon (macOS 26, Metal) or about 21 GB on an NVIDIA DGX Spark (CUDA 13). The f
33
  1.1.4 (the prebuilt builds in the GitHub release add NVFP4 pruned Drama 3, the voice-prefix cache and Cosmos3-Edge; the
34
  DGX Spark's Drama 3 runs on plotarmordev's NVFP4 fork, because stock 1.1.4 refuses the pruned checkpoint).
35
 
 
 
 
 
 
36
  ## Install
37
 
38
  The whole Boo stack, these voices included, installs with one command on a DGX Spark or an Apple silicon Mac:
@@ -42,7 +48,8 @@ curl -fsSL https://raw.githubusercontent.com/drowzeys/keys-ghost-machine-audio-v
42
  ```
43
 
44
  It checks memory and disk, fetches the TensorFold 1.1.4 builds, the models and these voices, and starts Boo; `kgm start`,
45
- `kgm stop` and `kgm status` run it after that. Settings, the Docker alternative and uninstalling are in the
 
46
  [One-shot install](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent#one-shot-install) section of the
47
  GitHub README.
48
 
@@ -85,6 +92,26 @@ A task handed to Hermes: the card shows only the tool's name while it works, the
85
 
86
  `previews/NAME.wav` is each voice saying "Hi, I am NAME. Tell me what you would like to make today."
87
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
88
  ## How they were made
89
 
90
  Every voice is a take of Drama 3's own built-in voice, generated by the model; **no recorded person is in any of
@@ -99,6 +126,7 @@ codec into reference codes. Each file says so in its `source` field.
99
  - `voices/spark/NAME.json`: `drama3-voice/1` (`[10, frames]` codes with `codes_sha256`), for
100
  [plotarmordev's TensorFold](https://github.com/plotarmordev/TensorFold) CUDA build.
101
  - `previews/NAME.wav`: a short sample of each.
 
102
 
103
  ## Use
104
 
@@ -118,7 +146,7 @@ a streaming client should ask again for the unspoken text, as Boo does.
118
 
119
  The voice codes are derived from Drama 3's output, so they fall under the
120
  [Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third
121
- parties need a separate licence from Fish Audio.
122
 
123
  Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues).
124
 
@@ -134,6 +162,13 @@ Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-
134
  | Drama 3 | The voice (speech synthesis); the 14 named voices are takes of its built-in voice | Fish Audio | Fish Audio Community License | [Hugging Face](https://huggingface.co/fishaudio/drama-3) |
135
  | drama-3-NVFP4-Pruned, NVFP4 CUDA fork and named voices | The 4-bit pruned Drama 3 checkpoint, the DGX Spark's Drama engine and `--voices` | plotarmordev | Fish Audio Community License (weights) · Apache-2.0 (engine) | [checkpoint](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned) · [fork](https://github.com/plotarmordev/TensorFold) |
136
  | TensorFold | The engine for Bolt, Drama 3 and Cosmos3-Edge, on Metal and CUDA | Ash Hart | Apache-2.0 | [GitHub](https://github.com/ashhart/TensorFold) |
 
 
 
 
 
 
 
137
  | Eyes of Kai | The continuity log: timestamped captions so Boo can answer about the past | tonyd2wild | MIT | [GitHub](https://github.com/tonyd2wild/eyes-of-kai) |
138
  | Hermes Agent | The back agent with tools and skills, over ACP | Nous Research | see repository | [GitHub](https://github.com/NousResearch/hermes-agent) |
139
  | qwen-audio-agent | Design inspiration: the frontstage/backstage split (a fast voice in front, a tool-using agent behind that reports back); no code from it is used | Qwen Audio team | Apache-2.0 | [GitHub](https://github.com/QwenAudio/qwen-audio-agent) |
 
12
  - voices
13
  - tensorfold
14
  - fish-audio
15
+ - kokoro
16
  ---
17
 
18
  # Keys Ghost Machine — audio/visual agent, powered by [TensorFold](https://github.com/ashhart/TensorFold)
19
 
20
  <video controls autoplay muted loop playsinline width="320" poster="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s-poster.jpg" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s.mp4"></video>
21
 
22
+ The voices (Drama 3), the promo and screenshots for Boo, and previews of its optional lite voice (Kokoro-82M).
23
 
24
  Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the
25
  audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent).
 
34
  1.1.4 (the prebuilt builds in the GitHub release add NVFP4 pruned Drama 3, the voice-prefix cache and Cosmos3-Edge; the
35
  DGX Spark's Drama 3 runs on plotarmordev's NVFP4 fork, because stock 1.1.4 refuses the pruned checkpoint).
36
 
37
+ **Lite voice instead (optional):** [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) (hexgrad, Apache-2.0) in place
38
+ of Drama 3: about 0.7 GB on a Mac or 3.5 GB on a DGX Spark and 0.4 GB on disk, against Drama 3's 21 GB. With it the
39
+ whole stack needs about 52 GB on a Mac (the 96 GB minimum stays, Bolt sets it) or about 44 GB on a DGX Spark (about 48
40
+ GB free is enough). It gives up these 14 voices, cloning, Drama 3's expressiveness and languages beyond English.
41
+
42
  ## Install
43
 
44
  The whole Boo stack, these voices included, installs with one command on a DGX Spark or an Apple silicon Mac:
 
48
  ```
49
 
50
  It checks memory and disk, fetches the TensorFold 1.1.4 builds, the models and these voices, and starts Boo; `kgm start`,
51
+ `kgm stop` and `kgm status` run it after that. Add `bash -s -- --voice lite` for the lite voice (Kokoro-82M, no Drama 3
52
+ download) or `--voice both` for a Full/Lite switch in the page. Settings, the Docker alternative and uninstalling are in the
53
  [One-shot install](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent#one-shot-install) section of the
54
  GitHub README.
55
 
 
92
 
93
  `previews/NAME.wav` is each voice saying "Hi, I am NAME. Tell me what you would like to make today."
94
 
95
+ ## Lite voice (Kokoro-82M)
96
+
97
+ Boo's optional lite voice runs [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) behind the same API as Drama 3
98
+ (MLX on the Mac, PyTorch CUDA on the DGX Spark). Measured on the live stacks:
99
+
100
+ | | DGX Spark, lite | DGX Spark, full | Mac, lite | Mac, full |
101
+ |---|---|---|---|---|
102
+ | Voice-to-voice, median | **0.33 s** | 0.55–0.77 s | 0.56–0.64 s | 0.47–0.60 s |
103
+ | Voice engine memory in use | **≈ 3.5 GB** | ≈ 42 GB | **≈ 0.7 GB** | ≈ 10 GB |
104
+ | Whole stack in use | **≈ 62 GB** | ≈ 101 GB | **≈ 55 GB** | ≈ 64 GB |
105
+
106
+ Choose the full voice (Drama 3, these 14 voices) when memory allows; choose lite when memory or disk is short or for
107
+ the fastest replies on a DGX Spark. Lite speaks English only in Boo (American and British voices; Kokoro has no
108
+ German), with Kokoro's stock voices: `bf_emma` (British female, default), `bf_isabella`, `bm_george`, `af_heart`,
109
+ `am_michael` and more.
110
+
111
+ `previews/kokoro/NAME.wav` is a Kokoro voice saying "Hello, I am Boo, in the lite voice. I have queued three takes of
112
+ the fox shot, and the first one should be ready in about a minute." (synthetic, from Kokoro's stock voices): `bf_emma`,
113
+ `bf_isabella`, `bm_george`, `af_heart`, `am_michael`.
114
+
115
  ## How they were made
116
 
117
  Every voice is a take of Drama 3's own built-in voice, generated by the model; **no recorded person is in any of
 
126
  - `voices/spark/NAME.json`: `drama3-voice/1` (`[10, frames]` codes with `codes_sha256`), for
127
  [plotarmordev's TensorFold](https://github.com/plotarmordev/TensorFold) CUDA build.
128
  - `previews/NAME.wav`: a short sample of each.
129
+ - `previews/kokoro/NAME.wav`: samples of the lite voice (Kokoro-82M stock voices; Apache-2.0).
130
 
131
  ## Use
132
 
 
146
 
147
  The voice codes are derived from Drama 3's output, so they fall under the
148
  [Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third
149
+ parties need a separate licence from Fish Audio. The Kokoro previews are Kokoro-82M output (Apache-2.0).
150
 
151
  Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues).
152
 
 
162
  | Drama 3 | The voice (speech synthesis); the 14 named voices are takes of its built-in voice | Fish Audio | Fish Audio Community License | [Hugging Face](https://huggingface.co/fishaudio/drama-3) |
163
  | drama-3-NVFP4-Pruned, NVFP4 CUDA fork and named voices | The 4-bit pruned Drama 3 checkpoint, the DGX Spark's Drama engine and `--voices` | plotarmordev | Fish Audio Community License (weights) · Apache-2.0 (engine) | [checkpoint](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned) · [fork](https://github.com/plotarmordev/TensorFold) |
164
  | TensorFold | The engine for Bolt, Drama 3 and Cosmos3-Edge, on Metal and CUDA | Ash Hart | Apache-2.0 | [GitHub](https://github.com/ashhart/TensorFold) |
165
+ | Kokoro-82M | The lite voice (optional, instead of Drama 3) | hexgrad | Apache-2.0 | [Hugging Face](https://huggingface.co/hexgrad/Kokoro-82M) |
166
+ | kokoro | Runs Kokoro-82M on PyTorch (DGX Spark) | hexgrad | Apache-2.0 | [GitHub](https://github.com/hexgrad/kokoro) |
167
+ | mlx-audio | Runs Kokoro-82M on MLX (Mac); weights from `mlx-community/Kokoro-82M-bf16` | Prince Canuma (Blaizzy) | MIT | [GitHub](https://github.com/Blaizzy/mlx-audio) |
168
+ | misaki | Kokoro's text-to-phoneme step (G2P) | hexgrad | Apache-2.0 | [GitHub](https://github.com/hexgrad/misaki) |
169
+ | spaCy and en_core_web_sm | English tokenising and tagging inside misaki | Explosion | MIT | [GitHub](https://github.com/explosion/spaCy) |
170
+ | espeak-ng (through phonemizer-fork and espeakng-loader) | misaki's fallback pronunciation for words its dictionary lacks | espeak-ng contributors | GPL-3.0 | [GitHub](https://github.com/espeak-ng/espeak-ng) |
171
+ | StyleTTS 2 | The architecture Kokoro-82M is built on (with an iSTFTNet decoder) | Yinghao Aaron Li et al. | MIT | [GitHub](https://github.com/yl4579/StyleTTS2) |
172
  | Eyes of Kai | The continuity log: timestamped captions so Boo can answer about the past | tonyd2wild | MIT | [GitHub](https://github.com/tonyd2wild/eyes-of-kai) |
173
  | Hermes Agent | The back agent with tools and skills, over ACP | Nous Research | see repository | [GitHub](https://github.com/NousResearch/hermes-agent) |
174
  | qwen-audio-agent | Design inspiration: the frontstage/backstage split (a fast voice in front, a tool-using agent behind that reports back); no code from it is used | Qwen Audio team | Apache-2.0 | [GitHub](https://github.com/QwenAudio/qwen-audio-agent) |
previews/kokoro/af_heart.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b22b79e463bedcd1e7d5fdb18084d2744328c12a08762819ab5c8289823e6621
3
+ size 352800
previews/kokoro/am_michael.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54f40f3e00351c6d645c24c01293c46d9e2efed1db3c495ba3243df0f618edb3
3
+ size 378854
previews/kokoro/bf_emma.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8cd6e798c9a08a6ae81731d3b1e52d57e6c7791907e30728bc93ab345b7a332
3
+ size 327600
previews/kokoro/bf_isabella.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7b047edfe06af791cae1f505597ca17bedf63b18124b2957962e5e0feb9bfea0
3
+ size 326950
previews/kokoro/bm_george.wav ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:58f5b590f3b6f6a6772b11de044f8099ecb9c73230472580184d7c6eaa59f93b
3
+ size 376332