Card: powered-by-TensorFold title, promo video, screenshots, TensorFold 1.1.4 notes, Credits table
Browse files- .gitattributes +6 -0
- README.md +54 -4
- media/promo-45s.mp4 +3 -0
- media/promo-landscape.mp4 +3 -0
- media/promo-poster.jpg +0 -0
- media/ui-desktop-light.png +3 -0
- media/ui-desktop.png +3 -0
- media/ui-hermes-card.png +3 -0
- media/ui-phone-camera.png +3 -0
.gitattributes
CHANGED
|
@@ -47,3 +47,9 @@ previews/seren.wav filter=lfs diff=lfs merge=lfs -text
|
|
| 47 |
previews/skye.wav filter=lfs diff=lfs merge=lfs -text
|
| 48 |
previews/vesper.wav filter=lfs diff=lfs merge=lfs -text
|
| 49 |
previews/wren.wav filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
previews/skye.wav filter=lfs diff=lfs merge=lfs -text
|
| 48 |
previews/vesper.wav filter=lfs diff=lfs merge=lfs -text
|
| 49 |
previews/wren.wav filter=lfs diff=lfs merge=lfs -text
|
| 50 |
+
media/promo-45s.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 51 |
+
media/promo-landscape.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
media/ui-desktop-light.png filter=lfs diff=lfs merge=lfs -text
|
| 53 |
+
media/ui-desktop.png filter=lfs diff=lfs merge=lfs -text
|
| 54 |
+
media/ui-hermes-card.png filter=lfs diff=lfs merge=lfs -text
|
| 55 |
+
media/ui-phone-camera.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -14,7 +14,9 @@ tags:
|
|
| 14 |
- fish-audio
|
| 15 |
---
|
| 16 |
|
| 17 |
-
#
|
|
|
|
|
|
|
| 18 |
|
| 19 |
Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the
|
| 20 |
audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent).
|
|
@@ -25,7 +27,29 @@ Vesper is Boo's default.
|
|
| 25 |
Drama 3 served by TensorFold with named voices (`--voices`): about 21 GB of weights (the
|
| 26 |
[NVFP4 pruned build](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned)); about 28 GB wired on a Mac with Apple
|
| 27 |
silicon (macOS 26, Metal) or about 21 GB on an NVIDIA DGX Spark (CUDA 13). The full Boo stack needs about 79 GB on a Mac
|
| 28 |
-
(96 GB minimum, 128 GB or more recommended) or about 61 GB on a DGX Spark; see the GitHub README.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
## The voices
|
| 31 |
|
|
@@ -75,7 +99,33 @@ curl -s http://127.0.0.1:8098/v1/audio/speech -H 'Content-Type: application/json
|
|
| 75 |
|
| 76 |
The voice codes are derived from Drama 3's output, so they fall under the
|
| 77 |
[Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third
|
| 78 |
-
parties need a separate licence from Fish Audio.
|
| 79 |
-
CREDITS.md.
|
| 80 |
|
| 81 |
Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
- fish-audio
|
| 15 |
---
|
| 16 |
|
| 17 |
+
# Keys Ghost Machine — audio/visual agent, powered by [TensorFold](https://github.com/ashhart/TensorFold)
|
| 18 |
+
|
| 19 |
+
The voices (Drama 3), the promo and screenshots for Boo.
|
| 20 |
|
| 21 |
Fourteen named voices for [Fish Audio Drama 3](https://huggingface.co/fishaudio/drama-3), made for **Boo**, the
|
| 22 |
audio/visual agent in [drowzeys/keys-ghost-machine-audio-visual-agent](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent).
|
|
|
|
| 27 |
Drama 3 served by TensorFold with named voices (`--voices`): about 21 GB of weights (the
|
| 28 |
[NVFP4 pruned build](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned)); about 28 GB wired on a Mac with Apple
|
| 29 |
silicon (macOS 26, Metal) or about 21 GB on an NVIDIA DGX Spark (CUDA 13). The full Boo stack needs about 79 GB on a Mac
|
| 30 |
+
(96 GB minimum, 128 GB or more recommended) or about 61 GB on a DGX Spark; see the GitHub README. Software: TensorFold
|
| 31 |
+
1.1.4 (the prebuilt builds in the GitHub release add NVFP4 pruned Drama 3, the voice-prefix cache and Cosmos3-Edge; the
|
| 32 |
+
DGX Spark's Drama 3 runs on plotarmordev's NVFP4 fork, because stock 1.1.4 refuses the pruned checkpoint).
|
| 33 |
+
|
| 34 |
+
## See it work
|
| 35 |
+
|
| 36 |
+
<video controls width="100%" poster="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-poster.jpg" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-landscape.mp4"></video>
|
| 37 |
+
|
| 38 |
+
Phone cut (46 s): <video controls width="320" src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/promo-45s.mp4"></video>
|
| 39 |
+
|
| 40 |
+
Real recordings of the page, hands-free. The voices are synthetic Drama 3 voices and the desk is a computer-made scene;
|
| 41 |
+
no person appears. On TensorFold 1.1.4, voice-to-voice is 0.47–0.60 s on a Mac (M5 Ultra) and 0.55–0.77 s on a DGX
|
| 42 |
+
Spark, with Drama 3's first audio in Vesper at 122 ms and 121 ms.
|
| 43 |
+
|
| 44 |
+
## Screenshots
|
| 45 |
+
|
| 46 |
+
| Phone (camera on) | Desktop |
|
| 47 |
+
|---|---|
|
| 48 |
+
| <img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-phone-camera.png" width="300" alt="Boo on a phone with the camera on, describing a desk with a WOW mug"> | <img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-desktop.png" width="560" alt="Boo on a desktop with its settings and a live-fact answer"><br><br><img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-desktop-light.png" width="560" alt="The same page in the light theme"> |
|
| 49 |
+
|
| 50 |
+
A task handed to Hermes: the card shows only the tool's name while it works, then the answer.
|
| 51 |
+
|
| 52 |
+
<img src="https://huggingface.co/drowzeys/keys-ghost-machine-audio-visual-agent/resolve/main/media/ui-hermes-card.png" width="640" alt="A Hermes task card showing terminal while working, then the answer">
|
| 53 |
|
| 54 |
## The voices
|
| 55 |
|
|
|
|
| 99 |
|
| 100 |
The voice codes are derived from Drama 3's output, so they fall under the
|
| 101 |
[Fish Audio Community License](https://huggingface.co/fishaudio/drama-3). Hosted services that generate speech for third
|
| 102 |
+
parties need a separate licence from Fish Audio.
|
|
|
|
| 103 |
|
| 104 |
Questions: [GitHub Issues](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/issues).
|
| 105 |
+
|
| 106 |
+
## Credits
|
| 107 |
+
|
| 108 |
+
| Component | What it does in Boo | Author / org | Licence | Link |
|
| 109 |
+
|---|---|---|---|---|
|
| 110 |
+
| Nemotron 3.5 Bolt | The front brain: answers most turns, with mood tags for the face | NVIDIA | NVIDIA Open Model License | [Hugging Face](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB) |
|
| 111 |
+
| Cosmos3-Edge | Camera questions and memory captions (its reasoner tower) | NVIDIA | OpenMDW 1.1 | [Hugging Face](https://huggingface.co/nvidia/Cosmos3-Edge) |
|
| 112 |
+
| Parakeet TDT 0.6B v3 | Speech to text | NVIDIA | CC-BY-4.0 | [Hugging Face](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) |
|
| 113 |
+
| parakeet-mlx | Runs Parakeet on the Mac | senstella | Apache-2.0 | [GitHub](https://github.com/senstella/parakeet-mlx) |
|
| 114 |
+
| NeMo | Runs Parakeet on the DGX Spark | NVIDIA | Apache-2.0 | [GitHub](https://github.com/NVIDIA/NeMo) |
|
| 115 |
+
| Drama 3 | The voice (speech synthesis); the 14 named voices are takes of its built-in voice | Fish Audio | Fish Audio Community License | [Hugging Face](https://huggingface.co/fishaudio/drama-3) |
|
| 116 |
+
| drama-3-NVFP4-Pruned, NVFP4 CUDA fork and named voices | The 4-bit pruned Drama 3 checkpoint, the DGX Spark's Drama engine and `--voices` | plotarmordev | Fish Audio Community License (weights) · Apache-2.0 (engine) | [checkpoint](https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned) · [fork](https://github.com/plotarmordev/TensorFold) |
|
| 117 |
+
| TensorFold | The engine for Bolt, Drama 3 and Cosmos3-Edge, on Metal and CUDA | Ash Hart | Apache-2.0 | [GitHub](https://github.com/ashhart/TensorFold) |
|
| 118 |
+
| Eyes of Kai | The continuity log: timestamped captions so Boo can answer about the past | tonyd2wild | MIT | [GitHub](https://github.com/tonyd2wild/eyes-of-kai) |
|
| 119 |
+
| Hermes Agent | The back agent with tools and skills, over ACP | Nous Research | see repository | [GitHub](https://github.com/NousResearch/hermes-agent) |
|
| 120 |
+
| qwen-audio-agent | Design inspiration: the frontstage/backstage split (a fast voice in front, a tool-using agent behind that reports back); no code from it is used | Qwen Audio team | Apache-2.0 | [GitHub](https://github.com/QwenAudio/qwen-audio-agent) |
|
| 121 |
+
| RED-SNOW 5.3 Flash | The back agent's model in the reference setup (a GLM-5.3-Flash build); any OpenAI-compatible model works | RED-SNOW | see model card | — |
|
| 122 |
+
| Silero VAD v5 | Hears when someone is speaking (`silero_vad.onnx` ships here) | Silero | MIT | [GitHub](https://github.com/snakers4/silero-vad) |
|
| 123 |
+
| MediaPipe Face Detector and Hand Landmarker | Gaze tracking, exact finger counts and which hand, in the page | Google | Apache-2.0 | [GitHub](https://github.com/google-ai-edge/mediapipe) |
|
| 124 |
+
| Three.js | Boo, the 3D ghost | three.js authors (mrdoob) | MIT | [GitHub](https://github.com/mrdoob/three.js) |
|
| 125 |
+
| ddgs | Web search for quick live facts | deedy5 | MIT | [GitHub](https://github.com/deedy5/ddgs) |
|
| 126 |
+
| wttr.in | Weather for quick live facts (data from WorldWeatherOnline) | Igor Chubin | Apache-2.0 | [GitHub](https://github.com/chubin/wttr.in) |
|
| 127 |
+
| NVIDIA PyTorch container | The Docker image's base (`nvcr.io/nvidia/pytorch:26.07-py3`) | NVIDIA | NVIDIA Deep Learning Container License | [NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch) |
|
| 128 |
+
| Qwen3.5-0.8B (earlier versions) | The first memory captioner | Qwen team, Alibaba Cloud | Apache-2.0 | [Hugging Face](https://huggingface.co/Qwen) |
|
| 129 |
+
| AEON-7 Nemotron 3 Nano Omni build (earlier versions) | The first camera model, from NVIDIA's Nemotron 3 Nano Omni | AEON-7 | NVIDIA Open Model License | [Hugging Face](https://huggingface.co/AEON-7) |
|
| 130 |
+
|
| 131 |
+
Full details: [CREDITS.md](https://github.com/drowzeys/keys-ghost-machine-audio-visual-agent/blob/main/CREDITS.md)
|
media/promo-45s.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3f1b0b65b3e1e34fe6e9c89cf6201f717d70fa9088dd61a77a6e924a5a89813b
|
| 3 |
+
size 4999521
|
media/promo-landscape.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c0fca1d697828ff053dfd7b695fb41613884cf8ff18ece9225efae6d62e3d9b8
|
| 3 |
+
size 6970480
|
media/promo-poster.jpg
ADDED
|
media/ui-desktop-light.png
ADDED
|
Git LFS Details
|
media/ui-desktop.png
ADDED
|
Git LFS Details
|
media/ui-hermes-card.png
ADDED
|
Git LFS Details
|
media/ui-phone-camera.png
ADDED
|
Git LFS Details
|