Spaces:
Paused
Paused
update README link
Browse files
README.md
CHANGED
|
@@ -1,5 +1,6 @@
|
|
| 1 |
---
|
| 2 |
-
|
|
|
|
| 3 |
emoji: π
|
| 4 |
colorFrom: yellow
|
| 5 |
colorTo: green
|
|
@@ -7,7 +8,6 @@ sdk: gradio
|
|
| 7 |
sdk_version: 5.35.0
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
-
---
|
| 11 |
|
| 12 |
# MomsVoice
|
| 13 |
|
|
@@ -15,15 +15,20 @@ pinned: false
|
|
| 15 |
|
| 16 |
Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gradio app on Hugging Face Spaces β no external APIs, no data leaves the server.
|
| 17 |
|
| 18 |
-
|
|
|
|
|
|
|
| 19 |
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
## Team members
|
|
|
|
| 22 |
- Minha Hwang [Linkedin](https://www.linkedin.com/in/minha-hwang-7440771/) / [Huggingface](https://huggingface.co/DrSquare)
|
| 23 |
- Jimin Gu [Linkedin](https://www.linkedin.com/in/99jimingu/) / [Huggingface](https://huggingface.co/Zoe09)
|
| 24 |
- Seunghwa Song [Linkedin](https://www.linkedin.com/in/sshtel/) / [Huggingface](https://huggingface.co/sshtel)
|
| 25 |
|
| 26 |
-
|
| 27 |
## Demo Flow
|
| 28 |
|
| 29 |
```
|
|
@@ -38,21 +43,25 @@ Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gra
|
|
| 38 |
|
| 39 |
## How It Works
|
| 40 |
|
| 41 |
-
|
| 42 |
-
|
|
| 43 |
-
|
|
| 44 |
-
| **
|
| 45 |
-
| **
|
| 46 |
-
| **
|
| 47 |
-
| **
|
|
|
|
|
|
|
| 48 |
|
| 49 |
## Models
|
| 50 |
|
| 51 |
-
|
| 52 |
-
|
|
| 53 |
-
|
|
| 54 |
-
| [Qwen3-TTS-
|
| 55 |
-
| [
|
|
|
|
|
|
|
| 56 |
|
| 57 |
The stock narration model and LFM Q&A model are preloaded at startup for fast response times. The 1.7B cloning model is loaded when cloning or cloned narration is needed.
|
| 58 |
|
|
@@ -67,27 +76,32 @@ The stock narration model and LFM Q&A model are preloaded at startup for fast re
|
|
| 67 |
|
| 68 |
## System Requirements
|
| 69 |
|
| 70 |
-
|
| 71 |
-
|
|
| 72 |
-
|
|
| 73 |
-
| Nvidia
|
| 74 |
-
|
|
|
|
|
|
|
|
| 75 |
|
| 76 |
**Hugging Face Spaces:**
|
|
|
|
| 77 |
- **A10G Small** β confirmed working, recommended for the hackathon demo
|
| 78 |
- **T4 Small** β runs with limitations; recommend disabling concurrent model loading
|
| 79 |
|
| 80 |
-
|
| 81 |
|
| 82 |
**Memory breakdown (approximate):**
|
| 83 |
|
| 84 |
-
|
| 85 |
-
|
|
| 86 |
-
|
|
| 87 |
-
|
|
| 88 |
-
| Qwen3-TTS-
|
| 89 |
-
|
|
| 90 |
-
|
|
|
|
|
|
|
|
| 91 |
|
| 92 |
The 1.7B cloning model is loaded on demand (not at startup), so idle VRAM usage is ~5β6 GB with only the narration and Q&A models preloaded.
|
| 93 |
|
|
@@ -138,18 +152,20 @@ python app.py
|
|
| 138 |
|
| 139 |
## Stories Included
|
| 140 |
|
| 141 |
-
|
| 142 |
-
|
|
| 143 |
-
|
|
| 144 |
-
| The Tale of
|
| 145 |
-
| The Tale of
|
| 146 |
-
| The Tale of
|
| 147 |
-
| The
|
| 148 |
-
| The
|
| 149 |
-
| The
|
| 150 |
-
| The Little
|
| 151 |
-
| The
|
| 152 |
-
| The
|
|
|
|
|
|
|
| 153 |
|
| 154 |
All stories are public domain from [Project Gutenberg](https://www.gutenberg.org/).
|
| 155 |
|
|
@@ -172,7 +188,6 @@ All inference runs on the Hugging Face Space GPU. Voice samples, story text, and
|
|
| 172 |
Hackathon project. Stories are public domain.
|
| 173 |
test push verification
|
| 174 |
|
| 175 |
-
|
| 176 |
## Application Demo
|
| 177 |
|
| 178 |
# MomsVoice β Demo Walkthrough
|
|
@@ -183,7 +198,7 @@ Step-by-step guide through the app with screenshots.
|
|
| 183 |
|
| 184 |
## Step 1: Explore the Story Shelf
|
| 185 |
|
| 186 |
-
|
| 187 |
|
| 188 |
Open the **Explore Space** tab to browse available stories displayed as warm bookshelf cards. Each card shows the cover art, title, author, and a short excerpt. Click **"Tap to Play"** on any card to open it in My Library, or click **"More Info"** for the full story description.
|
| 189 |
|
|
@@ -193,7 +208,7 @@ The featured story highlights the current pick. All 10 stories are public domain
|
|
| 193 |
|
| 194 |
## Step 2: Open My Library and Start Narration
|
| 195 |
|
| 196 |
-
|
| 197 |
|
| 198 |
Switch to the **My Library** tab. Click any story card to load it into the **Now Playing** panel on the right. The panel shows:
|
| 199 |
|
|
@@ -208,7 +223,7 @@ Narration starts immediately, playing one paragraph at a time using the TTS mode
|
|
| 208 |
|
| 209 |
## Step 3: Ask a Question About the Story
|
| 210 |
|
| 211 |
-
|
| 212 |
|
| 213 |
Scroll down in the Library panel to find the **Ask About the Story** section. Click **β Ask a Question** β narration pauses automatically and the microphone starts recording.
|
| 214 |
|
|
@@ -218,11 +233,12 @@ Speak your question naturally (e.g. *"What did Puss in Boots do to help his mast
|
|
| 218 |
|
| 219 |
## Step 4: Hear the Spoken Answer (LFM2.5-Audio)
|
| 220 |
|
| 221 |
-
|
| 222 |
|
| 223 |
The recorded question waveform appears in the audio widget. Below it, the **LFM2.5-Audio** model generates a grounded answer in 1β2 sentences, based only on the current story context.
|
| 224 |
|
| 225 |
The answer is delivered in two ways simultaneously:
|
|
|
|
| 226 |
- **Answer text** shown inline (e.g. *"Puss in Boots, a poor man, bargained for boots and a bag, and Grimalkin, ailient rabbit, rescued him from a king's wrath."*)
|
| 227 |
- **Answer Audio** waveform auto-plays the spoken response
|
| 228 |
|
|
@@ -232,9 +248,10 @@ After the answer finishes, narration resumes from where it paused.
|
|
| 232 |
|
| 233 |
## Step 5: Open Clone Voice Studio and Record Your Voice
|
| 234 |
|
| 235 |
-
|
| 236 |
|
| 237 |
Go to the **Clone Voice Studio** tab. Fill in:
|
|
|
|
| 238 |
- **Voice Nickname** β a name for this profile (e.g. *"Mom"*, *"Grandpa Joe"*)
|
| 239 |
- **Voice Gender Avatar** β Female or Male
|
| 240 |
|
|
@@ -244,7 +261,7 @@ A story script is displayed to read aloud. Click the **Record** button and read
|
|
| 244 |
|
| 245 |
## Step 6: Analyze & Synthesize the Vocal Profile
|
| 246 |
|
| 247 |
-
|
| 248 |
|
| 249 |
Click **Analyze & Synthesize Customized Vocal Profile**. The **Status Pipeline Visualizer** on the right tracks progress through extraction, de-noising, and pitch tuning stages.
|
| 250 |
|
|
@@ -254,9 +271,10 @@ A progress bar shows synthesis percentage (e.g. *"Generating voice preview... 70
|
|
| 254 |
|
| 255 |
## Step 7: Voice Profile Ready
|
| 256 |
|
| 257 |
-
|
| 258 |
|
| 259 |
When synthesis completes, the Status Pipeline Visualizer shows:
|
|
|
|
| 260 |
- Your **avatar** and nickname with a **"Cloned successfully"** badge
|
| 261 |
- *"Synthesized today using Qwen3-TTS voice cloning"*
|
| 262 |
- A **preview audio player** β tap Play to hear a short sample in your cloned voice
|
|
@@ -268,8 +286,8 @@ The profile is saved to disk and will persist across Space restarts.
|
|
| 268 |
|
| 269 |
## Step 8: Story Narration in Your Cloned Voice
|
| 270 |
|
| 271 |
-
|
| 272 |
|
| 273 |
Return to **My Library** and select a story. The Now Playing panel shows **Active Narrator: [your nickname]** β confirming the cloned voice is active.
|
| 274 |
|
| 275 |
-
Narration plays in your recorded voice, paragraph by paragraph (*"Paragraph 1 / 32"*). The story text scrolls in the reader below the player. All TTS inference runs locally on the Space GPU β no audio leaves the server.
|
|
|
|
| 1 |
---
|
| 2 |
+
|
| 3 |
+
## title: MomsVoice
|
| 4 |
emoji: π
|
| 5 |
colorFrom: yellow
|
| 6 |
colorTo: green
|
|
|
|
| 8 |
sdk_version: 5.35.0
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
|
|
|
| 11 |
|
| 12 |
# MomsVoice
|
| 13 |
|
|
|
|
| 15 |
|
| 16 |
Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gradio app on Hugging Face Spaces β no external APIs, no data leaves the server.
|
| 17 |
|
| 18 |
+
## build-small-hackathon Link
|
| 19 |
+
|
| 20 |
+
π [https://huggingface.co/spaces/build-small-hackathon/MomsVoiceAI](https://huggingface.co/spaces/build-small-hackathon/MomsVoiceAI)
|
| 21 |
|
| 22 |
+
π **Live Demo at personal Spaces**
|
| 23 |
+
[https://huggingface.co/spaces/DrSquare/MomsVoice](https://huggingface.co/spaces/DrSquare/MomsVoice)
|
| 24 |
+
[https://huggingface.co/spaces/sshtel/Huggingface_Hack](https://huggingface.co/spaces/sshtel/Huggingface_Hack)
|
| 25 |
|
| 26 |
## Team members
|
| 27 |
+
|
| 28 |
- Minha Hwang [Linkedin](https://www.linkedin.com/in/minha-hwang-7440771/) / [Huggingface](https://huggingface.co/DrSquare)
|
| 29 |
- Jimin Gu [Linkedin](https://www.linkedin.com/in/99jimingu/) / [Huggingface](https://huggingface.co/Zoe09)
|
| 30 |
- Seunghwa Song [Linkedin](https://www.linkedin.com/in/sshtel/) / [Huggingface](https://huggingface.co/sshtel)
|
| 31 |
|
|
|
|
| 32 |
## Demo Flow
|
| 33 |
|
| 34 |
```
|
|
|
|
| 43 |
|
| 44 |
## How It Works
|
| 45 |
|
| 46 |
+
|
| 47 |
+
| Step | What Happens | Model |
|
| 48 |
+
| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- |
|
| 49 |
+
| **Clone** | Parent records or uploads 15β30s of audio. Speaker embedding is extracted and cached. | Qwen3-TTS-1.7B Base |
|
| 50 |
+
| **Listen** | Story plays in the cloned voice as interruptible sentence chunks. Falls back to CustomVoice 0.6B stock voice (speaker "vivian") if no clone. | Qwen3-TTS-1.7B / CustomVoice 0.6B |
|
| 51 |
+
| **Ask** | Child taps Ask β narration pauses, microphone auto-starts. When recording stops, the question is sent directly to the Q&A audio model. | LFM2.5-Audio-1.5B |
|
| 52 |
+
| **Answer** | A short grounded answer is generated from the story context and spoken by the low-latency Q&A audio model. It does not use the cloned narrator voice. | LFM2.5-Audio-1.5B |
|
| 53 |
+
| **Resume** | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
|
| 54 |
+
|
| 55 |
|
| 56 |
## Models
|
| 57 |
|
| 58 |
+
|
| 59 |
+
| Model | Role | Size |
|
| 60 |
+
| ----------------------------------------------------------------------------------------- | ------------------------------------------------------------- | ----------- |
|
| 61 |
+
| [Qwen3-TTS-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) | Zero-shot voice cloning + TTS | 1.7B params |
|
| 62 |
+
| [Qwen3-TTS-0.6B-CustomVoice](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice) | Fast stock-voice TTS (predefined speakers) | 0.6B params |
|
| 63 |
+
| [LiquidAI/LFM2.5-Audio-1.5B](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) | End-to-end story Q&A with audio input and spoken audio output | 1.5B params |
|
| 64 |
+
|
| 65 |
|
| 66 |
The stock narration model and LFM Q&A model are preloaded at startup for fast response times. The 1.7B cloning model is loaded when cloning or cloned narration is needed.
|
| 67 |
|
|
|
|
| 76 |
|
| 77 |
## System Requirements
|
| 78 |
|
| 79 |
+
|
| 80 |
+
| GPU | VRAM | Status | Notes |
|
| 81 |
+
| ------------------- | ----- | ------------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
| 82 |
+
| Nvidia A10G (24 GB) | 24 GB | β
Recommended | All three models load comfortably; narration + Q&A run without memory pressure |
|
| 83 |
+
| Nvidia T4 (16 GB) | 16 GB | β οΈ Functional | Works but is tight β voice cloning model may OOM if loaded concurrently with LFM; restart may be needed between heavy operations |
|
| 84 |
+
| CPU only | β | β Not viable | LFM2.5-Audio and Qwen3-TTS inference is too slow for real-time use |
|
| 85 |
+
|
| 86 |
|
| 87 |
**Hugging Face Spaces:**
|
| 88 |
+
|
| 89 |
- **A10G Small** β confirmed working, recommended for the hackathon demo
|
| 90 |
- **T4 Small** β runs with limitations; recommend disabling concurrent model loading
|
| 91 |
|
| 92 |
+
HuggingFace Space Hardware Options
|
| 93 |
|
| 94 |
**Memory breakdown (approximate):**
|
| 95 |
|
| 96 |
+
|
| 97 |
+
| Model | VRAM (bfloat16) |
|
| 98 |
+
| ----------------------------------- | --------------- |
|
| 99 |
+
| LFM2.5-Audio-1.5B | ~3 GB |
|
| 100 |
+
| Qwen3-TTS-0.6B-CustomVoice | ~1.2 GB |
|
| 101 |
+
| Qwen3-TTS-1.7B-Base (cloning) | ~3.4 GB |
|
| 102 |
+
| KV cache + activations at inference | ~1β2 GB |
|
| 103 |
+
| **Total (all models loaded)** | **~8β9 GB** |
|
| 104 |
+
|
| 105 |
|
| 106 |
The 1.7B cloning model is loaded on demand (not at startup), so idle VRAM usage is ~5β6 GB with only the narration and Q&A models preloaded.
|
| 107 |
|
|
|
|
| 152 |
|
| 153 |
## Stories Included
|
| 154 |
|
| 155 |
+
|
| 156 |
+
| Story | Words | Source |
|
| 157 |
+
| ---------------------------------- | ----- | ------------------- |
|
| 158 |
+
| The Tale of Peter Rabbit | 948 | Beatrix Potter |
|
| 159 |
+
| The Tale of Benjamin Bunny | 1,118 | Beatrix Potter |
|
| 160 |
+
| The Tale of Jemima Puddle-Duck | 1,245 | Beatrix Potter |
|
| 161 |
+
| The Tale of Tom Kitten | 691 | Beatrix Potter |
|
| 162 |
+
| The History of Tom Thumb | 2,912 | Traditional |
|
| 163 |
+
| The Story of the Three Little Pigs | 956 | Traditional |
|
| 164 |
+
| The Little Red Hen | 1,295 | Traditional |
|
| 165 |
+
| The Little Gingerbread Man | 1,823 | Traditional |
|
| 166 |
+
| The Sleeping Beauty | 1,783 | Traditional |
|
| 167 |
+
| The Adventures of Puss in Boots | 503 | Traditional (verse) |
|
| 168 |
+
|
| 169 |
|
| 170 |
All stories are public domain from [Project Gutenberg](https://www.gutenberg.org/).
|
| 171 |
|
|
|
|
| 188 |
Hackathon project. Stories are public domain.
|
| 189 |
test push verification
|
| 190 |
|
|
|
|
| 191 |
## Application Demo
|
| 192 |
|
| 193 |
# MomsVoice β Demo Walkthrough
|
|
|
|
| 198 |
|
| 199 |
## Step 1: Explore the Story Shelf
|
| 200 |
|
| 201 |
+
Explore Space
|
| 202 |
|
| 203 |
Open the **Explore Space** tab to browse available stories displayed as warm bookshelf cards. Each card shows the cover art, title, author, and a short excerpt. Click **"Tap to Play"** on any card to open it in My Library, or click **"More Info"** for the full story description.
|
| 204 |
|
|
|
|
| 208 |
|
| 209 |
## Step 2: Open My Library and Start Narration
|
| 210 |
|
| 211 |
+
My Library β Now Playing
|
| 212 |
|
| 213 |
Switch to the **My Library** tab. Click any story card to load it into the **Now Playing** panel on the right. The panel shows:
|
| 214 |
|
|
|
|
| 223 |
|
| 224 |
## Step 3: Ask a Question About the Story
|
| 225 |
|
| 226 |
+
Voice Q&A Panel
|
| 227 |
|
| 228 |
Scroll down in the Library panel to find the **Ask About the Story** section. Click **β Ask a Question** β narration pauses automatically and the microphone starts recording.
|
| 229 |
|
|
|
|
| 233 |
|
| 234 |
## Step 4: Hear the Spoken Answer (LFM2.5-Audio)
|
| 235 |
|
| 236 |
+
LFM Answer Audio
|
| 237 |
|
| 238 |
The recorded question waveform appears in the audio widget. Below it, the **LFM2.5-Audio** model generates a grounded answer in 1β2 sentences, based only on the current story context.
|
| 239 |
|
| 240 |
The answer is delivered in two ways simultaneously:
|
| 241 |
+
|
| 242 |
- **Answer text** shown inline (e.g. *"Puss in Boots, a poor man, bargained for boots and a bag, and Grimalkin, ailient rabbit, rescued him from a king's wrath."*)
|
| 243 |
- **Answer Audio** waveform auto-plays the spoken response
|
| 244 |
|
|
|
|
| 248 |
|
| 249 |
## Step 5: Open Clone Voice Studio and Record Your Voice
|
| 250 |
|
| 251 |
+
Clone Studio β Recording
|
| 252 |
|
| 253 |
Go to the **Clone Voice Studio** tab. Fill in:
|
| 254 |
+
|
| 255 |
- **Voice Nickname** β a name for this profile (e.g. *"Mom"*, *"Grandpa Joe"*)
|
| 256 |
- **Voice Gender Avatar** β Female or Male
|
| 257 |
|
|
|
|
| 261 |
|
| 262 |
## Step 6: Analyze & Synthesize the Vocal Profile
|
| 263 |
|
| 264 |
+
Clone Studio β Processing
|
| 265 |
|
| 266 |
Click **Analyze & Synthesize Customized Vocal Profile**. The **Status Pipeline Visualizer** on the right tracks progress through extraction, de-noising, and pitch tuning stages.
|
| 267 |
|
|
|
|
| 271 |
|
| 272 |
## Step 7: Voice Profile Ready
|
| 273 |
|
| 274 |
+
Clone Studio β Complete
|
| 275 |
|
| 276 |
When synthesis completes, the Status Pipeline Visualizer shows:
|
| 277 |
+
|
| 278 |
- Your **avatar** and nickname with a **"Cloned successfully"** badge
|
| 279 |
- *"Synthesized today using Qwen3-TTS voice cloning"*
|
| 280 |
- A **preview audio player** β tap Play to hear a short sample in your cloned voice
|
|
|
|
| 286 |
|
| 287 |
## Step 8: Story Narration in Your Cloned Voice
|
| 288 |
|
| 289 |
+
Narration with Cloned Voice
|
| 290 |
|
| 291 |
Return to **My Library** and select a story. The Now Playing panel shows **Active Narrator: [your nickname]** β confirming the cloned voice is active.
|
| 292 |
|
| 293 |
+
Narration plays in your recorded voice, paragraph by paragraph (*"Paragraph 1 / 32"*). The story text scrolls in the reader below the player. All TTS inference runs locally on the Space GPU β no audio leaves the server.
|