Spaces:
Paused
Paused
Commit Β·
3cfcf98
1
Parent(s): 57efb6d
docs: update README - reflect Qwen3-TTS, voice-first Q&A, all current features
Browse files
README.md
CHANGED
|
@@ -9,49 +9,64 @@ app_file: app.py
|
|
| 9 |
pinned: false
|
| 10 |
---
|
| 11 |
|
| 12 |
-
#
|
| 13 |
|
| 14 |
**A parent records their voice. A bedtime story plays in that voice. The child can ask questions and hear answers β all in mom or dad's voice.**
|
| 15 |
|
| 16 |
Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gradio app on Hugging Face Spaces β no external APIs, no data leaves the server.
|
| 17 |
|
| 18 |
-
|
|
|
|
|
|
|
| 19 |
|
| 20 |
```
|
| 21 |
π€ Record 15s of your voice β π Pick a story β βΆοΈ Story plays in your voice
|
| 22 |
β
|
| 23 |
-
β Child taps Ask β Story pauses
|
| 24 |
β
|
| 25 |
-
Child asks a question β
|
| 26 |
β
|
| 27 |
-
|
| 28 |
```
|
| 29 |
|
| 30 |
## How It Works
|
| 31 |
|
| 32 |
| Step | What Happens | Model |
|
| 33 |
|---|---|---|
|
| 34 |
-
| **Clone** | Parent records or uploads 15β30s of audio. Speaker embedding is extracted and cached
|
| 35 |
-
| **Listen** | Story plays in the cloned voice as interruptible sentence chunks. Falls back to
|
| 36 |
-
| **Ask** | Child taps Ask
|
| 37 |
-
| **Answer** | A short grounded answer is generated from the story context and spoken in the
|
| 38 |
-
| **Resume** | Story continues from where it left off. | Cached chunks |
|
| 39 |
|
| 40 |
## Models
|
| 41 |
|
| 42 |
| Model | Role | Size |
|
| 43 |
|---|---|---|
|
| 44 |
| [Qwen3-TTS-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) | Zero-shot voice cloning + TTS | 1.7B params |
|
| 45 |
-
| [
|
| 46 |
-
| [Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) | Story Q&A | 3B params
|
| 47 |
-
| [Whisper-small](https://huggingface.co/openai/whisper-small) | Child speech-to-text | 244M params
|
|
|
|
|
|
|
| 48 |
|
| 49 |
## Tech Stack
|
| 50 |
|
| 51 |
-
- **UI:** Gradio 5.
|
| 52 |
-
- **Runtime:** Single `app.py` process on HF Spaces (
|
| 53 |
- **Stories:** 10 public domain `.txt` files (downloaded and cleaned from Project Gutenberg)
|
| 54 |
-
- **Storage:** In-memory session cache
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
## Quick Start (Local Dev)
|
| 57 |
|
|
@@ -66,22 +81,23 @@ python app.py
|
|
| 66 |
## Project Structure
|
| 67 |
|
| 68 |
```
|
| 69 |
-
βββ app.py # Main Gradio app (UI + wiring)
|
| 70 |
-
βββ voice_clone.py # Qwen3-TTS voice cloning + profile
|
| 71 |
-
βββ tts.py # Unified TTS interface (
|
| 72 |
-
βββ inference.py # ASR (Whisper) + Q&A (Qwen2.5-3B-Instruct)
|
| 73 |
-
βββ requirements.txt # Python deps
|
| 74 |
βββ stories/ # 10 cleaned public domain story texts (TTS-ready)
|
| 75 |
βββ story_downloader/ # Story acquisition & cleaning pipeline
|
| 76 |
β βββ gutenberg_downloader.py # Reusable Project Gutenberg downloader/parser
|
| 77 |
β βββ download_stories.py # Downloads 10 children's stories
|
| 78 |
β βββ clean_stories.py # Strips Gutenberg boilerplate for TTS
|
| 79 |
-
βββ static/
|
| 80 |
βββ assets/covers/ # Story cover images
|
|
|
|
| 81 |
βββ test_modules/ # Component and integration tests
|
| 82 |
βββ mission.md # Product vision
|
| 83 |
βββ sprint.md # 2-day hackathon sprint plan
|
| 84 |
-
βββ tech_stack.md
|
| 85 |
βββ future_mobile_app_considerations.md # Mobile deployment guidance
|
| 86 |
```
|
| 87 |
|
|
@@ -100,19 +116,20 @@ python app.py
|
|
| 100 |
| The Sleeping Beauty | 1,783 | Traditional |
|
| 101 |
| The Adventures of Puss in Boots | 503 | Traditional (verse) |
|
| 102 |
|
| 103 |
-
All stories are public domain from [Project Gutenberg](https://www.gutenberg.org/).
|
| 104 |
|
| 105 |
## Key Design Decisions
|
| 106 |
|
| 107 |
- **All local inference** β voice, Q&A, and ASR run on the Space GPU. No external APIs.
|
| 108 |
-
- **Interruptible chunked streaming** β
|
| 109 |
-
- **
|
| 110 |
-
- **
|
| 111 |
-
- **
|
|
|
|
| 112 |
|
| 113 |
## Privacy
|
| 114 |
|
| 115 |
-
All inference runs on the Hugging Face Space GPU. Voice samples, story text, and generated audio stay on the server runtime
|
| 116 |
|
| 117 |
## License
|
| 118 |
|
|
|
|
| 9 |
pinned: false
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# MomsVoice
|
| 13 |
|
| 14 |
**A parent records their voice. A bedtime story plays in that voice. The child can ask questions and hear answers β all in mom or dad's voice.**
|
| 15 |
|
| 16 |
Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gradio app on Hugging Face Spaces β no external APIs, no data leaves the server.
|
| 17 |
|
| 18 |
+
π **Live Demo:** [https://huggingface.co/spaces/DrSquare/MomsVoice](https://huggingface.co/spaces/DrSquare/MomsVoice)
|
| 19 |
+
|
| 20 |
+
## Demo Flow
|
| 21 |
|
| 22 |
```
|
| 23 |
π€ Record 15s of your voice β π Pick a story β βΆοΈ Story plays in your voice
|
| 24 |
β
|
| 25 |
+
β Child taps Ask β Story pauses, mic auto-starts
|
| 26 |
β
|
| 27 |
+
Child asks a question β Answer auto-plays in narrator's voice
|
| 28 |
β
|
| 29 |
+
β©οΈ Resume β Story continues from where it paused
|
| 30 |
```
|
| 31 |
|
| 32 |
## How It Works
|
| 33 |
|
| 34 |
| Step | What Happens | Model |
|
| 35 |
|---|---|---|
|
| 36 |
+
| **Clone** | Parent records or uploads 15β30s of audio. Speaker embedding is extracted and cached. | Qwen3-TTS-1.7B Base |
|
| 37 |
+
| **Listen** | Story plays in the cloned voice as interruptible sentence chunks. Falls back to CustomVoice 0.6B stock voice (speaker "vivian") if no clone. | Qwen3-TTS-1.7B / CustomVoice 0.6B |
|
| 38 |
+
| **Ask** | Child taps Ask β narration pauses, microphone auto-starts. When recording stops, question is transcribed and answer auto-generates. | Whisper-small (ASR) |
|
| 39 |
+
| **Answer** | A short grounded answer is generated from the full story context and spoken in the narrator's voice. Auto-plays immediately. | Qwen2.5-3B-Instruct + Qwen3-TTS |
|
| 40 |
+
| **Resume** | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
|
| 41 |
|
| 42 |
## Models
|
| 43 |
|
| 44 |
| Model | Role | Size |
|
| 45 |
|---|---|---|
|
| 46 |
| [Qwen3-TTS-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) | Zero-shot voice cloning + TTS | 1.7B params |
|
| 47 |
+
| [Qwen3-TTS-0.6B-CustomVoice](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice) | Fast stock-voice TTS (predefined speakers) | 0.6B params |
|
| 48 |
+
| [Qwen2.5-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct) | Story Q&A (context-grounded answers) | 3B params |
|
| 49 |
+
| [Whisper-small](https://huggingface.co/openai/whisper-small) | Child speech-to-text | 244M params |
|
| 50 |
+
|
| 51 |
+
All models are preloaded at startup for fast response times.
|
| 52 |
|
| 53 |
## Tech Stack
|
| 54 |
|
| 55 |
+
- **UI:** Gradio 5.35.0 with custom CSS/JS (Google Stitch-inspired warm design)
|
| 56 |
+
- **Runtime:** Single `app.py` process on HF Spaces (GPU: A10G or larger recommended)
|
| 57 |
- **Stories:** 10 public domain `.txt` files (downloaded and cleaned from Project Gutenberg)
|
| 58 |
+
- **Storage:** In-memory session cache + disk-persisted voice profiles (`Voice_Profile/`)
|
| 59 |
+
- **TTS Backends:** Qwen3-TTS Base (cloned voice) + CustomVoice 0.6B (stock fallback)
|
| 60 |
+
|
| 61 |
+
## Features
|
| 62 |
+
|
| 63 |
+
- π€ **Voice Cloning** β Record 15s, get a narrator that sounds like you
|
| 64 |
+
- π **Explore & Library** β Browse and select stories with clickable book cards
|
| 65 |
+
- βΆοΈ **Chunked Streaming** β Stories play paragraph-by-paragraph with real-time highlighting
|
| 66 |
+
- β **Voice Q&A** β Ask questions about the story; auto-submit on recording stop, autoplay answer
|
| 67 |
+
- βΈ **Smart Pause** β Playback pauses on Ask, tab switch (Explore/Clone Studio), or Pause button
|
| 68 |
+
- ποΈ **Voice Selection** β Click voice cards to switch between cloned narrators
|
| 69 |
+
- πΎ **Persistent Profiles** β Cloned voices survive app restarts
|
| 70 |
|
| 71 |
## Quick Start (Local Dev)
|
| 72 |
|
|
|
|
| 81 |
## Project Structure
|
| 82 |
|
| 83 |
```
|
| 84 |
+
βββ app.py # Main Gradio app (UI + all event wiring)
|
| 85 |
+
βββ voice_clone.py # Qwen3-TTS voice cloning + profile persistence
|
| 86 |
+
βββ tts.py # Unified TTS interface (Base cloned + CustomVoice stock)
|
| 87 |
+
βββ inference.py # ASR (Whisper-small) + Q&A (Qwen2.5-3B-Instruct)
|
| 88 |
+
βββ requirements.txt # Python deps (gradio, qwen-tts, transformers, torch)
|
| 89 |
βββ stories/ # 10 cleaned public domain story texts (TTS-ready)
|
| 90 |
βββ story_downloader/ # Story acquisition & cleaning pipeline
|
| 91 |
β βββ gutenberg_downloader.py # Reusable Project Gutenberg downloader/parser
|
| 92 |
β βββ download_stories.py # Downloads 10 children's stories
|
| 93 |
β βββ clean_stories.py # Strips Gutenberg boilerplate for TTS
|
| 94 |
+
βββ static/style.css # Custom CSS for warm storybook UI
|
| 95 |
βββ assets/covers/ # Story cover images
|
| 96 |
+
βββ Voice_Profile/ # Persisted cloned voice profiles
|
| 97 |
βββ test_modules/ # Component and integration tests
|
| 98 |
βββ mission.md # Product vision
|
| 99 |
βββ sprint.md # 2-day hackathon sprint plan
|
| 100 |
+
βββ tech_stack.md # Technical architecture
|
| 101 |
βββ future_mobile_app_considerations.md # Mobile deployment guidance
|
| 102 |
```
|
| 103 |
|
|
|
|
| 116 |
| The Sleeping Beauty | 1,783 | Traditional |
|
| 117 |
| The Adventures of Puss in Boots | 503 | Traditional (verse) |
|
| 118 |
|
| 119 |
+
All stories are public domain from [Project Gutenberg](https://www.gutenberg.org/).
|
| 120 |
|
| 121 |
## Key Design Decisions
|
| 122 |
|
| 123 |
- **All local inference** β voice, Q&A, and ASR run on the Space GPU. No external APIs.
|
| 124 |
+
- **Interruptible chunked streaming** β sentences synthesized and played one at a time for fast start and clean pause/resume.
|
| 125 |
+
- **Voice-first Q&A** β microphone auto-starts on Ask; answer auto-plays when ready. No manual button clicks needed.
|
| 126 |
+
- **Full story context** β Q&A uses the entire story (or top-5 relevant paragraphs for long stories) plus current position for grounded answers.
|
| 127 |
+
- **Smart pause** β playback pauses on Ask, tab switch, or Pause button. Resume continues from exact position.
|
| 128 |
+
- **Privacy-first** β no audio leaves the server, no user accounts, no external database.
|
| 129 |
|
| 130 |
## Privacy
|
| 131 |
|
| 132 |
+
All inference runs on the Hugging Face Space GPU. Voice samples, story text, and generated audio stay on the server runtime. Voice profiles are saved to the Space's persistent storage but no data is sent to external services.
|
| 133 |
|
| 134 |
## License
|
| 135 |
|