Build CPU-capable Speaking MIDI audio and VoiceDesign demo

README.md

@@ -55,25 +55,29 @@ All four spectrogram panels use the same color scale. Metrics compare the input-
 
 The fitting core is vendored from [Speaking MIDI](https://github.com/k-l-lambda/speaking-midi/tree/932f8eb), with a portable SoundFont path and an optional quick-refinement flag. See that project for full experiments, derivations, and reproducible demo provenance.
 
-## CPU and GPU execution
+## CPU and ZeroGPU execution
 
-The Space defaults to `INFERENCE_DEVICE=cpu`. TTS uses float32 on CPU and SDPA attention; no FlashAttention or ZeroGPU decorator is required. A separate process loads only VoiceDesign for text jobs and releases its memory before piano fitting. The fitting worker also runs on CPU, even when a GPU is present. Model downloads are cached between requests on the running instance. CPU-basic hardware can take several minutes; the first request additionally downloads the model. Each worker has a 30-minute timeout.
+The hosted [Speaking MIDI Space](https://huggingface.co/spaces/k-l-lambda/speaking-midi) uses `INFERENCE_DEVICE=zerogpu`. VoiceDesign loads at startup with CUDA placement and bfloat16, following the ZeroGPU model-loading contract. A `@spaces.GPU(duration=120)` function requests a shared GPU only for speech generation. MIDI fitting runs in a separate CPU worker and does not consume GPU quota. ZeroGPU requests can queue or fail when the visitor's GPU quota is exhausted; the precomputed speech examples can still be fitted without generating another voice.
 
-To run locally:
+The requirements use `torch==2.10.0` and matching torchaudio. Do not add `+cpu` or the CPU-only wheel index to the hosted requirements: ZeroGPU rejects that version specification and needs GPU-capable PyTorch.
+
+Local execution defaults to `INFERENCE_DEVICE=cpu`. TTS uses float32 with SDPA in a separate process, releasing model memory before MIDI fitting. No GPU is needed for this path.
 
 ```bash
 # Python 3.10; install libfluidsynth3, libsndfile1 and sox using your OS package manager.
 python -m venv .venv
 . .venv/bin/activate
 pip install -r requirements.txt gradio==6.17.3
-python app.py
+INFERENCE_DEVICE=cpu python app.py
 ```
 
-Optional environment variables: `QWEN_MODEL_PATH` selects an existing local VoiceDesign snapshot; `PORT` defaults to 7860; `OMP_NUM_THREADS` defaults to 2. For GPU execution, install CUDA-compatible PyTorch/torchaudio wheels instead of the CPU wheels and set `INFERENCE_DEVICE=cuda` (or `auto` for detection). CUDA TTS uses bfloat16. The deployment targets CPU-basic and does not request GPU hardware. On 2026-10-01, Hugging Face rejected Space creation for the current account with HTTP 402: even CPU-basic Gradio hosting requires a PRO subscription. An eligible account is required to deploy; local execution works without that subscription.
+Optional variables: `QWEN_MODEL_PATH` selects a local VoiceDesign snapshot; `PORT` defaults to 7860; `OMP_NUM_THREADS` defaults to 2. `INFERENCE_DEVICE=cuda` enables a regular local GPU, and `auto` detects CUDA availability. Keep `zerogpu` for a configured ZeroGPU runtime. For a smaller CPU-only local install, install PyTorch and torchaudio from the CPU wheel index separately; those wheels must not be used on ZeroGPU.
+
+Each CPU worker has a 30-minute timeout. Hosted GPU time is limited to 120 seconds per speech call. The Space uses shared ZeroGPU hardware, without dedicated paid hardware. The local directory remains `hf-speaking-midi`.
 
 ## Relationship to the Qwen Space
 
-The [official Qwen3-TTS Space](https://huggingface.co/spaces/Qwen/Qwen3-TTS/blob/6ae5dfa/app.py) was inspected as the backend reference. This app follows its `Qwen3TTSModel.from_pretrained(..., attn_implementation='sdpa')` and `generate_voice_design(text, language, instruct)` flow. Our backend is written separately and loads one model on demand, supports float32 CPU inference, and limits generation to 512 new tokens. We deliberately use VoiceDesign 1.7B because the 0.6B CustomVoice model does not honor natural-language voice instructions.
+The [official Qwen3-TTS Space](https://huggingface.co/spaces/Qwen/Qwen3-TTS/blob/6ae5dfa/app.py) was inspected as the backend reference. This app follows its `Qwen3TTSModel.from_pretrained(..., attn_implementation='sdpa')` and `generate_voice_design(text, language, instruct)` flow. Our backend is written separately, supports both startup-loaded ZeroGPU inference and on-demand float32 CPU inference, and limits generation to 512 new tokens. We deliberately use VoiceDesign 1.7B because the 0.6B CustomVoice model does not honor natural-language voice instructions.
 
 ## Assets and attribution
 
@@ -81,4 +85,4 @@ The bundled `assets/gm.sf3` is MuseScore General v0.2, with SHA-256 `5b85b6c2c61
 
 ## Deployment
 
-Authenticate using `hf auth login` or set `HF_TOKEN`, then run `python deploy.py`. The script creates `<authenticated-user>/hf-speaking-midi` and commits the application and assets to that Space. It never upgrades an account or requests paid hardware. See [validation results](VALIDATION.md) for what has actually been tested.
+Authenticate using `hf auth login` or set `HF_TOKEN`, then run `python deploy.py`. The script selects ZeroGPU hardware and creates `<authenticated-user>/speaking-midi` and commits the application and assets to that Space. It never upgrades an account or requests paid hardware. See [validation results](VALIDATION.md) for what has actually been tested.

app.py

@@ -18,8 +18,11 @@ ROOT = Path(__file__).resolve().parent
 VOICE = '一位温柔自信的年轻女性，声音清晰，语气亲切，表达轻快而富有感染力。'
 MODEL = 'Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign'
 DEVICE = os.getenv('INFERENCE_DEVICE', 'cpu')
-if DEVICE not in {'cpu', 'cuda', 'auto'}:
-    raise ValueError('INFERENCE_DEVICE must be cpu, cuda or auto')
+if DEVICE not in {'cpu', 'cuda', 'auto', 'zerogpu'}:
+    raise ValueError('INFERENCE_DEVICE must be cpu, cuda, auto or zerogpu')
+if DEVICE == 'zerogpu':
+    from zero_tts import generate_speech as generate_zero_speech
+
 RUNS = ROOT / 'outputs' / 'runs'
 RUNS.mkdir(parents=True, exist_ok=True)
 
@@ -27,8 +30,9 @@ RUNS.mkdir(parents=True, exist_ok=True)
 def worker(command, log, timeout=1800):
     """Bound each job and release all model memory when the child exits."""
     env = os.environ.copy()
-    if DEVICE == 'cpu':
+    if DEVICE in {'cpu', 'zerogpu'}:
         env['CUDA_VISIBLE_DEVICES'] = ''
+        env['SPACES_ZERO_GPU'] = '0'
     with log.open('w') as stream:
         process = subprocess.Popen([sys.executable, *command], cwd=ROOT, env=env,
                                    stdout=stream, stderr=subprocess.STDOUT)
@@ -99,7 +103,7 @@ def run(mode, audio, text, language, description, seconds, budget, refine, progr
     if mode == 'Text → speech → MIDI':
         if not text.strip() or len(text) > 300 or not description.strip() or len(description) > 600:
             raise gr.Error('Enter 1–300 text characters and a voice description of 1–600 characters.')
-        yield ('1/4 · Generating speech. CPU VoiceDesign can take several minutes on first use.', *empty)
+        yield ('1/4 · Generating speech. '+('Waiting for a shared GPU.' if DEVICE == 'zerogpu' else 'CPU VoiceDesign can take several minutes on first use.'), *empty)
         progress(.05, desc='Loading VoiceDesign and generating speech')
         prompts = directory/'prompts.json'
         prompts.write_text(json.dumps([dict(name='speech', text=text.strip(), language=language, instruct=description.strip())]))
@@ -107,7 +111,11 @@ def run(mode, audio, text, language, description, seconds, budget, refine, progr
                    '--samples', str(prompts), '--out', str(directory)]
         if os.getenv('QWEN_MODEL_PATH'):
             command += ['--model-path', os.environ['QWEN_MODEL_PATH']]
-        worker(command, directory/'tts.log')
+        if DEVICE == 'zerogpu':
+            wav, rate = generate_zero_speech(text.strip(), language, description.strip())
+            sf.write(directory/'speech.wav', wav, rate, subtype='FLOAT')
+        else:
+            worker(command, directory/'tts.log')
         audio = str(directory/'speech.wav')
     duration = prepare_audio(audio, directory/'input.wav', seconds)
     yield (f'2/4 · Analyzing the first {duration:.2f} s and building piano templates.',

deploy.py

@@ -4,10 +4,11 @@ from huggingface_hub import HfApi
 
 if __name__ == '__main__':
     api = HfApi()
-    repo = f"{api.whoami()['name']}/hf-speaking-midi"
-    api.create_repo(repo, repo_type='space', space_sdk='gradio', exist_ok=True)
+    repo = f"{api.whoami()['name']}/speaking-midi"
+    api.create_repo(repo, repo_type='space', space_sdk='gradio', space_hardware='zero-a10g', exist_ok=True)
+    api.add_space_variable(repo, 'INFERENCE_DEVICE', 'zerogpu')
     commit = api.upload_folder(repo_id=repo, repo_type='space', folder_path=Path(__file__).parent,
-        allow_patterns=['app.py', 'deploy.py', 'README.md', 'VALIDATION.md', 'requirements.txt', 'packages.txt',
+        allow_patterns=['app.py', 'zero_tts.py', 'deploy.py', 'README.md', 'VALIDATION.md', 'requirements.txt', 'packages.txt',
                         '.gitignore', 'speaking_midi/*.py', 'scripts/*.py', 'tests/*.py',
                         'assets/*', 'examples/**'],
         commit_message='Build CPU-capable Speaking MIDI audio and VoiceDesign demo')

requirements.txt

@@ -1,6 +1,5 @@
---extra-index-url https://download.pytorch.org/whl/cpu
-torch==2.10.0+cpu
-torchaudio==2.10.0+cpu
+torch==2.10.0
+torchaudio==2.10.0
 qwen-tts==0.1.1
 transformers==4.57.3
 accelerate==1.12.0
@@ -10,3 +9,4 @@ soundfile==0.13.1
 pyfluidsynth==1.4.0
 mido==1.3.3
 matplotlib==3.10.8
+spaces

zero_tts.py

@@ -0,0 +1,25 @@
+"""ZeroGPU VoiceDesign entry point; CPU execution remains in the CLI worker."""
+import spaces  # Initialize ZeroGPU before importing torch or loading the model.
+import os
+import numpy as np
+import torch
+from huggingface_hub import snapshot_download
+from qwen_tts import Qwen3TTSModel
+
+MODEL_ID = 'Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign'
+REVISION = '5ecdb67327fd37bb2e042aab12ff7391903235d3'
+path = os.getenv('QWEN_MODEL_PATH') or snapshot_download(MODEL_ID, revision=REVISION)
+# ZeroGPU records startup CUDA placements and restores them when a GPU is allocated.
+model = Qwen3TTSModel.from_pretrained(
+    path, device_map='cuda', dtype=torch.bfloat16, attn_implementation='sdpa')
+
+
+@spaces.GPU(duration=120)
+def generate_speech(text, language, description):
+    torch.manual_seed(42)
+    np.random.seed(42)
+    with torch.inference_mode():
+        wavs, rate = model.generate_voice_design(
+            text=text, language=language, instruct=description,
+            non_streaming_mode=True, max_new_tokens=512)
+    return np.asarray(wavs[0], dtype=np.float32), rate