README.md

metadata

title: WaveGen
emoji: 🔊
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 6.17.3
python_version: 3.12.12
app_file: app.py
fullWidth: true
license: mit
models:
  - LeeSangHoon/WaveGen
short_description: WaveGen V-Spec and Flash text-to-speech on ZeroGPU

WaveGen

WaveGen: Truly End-to-End Waveform Generation via Internal Spectral Trajectory Learning
Sang-Hoon Lee and Ha-Yeong Choi · Ajou University and KT · NeurIPS 2026

Source Code · Models · Demo Page · Benchmark Demo

Generate English or Mandarin Chinese speech from a reference recording, its exact transcript, and the text to speak. The model emits 24 kHz waveform audio directly.

Model	Default sampling
WaveGen V-Spec — Recommended, selected by default	1400k EMA, 50 Euler steps, Prompt Anchor, Scheduled K3, CFG 1.25
WaveGen-Flash	48k EMA, four Euler steps, time grid [0, 0.1, 0.25, 0.5, 1.0], CFG 0

Both models default to initial Mel temperature 0 and skip the unused final Mel output head. Inference dropout follows each model's configuration. Model weights remain FP32; synthesis uses CUDA BF16 autocast. The reference plus predicted generated speech must fit within 30 seconds. Split longer text into shorter passages; the app does not silently truncate it or automatically split it.

Deploy this folder to Spaces

1. Create a Gradio Space under the intended account. Keep it Private while validating it, and select ZeroGPU hardware if the account is eligible. This README's SDK settings do not allocate hardware or change visibility.
2. Upload the contents of this folder to the Space repository root, including src/, the license files, requirements.txt, constraints.txt, and packages.txt. The root must contain app.py and this README.
3. If the model repository is private, add HF_TOKEN under the Space's Settings → Secrets, using a token with read access to LeeSangHoon/WaveGen. Keep the token out of source files, public variables, and commit messages. No token is needed once the selected model revision is publicly readable.
4. Let the Space build, then test the included example with both models and verify a downloaded WAV. Make the Space public only when ready for release.

The Space downloads each model's Safetensors checkpoint, YAML, and vocabulary at startup. The model repository and revision can be set with Space Variables:

Variable	Default
WAVEGEN_MODEL_REPO	LeeSangHoon/WaveGen
WAVEGEN_MODEL_REVISION	da53c835a39272c05e01526fe3c10b2b9d4ce60b

The default revision pins both model bundles to the same release. Use another full commit SHA to update them deliberately. Model files and authentication credentials are not included in this source bundle.

ZeroGPU runtime

The entry point imports spaces before PyTorch, loads the models during startup, and places their modules on CUDA before serving requests. The synthesis callback is wrapped with @spaces.GPU; model selection does not download weights or reload a model inside that callback. The app processes one generation at a time and returns a WAV file to preserve the model's output amplitude.

ZeroGPU allocates a GPU for each request and releases it afterward. Queue time and daily usage quotas depend on the visitor's Hugging Face account. Running the app on a local CUDA device verifies model integration, but does not itself verify Hugging Face's GPU allocation and worker lifecycle. A successful Space build and on-Space request are still required before claiming ZeroGPU deployment is tested.

See the official ZeroGPU documentation and Spaces configuration reference.

Versions and local verification

The existing development environment is not modified. The Space creates a separate Linux environment. Its inference and UI dependencies are pinned to the versions inspected in that environment, including:

• Gradio 6.17.3
• Spaces SDK 0.51.3 in the current build configuration, managed by ZeroGPU (local SDK 0.50.4 unchanged)
• PyTorch and TorchAudio 2.9.1+cu128
• Hugging Face Hub 0.36.2
• NumPy 2.2.6

The remote Python runtime is 3.12.12, a documented ZeroGPU runtime. The local Python 3.12.14 installation remains unchanged. Linux CUDA dependencies are provided by the pinned PyTorch wheels; hf-xet 1.8.0 is supplied for the Hub's Linux download transport. One existing transitive dependency cannot be copied verbatim into a fresh build: the local Pillow 13.0.0 exceeds Gradio 6.17.3's declared Pillow <13 requirement. The isolated Space therefore pins Pillow 12.2.0, while the local installation remains untouched. The platform's automatic Gradio MCP extra also requires Pydantic 2.12.5 and pydantic-core 2.41.5; the local versions remain 2.14.2 and 2.50.2. Exact deployment pins are inline in requirements.txt; constraints.txt is the reference manifest, including the observed platform-managed Spaces SDK. Spaces mounts only the requirements file during installation, so it cannot include a relative constraints file at that stage. The Torch requirement uses 2.9.1 for ZeroGPU's configuration parser, with the CUDA 12.8 wheel index.

For local CUDA verification with already downloaded model folders, run this from the bundle folder using the existing Python interpreter:

python app.py --local --model-dir /path/to/WaveGen/ckpts

The --local path uses the same models and sampling settings without requesting a Hugging Face GPU allocation. No package installation is necessary when using the already validated development environment.

The src/ files are unchanged copies of the inference APIs and the generator modules they import. Shared generator definitions include their original loss definitions, but training entry points, dataset loaders, evaluation tools, optimizer state, and checkpoints are excluded. SOURCE_SHA256SUMS records the copied files for comparison with the original source.

Responsible use and license

Use reference speech only with the speaker's consent and permission to use the recording. Do not use generated audio for deceptive impersonation, fraud, or harassment. Identify shared outputs as AI-generated and avoid uploading private or sensitive recordings to a public demo.

Source code is distributed under the MIT License, with the original third-party notices preserved. Model weights are licensed separately under CC BY-NC 4.0, as stated in the model card.