README.md

metadata

title: SemBridge — Speech & Singing
emoji: 🎤
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: 6.30.0
app_file: app.py
short_description: Unified TTS, singing, and mixed speech+song generation
python_version: '3.10'
startup_duration_timeout: 1h

SemBridge — Speech & Singing, one continuous voice

Demo for ASLP-lab/SemBridge — Semantic Token Anchoring for Bridging discrete semantic supervision and continuous acoustic generation. The unified model performs text-to-speech, note-conditioned singing (SVS), and mixed speech+singing with one reference voice, in English and Chinese.

• Tab 🗣️ Text to Speech — type a sentence, choose a packaged reference voice (or upload your own recording + transcript), and listen.
• Tab 🎵 Singing — enter a score as lyric|pitch|duration_seconds lines and render it with a sung reference voice.
• Tab 🎭 Mixed — run one of the paper's official speech → singing → speech sequences end to end.

Everything runs the authors' own sembridge inference package (bundled in this Space) against the released checkpoint: 64-D 25 Hz continuous latents, 2-frame patches, BigVGAN-Large / native causal decoding, 44.1 kHz mono WAV output, capped at 500 patches (40 s) per request.

The ten packaged voices and the singing scores are the exact inputs from the official demo page — Seed-TTS speech references and GTSinger / Opencpop sung references. Those assets retain their original rights and terms.

Technical notes

• ZeroGPU (zero-a10g), bf16, ~817M-parameter generator + 740M acoustic codecs.
• @spaces.GPU(duration=…) uses a callable estimator so each request reserves GPU time based on its expected length (flow steps × patch count) instead of a flat worst case.
• Reference audio is downmixed and resampled to 44.1 kHz automatically; provide ~4–10 s of clean audio plus its exact transcript.