OSS Inference Code: sub 50 ms TTFA at 10 requests per second, sub 100 ms TTFA at 20 requests per second
#56
by the-sweater-cat - opened
We built a Qwen3-TTS CustomVoice 1.7B implementation that achieves 10 requests per second (RPS) and sub-50 ms p95 time-to-first-audio (TTFA) while maintaining real-time playback on 1x NVIDIA H100. Even at 20 RPS, it sustains sub-80 ms p95 TTFA.
The code and container is open sourced here: https://github.com/nari-labs/nari-qwen3-tts.
Read the blog for a detailed breakdown: https://nari-labs.com/blog/qwen3-tts-speed-cost-frontier/