Spaces:
Paused
Paused
Commit History
-
Hand motion continuity to the next chunk from the last SHOWN frame, not the end of the computed chunk
JasonZhai-9936 +1: Claude Opus 5.5prev_x/prev_wa and current_dampening were always taken from frame 50 of each computed chunk. Since idle filler between TTS chunks is now cut short (~5 frames) when the next chunk arrives, chunk 2+ continued from an idle pose/motion strength (~0.3) that was never displayed, so the transition smoothing appeared to vanish after each reply's first chunk. Also fixes the smaller jump from a clip's last partial render chunk (padding frames). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>0e35357 -
Stop pre-rendering seconds of idle between TTS chunks of a reply
JasonZhai-9936 +1: Claude Opus 5.5With local Breeze TTS the next chunk often finishes after the current chunk's video has already rendered (FLOAT renders ~2x real time); the render loop then filled the 3s frame buffer with idle frames and the next chunk's speech_start queued behind them — a 2.75s pause between chunks in the sample log (c2 queued behind 72 frames). Mid-reply, idle now keeps at most a 5-frame (0.2s) lead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>48184a3 -
Render FLOAT on a high-priority CUDA stream so Breeze tail-chunk generation can't starve it
JasonZhai-9936 +1: Claude Opus 5.5While Breeze generated chunks 2..N, the first clip's FLOAT render slowed to 2.6-3x slower than real time (ode 0.03s -> 0.16s), playing slowed and out of sync; renders were normal as soon as Breeze finished. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>d3a6834 -
Fix FLOAT streamer crashing during Breeze CUDA-graph warmup
JasonZhai-9936 +1: Claude Opus 5.5Breeze's fast-path warmup captures CUDA graphs; the already-running streamer render loop called torch.randn on the shared default CUDA generator mid-capture and failed every chunk with "Offset increment outside graph capture". Warm Breeze up before starting the streamer, and give the streamer its own generator so it never shares RNG state with Breeze's graphs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>ba6b49a -
Revert "Add /api/timeline" — not wanted
JasonZhai-9936 +1: Claude Opus 5.5This reverts commit 059172b. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>945c904 -
Add /api/timeline: compact per-turn latency/sync summary (last 10 turns, ?full=1 for every line) so results can be copied without scraping the Space log viewer
JasonZhai-9936 +1: Claude Opus 5.5Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>059172b -
Cut first-response latency and stop A/V lag accumulating across chunks
JasonZhai-9936 +1: Claude Opus 5.5- split_sentences: cap the first chunk at 12 words (splitting a long opening sentence at a comma/dash), since TTS time scales with clip length and the first chunk's TTS sits directly on the reply's critical path. The old "12 words OR 2 sentences" rule glued a short first sentence to any-length second one, producing 8-13s first chunks. Later chunks cap at 25 words. - Preload: server sends a {"type":"preload"} WS event as soon as each chunk's TTS finishes (bypassing the frame buffer); the client fetches the clip then, so it plays the instant its speech_start arrives. Previously every clip started ~90ms after its video and the lag accumulated chunk over chunk (85 -> 188 -> 289 -> 398ms over a 4-chunk reply). - Timing log now says whether each clip was preloaded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>8458b91 -
Stop counting frame-buffer backpressure as render time — the SLOWER THAN REAL TIME warning fired whenever the buffer was full, i.e. when rendering was actually ahead
JasonZhai-9936 +1: Claude Opus 5.5Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>b84ee66 -
Add per-turn timeline logging: every stage of a reply (LLM, per-chunk TTS, feature encode, speech render, WS delivery, client audio start/end) logs in order as [T<turn> +<secs>] so sync/latency problems can be read straight from a copied log
JasonZhai-9936 +1: Claude Opus 5.5Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>abc34f2 -
Demote idle-chunk render logs to DEBUG so copied logs aren't flooded; keep speech-chunk timings at INFO
JasonZhai-9936 +1: Claude Opus 5.5Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>f4e3788
-
Fix WAV concat crash on Orpheus multi-piece TTS, ensure speech_end fires even when the final chunk fails, recalibrate barge-in threshold
JasonZhai-993681a267d -
Add barge-in: speaking while the avatar responds interrupts it and starts a new turn
JasonZhai-9936b6f2498 -
Add hands-free continuous voice mode with client-side VAD auto-submit
JasonZhai-993639407cd -
Cut avatar response latency: preempt idle frame rendering immediately when speech arrives, cap sentence merging in TTS chunking
JasonZhai-99363d8aa29 -
Log per-chunk GPU render timing in the FLOAT generation loop
JasonZhai-993662e5c7f
-
Update float_streamer.py
Verified signatured9d6205 -
Update float_streamer.py
Verified signature347e66f -
Update float_streamer.py
Verified signatured56f673 -
Update float_streamer.py
Verified signature8d85afa -
Update float_streamer.py
Verified signature4e883e1
-
Update float_streamer.py
Verified signature9ac0575 -
Update float_streamer.py
Verified signature91317dc -
Update float_streamer.py
Verified signature92d6383 -
Update float_streamer.py
Verified signature5edcdc7 -
Update float_streamer.py
Verified signaturea0ae063 -
Update float_streamer.py
Verified signature1a2979f -
Update float_streamer.py
Verified signature55aac57 -
Update float_streamer.py
Verified signatureffd00cc -
Update float_streamer.py
Verified signature806cefe -
Update float_streamer.py
Verified signaturee1d8608 -
Update float_streamer.py
Verified signature1ff5b72 -
Update float_streamer.py
Verified signature789cf28 -
Update float_streamer.py
Verified signature94cdccd -
Upload float_streamer.py
Verified signaturea9ae5ea