Spaces:
Running
Running
Commit History
-
feat(voice): streaming TTS end-to-end with diagnostics, Aidoru theming, visible TTFA
kitsunechatThis is the cumulative "kitsunechat v1" commit — every fix, every theme overhaul, and every diagnostic we have landed since forking tylermullen/Gemma4. CHANGELOG.md is the narrative form of this commit; the bullets below are the headline. ======================================================================== THE PIPELINE — what was actually built ======================================================================== The chat now speaks as the LLM types. The flow: user presses Send -> MediaPipe LlmService streams tokens into llmService.history$ -> tts-bridge.js subscribes to history$, diffs new chars, extracts sentences on boundary -> tts.generate(sentence) (Pocket TTS worker, 24 kHz Float32) -> diffusion chunks streamed back as they are produced -> resampled 24k -> native in JS, batched for 30 ms -> AudioWorkletNode plays FIFO, gapless -> speakers + same PCM drives avatar lip-sync …ac1593b
-
fix(ttfa): subscribe to llmService.history$ instead of DOM observer
kitsunechatThe previous MutationObserver-on-shadow-DOM approach was defeated by LitElement's repeat() directive, which keys on message.text.length and recreates .message-wrapper elements on every text update. By the time our observer re-attached, the LLM had often already finished, so we fell into maybeFullResponseFallback and TTS'd the whole 5000-char response in one go — that's why audio only started after the full text was on screen. New approach: subscribe to llmChat.llmService.history (RxJS BehaviorSubject). Each emission has the cumulative model text. Diff against last-seen, extract sentences on boundary, enqueue to TTS. This bypasses DOM entirely and reacts to each MediaPipe partial result. Also fix: discarding=true was getting stuck when the OLD generation hadn't yet pumped any TTS (no onDone would fire to clear it). Now onNewGen only sets discarding=true if busy=true (OLD TTS in flight). Cache-buster bump ?v=fix3.4f371a1 -
fix(air-gaps): strip fade envelope from worklet, batch audio chunks, add diagnostics
kitsunechat- audio-worklet.js: remove 20ms fade-in/out per chunk — the fade was creating audible dips to silence at every chunk boundary. Pocket TTS emits one chunk per diffusion step, so with fade every step haloed in silence = the air gaps every 1 second. - tts-bridge.js: collect diffusion chunks for BATCH_MS (30ms) and post one combined Float32 to the worklet (matches tts-site's flushAudioBatch pattern). Reduces main-thread churn and gives the worklet larger contiguous pieces. - tts-bridge.js: lower streaming thresholds (MIN_FIRST_CHARS=20, MIN_CHUNK_CHARS=12) in case the LLM ever streams to DOM. - tts-bridge.js: expose window.ttsBridgeStats — diffusion chunk count, worklet post count, samples posted, per-chunk timing (gap, max, sum), generation duration, discard drops. Read by GET /snapshot. - Bump cache-busters to ?v=fix2 (including audio-worklet.js).ef15735 -
Add cache-buster query strings to script tags
kitsunechat417734a -
tts-bridge: fix discarding-stuck bug via count-based active detection
kitsunechatBug: scanChatHistory fired on every text mutation during LLM streaming. Because LitElement apparently recreates .message-wrapper.model-message elements on re-render, `active !== previousActiveWrapper` was true on nearly every scan. Each mismatch called onActiveWrapperChanged, which sets `discarding = true` (drops in-flight audio chunks) and only resets on the OLD worker's `onDone`. With the OLD worker constantly being discarded mid-generation, onDone rarely fired, so `discarding` stayed stuck at true → user saw text but no audio. Fix: - Track `previousWrapperCount` and only treat the active as "changed" when the wrapper COUNT increases (i.e., a new prompt was sent). - When the active reference changes but the count is stable, the wrapper was recreated mid-stream of the SAME message — re-target the existing stream's MutationObserver to the new element instead of discarding. - Add a 45s safety timeout that force-resets `discarding` if it ever gets stuck again.209cffb -
Queue-clear on active switch + JS-side resampling
Verified signatured52f4c5 -
Queue-clear on active switch + JS-side resampling
Verified signatureef3e68e
-
Resample 24kHz->native in worklet + diagnostic logging
Verified signature641bcff -
Resample 24kHz->native in worklet + diagnostic logging
Verified signaturee183efc -
Streaming TTS: worklet playback + 30ms crossfade
Verified signature8fb3c5e -
Streaming TTS: worklet playback + 30ms crossfade
Verified signaturedfb7d0c -
Add wasm/genai_wasm_module_internal.wasm
Verified signaturec33e468 -
Add wasm/genai_wasm_module_internal.js
Verified signaturec332788 -
Add wasm/genai_wasm_internal.wasm
Verified signature1da26f9 -
Add wasm/genai_wasm_internal.js
Verified signatured5a89cf -
Add pkg/tts_wasm_bg.wasm
Verified signature57372cd -
Add pkg/tts_wasm.js
Verified signature4c563b5 -
Add .gitattributes
Verified signature8a7894c -
Add README.md
Verified signaturee1c924c -
Add audio-worklet.js
Verified signature06863b0 -
Add worker.js
Verified signature30f90b6 -
Add tts-client.js
Verified signature4f71a28 -
Add tts-bridge.js
Verified signature031bd26 -
Add bundle.js
Verified signatureb4b1c3b -
Add index.html
Verified signature23ba245 -
initial commit
Verified signature5dc2f35