Spaces:
Running
Running
Commit History
-
model.ts: rename ONNX input ref_s → style for kokoro-js / kokoro-onnx interop
Verified signatureMatches the upstream Kokoro ecosystem convention (kokoro-js calls it 'style', thewh1teagle/kokoro-onnx calls it 'style', upstream KModelForONNX's signature is just a Python parameter name and doesn't appear in the ONNX surface). Function signature on synthesize() keeps the 'refS' argument name for backward compatibility — it's the same [1, 256] voicepack slice either way; only the wire format changed. Coupled with the new ONNX model upload that renames the input on the hub side. Brief ~30 sec deploy gap during which old browser cache might fail one synth — recovers on refresh.cc0aa08
-
ui: group voice picker by gender via <optgroup> (24 voices was unwieldy as flat list)
Verified signatureFemale voices are listed first under 'आवाज (स्त्री) — Female voices' optgroup, then male under 'आवाज (पुरुष) — Male voices'. Manifest order is preserved within each group so Marathi-trained voices float to the top above the stock-Kokoro crossovers and synthetic ones. The trailing '— female'/'— male' is dropped from each option label since the optgroup label now conveys it. Native <optgroup> chosen over a custom card grid for round one — keeps the dropdown a single tap on mobile, accessible by default, no JS state to manage. A card-grid + filter-chips upgrade is plausible follow-up if voice count grows further or browse-by-source becomes useful.7e1c310 -
synthesize: strip BOS+EOS audio from output (keep BOS in model input)
Verified signatureUser feedback: yesterday's BOS-prepend fix removed the leading-consonant eat on SpringLab voices ('मुंबई' now sounds like Mumbai not umbai), but Rasa voices (Asha + Vivek) gained an audible 'umm' at the start of every utterance. Diagnosis: Rasa training audio kept natural pre-speech sounds (breath, soft throat clearing). The decoder learned BOS audio = those breathy pre-vowel sounds. SpringLab clips were trimmed at corpus level so its BOS audio = silence. With BOS now in input, Rasa's BOS region surfaces as the umm; SpringLab's stays silent. Fix: keep BOS in the *model input* (predictor still gets boundary-token context → still allocates proper duration to first content phoneme → m-eat stays fixed) but strip the BOS audio prefix from the *output* buffer. Same for EOS at the end. startSample = bosFrames * HOP endSample = audio.length - eosFrames * HOP audio = fullAudio.subarray(startSample, endSample) leadOffsetSec drops to 0 since the audio buffer now starts directly …b367e53 -
synthesize: prepend BOS/EOS to inputIds (was missing — caused leading-consonant eat)
Verified signatureKokoro's KModel.forward (Python) does `[0, *input_ids, 0]` before calling forward_with_tokens. Our ONNX export exposes forward_with_tokens directly so the caller has to wrap. The TS pipeline never did. Symptom: leading consonants on SpringLab voices got perceptually eaten ('मुंबई' → 'umbai'; the m vanished). Without BOS, the iSTFTNet decoder's window-startup transient (~30-50 ms unreliable output) lands on the first content phoneme. With BOS, the predictor allocates real duration to the boundary token and the decoder produces clean pre- content audio before the first phoneme. Implementation: - synthesize() wraps inputIds = [0, ...ids, 0] before model call - strips BOS+EOS positions from the returned pred_dur so it aligns 1:1 with caller's original inputIds (buildTimings unchanged) - exposes leadOffsetSec = bosFrames * 600 / 24000 so caller can shift content phoneme times into the right slot of the audio buffer (which still includes the BOS warmup audio at position 0…73b1551 -
audio: prepend 80ms leading silence to fix SpringLab utterance-initial consonant eat
Verified signatureMukta + Dnyanesh (SpringLab voices) drop the leading consonant of the first word. mˈumbˌi (मुंबई) plays as 'umbai' — the m gets eaten. Asha + Vivek (Rasa) don't show it because their training audio preserved natural pre-speech silence; SpringLab clips were aggressively trimmed at corpus level so the voicepack encodes 'speak from sample zero'. The iSTFTNet decoder's window-startup transient (~30-50 ms unreliable output) then collides with the first phoneme. Fix: 80 ms of zeros at audio start. Universal across voices (costs nothing on Rasa, fixes SpringLab). Phoneme + word timings shifted forward by 80 ms via the existing audioOffsetSamples accumulator, so the highlight bar stays aligned with playback.e8c8870
-
punctuation pauses: splice silence after , ; : — . ! ? …
Verified signatureThe Marathi v0.2 predictor under-allocates duration to punctuation tokens — likely because training data was silence-trimmed during prep, so the predictor never saw enough natural pauses around commas/periods to learn from. Result: long sentences with internal commas synthesize without audible breaks; the user requested a fix on a real-world example ("...बघितला मी, पण त्यांच्याकडे..." with no pause after मी,). Client-side fix: after each chunk's synth, walk the phoneme array, detect punctuation tokens, splice extra silence into the audio buffer at each punctuation phoneme's end-time. Adjust phoneme + word timings forward to keep the highlight bar aligned with the spliced audio. Pause durations: , ; : — → 200-250 ms (minor break) . ! ? … → 300 ms (major break) This is a workaround until v0.6 retrains with preserved punctuation prosody — the predictor will then produce real pauses natively and this hack becomes obsolete (and can be deleted).c0f087e -
metrics: TTFB (time to first audio) replaces total-synth in headline metric
Verified signatureFor multi-chunk inputs the perceived latency is when the FIRST chunk's audio is ready, not when the full utterance finishes. Streaming would play it immediately; even without streaming, this is the more honest number to surface. Single-chunk inputs: TTFB = total synth (unchanged). Card now shows TTFB as the headline; total synth time is in the sub-context line.aed4de5 -
ui: performance metrics card + voice ref-clip download + WAV download
Verified signatureThree additions to the demo, all surface-level (no model/voicepack changes): 1. Voice ref-clip download — small "🎧 Reference clip" link next to the voice picker. Each voice serves a representative training-data clip (~5-7s MP3, ~70 KB) so users can hear the actual speaker, not just the TTS output. Clips per voice: mf_asha → rasa marathi_female_00000 mm_vivek → rasa marathi_male_00000 mf_mukta → springlab springlab_female_000000 mm_dnyanesh → springlab springlab_male_000000 2. Performance metrics card — new section below the transcript with three metrics: Synth time (ms, model forward only) Real-Time Factor (audio_seconds / synth_seconds; >1× = faster than realtime; typical 5-10× on WebGPU, 1-3× on WASM fallback) Tokens / sec (with token + frame counts in subtext) Hidden until first synth completes; matches the Pocket-TTS-web diagnostics style the user pointed at. 3. Download synt…3896c3f -
speed slider now scales pred_dur (was playback-rate only)
Verified signatureRe-exported v0.2 ONNX with speed as a third dynamic input (float32[1]); KModel.forward_with_tokens divides duration_proj output by speed before rounding, so 0.85x actually allocates +18% frames per phoneme — consonants get room to breathe instead of being crammed and played back slow. Wiring: - model.ts: synthesize(ids, refS, speed) passes a Tensor("float32", [speed], [1]) as the 'speed' ONNX input. - main.ts: pulls speedEl.value into the synthesize call. audioEl.playbackRate stays at 1.0 — the buffer is already at the requested pacing, so per-word timings line up with playback. Side effect: re-export with torch 2.6 produced ~22% longer audio at speed=1.0 than the previous TORCH26 export (Pearson r 0.992 vs old — same audio shape, slight numerical drift from the InstanceNorm-train warning). Net: at speed=0.85, perceived pacing comparable to old slider=0.85 setup.cc4d90f
-
feat: per-word display shows user's original Roman input for substituted loanwords
Verified signaturef804bbe -
cleanup: remove verification marker
Verified signaturebd52a50 -
fix: actually deploy textarea-preserving build (build entry was contaminated by previous dist)
Verified signature6165c84 -
src: align main.ts source with deploy (in-memory transliterate)
Verified signaturee0a5ee7 -
loanword preprocessor + re-exported v0.2 ONNX (torch 2.6)
Verified signatureTwo changes that take the demo from broken to working: 1. **Loanword preprocessor.** Latin English words in the input (e.g., "Weekend ला movie बघायचा plan आहे का?") get auto-replaced with their conventional Devanagari spellings (वीकेंड ला मोव्ही बघायचा प्लॅन आहे का?) before phonemization. The 19,436-entry map is built from the IndicCMix loanword corpus, frequency-deduped, lower-cased keys (~700 KB, fetched in parallel with model load). Routes Hinglish through espeak mr G2P (Marathi-IPA neighborhood the decoder knows) instead of espeak en G2P (English-IPA the decoder produces "weekenda"-style approximations for). Sanscript ITRANS button now does dict-lookup first + sanscript fallback. 2. **Local-model fallback toggle** for dev: ?local query loads ONNX from /models/bol-tts/ instead of HF (used during testing the re-export). Default behavior unchanged for production users. Note: paired with `shreyask/bol-tts-marathi-onnx` re-uploaded with a fresh torch-2.6…761f477
-
v0.2 polish: per-voice default speeds + trailing audio pad
Verified signature- voicepacks.json: Asha 0.85, Vivek 0.9, Mukta 0.9, Dnyanesh 0.9 (matches v0.1 Asha/Vivek natural-pace docs; SPRINGLab voices sound natural at the same range per listening tests) - main.ts: 80 ms trailing pad after last chunk's audio so ISTFTNet output's final phoneme release transient isn't clipped at buffer edge — fixes "last word eaten" perception across all voicepacks7cc7cf8
-
voicepacks: gender-paired chips + dynamic textarea default
Verified signature- 4 marathi M/F chip pairs (was 1): थकलो/थकले, गेलो/गेले, मित्रांना म्हणालो / मैत्रिणींना म्हणाले, story - default_textarea per gender; main.ts swaps in on initial load to match the default voice (female default uses मैत्रिणींना for natural Marathi instead of generic मित्रांना)693eb3e -
release: webgpu demo + source
Verified signature- 6 voicepacks (added Priya + Arjun from SPRINGLab) - voicepacks.json: single source of truth for voice list + per-gender chips - dynamic voice dropdown + chips (replaces hardcoded HTML) - ship full source (src/, package.json, tsconfig.json, vite.config.ts) so forks can rebuild without de-minifying - LFS: add glob assets/index-*.js + espeakng worker2bb8806