Loading live status…
500 public synthetic voice identities, about 50 selected clips per voice, 100,000 procedural 24-second conversation clocks and 500,000 selected supervised steps. The source archives were accessed using bounded HTTP ranges; full archives and conversation WAV files were not downloaded or retained. The immutable speaker split is 480 training / 10 validation / 10 test. Follow-up V4 and V5 studies and the current integration report are linked below.
Every selected step uses the frozen Nemotron layer-15 states from the causal eight-second window ending at that step. We store the last 25 states (80 ms each), plus optional voiced full-window mean and standard deviation. Students predict the identity active in the newest 320 ms. Thus we compare relevant earlier hidden states, not only each chunk’s final state. Pooling spans of 320–2,000 ms are not audio-history limits: the hidden states already contain the encoder’s earlier context.
| Candidate | Trainable parameters | Idea |
|---|---|---|
| Attentive statistics MLP | 582,019 | Current-step query selects past frames; weighted mean/std. |
| Query GRU | 648,067 | Small causal recurrent context plus query pooling. |
| Causal temporal CNN | 600,835 | Three dilated depthwise temporal blocks. |
| Small Transformer | 817,155 | Two causal attention blocks, 128 hidden dimensions. |
Targets are 192-dimensional frozen ECAPA embeddings from complete clean source utterances. Separate enrollment clips establish voice prototypes. We train centered and raw cosine regression with a small normalized-MSE term, prototype contrastive supervision and speech/silence/overlap classification. Evaluation includes 320, 640, 1,040 and 2,000 ms context; optional eight-second pooled features are reported separately.
Architecture/checkpoint selection uses validation identities only: macro speaker accuracy, EER and source-utterance teacher cosine. The selected model is then evaluated on unseen test speakers, including validation-calibrated unknown-speaker rejection, silence/overlap recall and noise/reverb/telephone/echo subsets. Shared source utterances make windows correlated; no human-speaker or secure authentication claims.
First measured extraction batch: four eight-second windows in approximately 0.147 seconds on the current RTX 3090, excluding network and storage. That is a short synthetic-input measurement, not a promised sustained rate. The live browser has priority between batches. One GPU is used for frozen encoder extraction; no duplicate ASR is loaded. Student training is sequential and capped at 10% of GPU memory. The live TTS model stays on the other GPU.
Storage plan: approximately 15 GB feature arrays, bounded source cache, at least 5 GiB disk reserve, no private microphone recordings or enrolled voiceprints. Source collection has a 14 GB network reservation budget and feature arrays a 22 GB maximum. Existing unrelated models and files are untouched. The current demo explicitly enables the completed V4/V5 combination at the user's request, together with diarization and full-utterance ECAPA checks; the historical small V2 pilot is not silently enabled.
Training results and full ablations (when available) ↗ · Earlier small pilot and its limitations ↗ · Reproducible demo source ↗
New study: short-state specialization, stable speaker targets, layer mixing and hard-example learning ↗. Separate report and live progress; original results above remain unchanged.
Completed context-specific optimization: 320 ms vs 2 seconds vs 3.04 seconds ↗. Forty-five matched training runs, three seeds, current-speaker onset slices and speaker-disjoint evaluation. The live demo uses completed V4 fast cues and the validation-selected V5 two-second reference; three seconds is selectable. Current integration, verification and limitations ↗.