Smart Turn v3.2-Interview (8.0M Open-Weight Checkpoint)

Smart Turn v3.2-Interview is an open-weight 8,000,386-parameter audio-only end-of-utterance (EOU) / response-readiness detector released alongside Pikaia Interview EOU Benchmark: Evaluating Response Readiness in Bilingual Spoken Interviews.

It preserves the exact unmodified architecture, feature extractor, parameter count (8.0M), and drop-in ONNX Runtime interface of pipecat-ai/smart-turn-v3 (smart-turn-v3.2-gpu.onnx), fine-tuned for 1 epoch on 3,891 speaker-disjoint Mandarin and English spoken-interview decision points (262 distinct participants across 264 interviews).

Controlled Same-Architecture Evaluation (8.0M Audio-Only, Matched Released WAVs)

Holding the 8.0M Whisper-Tiny encoder + attention-pooling classifier completely fixed tests how far in-domain interview response_ready fine-tuning closes the compact-to-Speech-LLM gap (Holm–Bonferroni p_Holm controlled across the 8 primary comparisons):

Evaluation Split / Subset Upstream Smart Turn v3.2 (Zero-Shot) Smart Turn v3.2-Interview (dev-ranked seed29) 3-Seed Individual Mean ± SD (seeds 7, 17, 29) 3-Seed Ensemble
Tier 2 test 300 ms ROC-AUC (N=1,100) 0.8414 0.8989 (+0.0575, p=0.0010, p_Holm=0.0080) 0.8918 ± 0.0071 0.8976
Tier 2 test 300 ms R@5% (N=1,100) 41.09% 62.07% (+20.98 pp, p=0.0010, p_Holm=0.0080) 58.48% ± 3.25% 63.36%
Tier 2 Cascade R@5% (N=788) 41.43% 64.56% (+23.13 pp, p=0.0020, p_Holm=0.0080) 60.18% ± 3.84% 60.73%
Tier 2 Cascade Wait@5% (N=788) 713.0 ms 555.1 ms (-157.9 ms, p=0.0040, p_Holm=0.0120) 594.7 ± 35.4 ms 578.9 ms
Pre-Cutoff Strict Causal-243 AUC (N=243) 0.6930 0.8012 (+0.1082, p=0.0010, p_Holm=0.0080) 0.7593 ± 0.0393 0.7770
Dual-Human Agreed Gold AUC (N=42) 0.8707 0.9705 (+0.0998, nominal p=0.0325, p_Holm=0.0650) 0.9592 ± 0.0099 0.9637
External LiveKit eot-bench 300 ms AUC (N=1,058) 0.7998 0.8391 (+0.0394, p=0.0010, p_Holm=0.0080) 0.8425 ± 0.0054 0.8474
External LiveKit eot-bench Cascade Wait@5% 780.4 ms 734.5 ms (-45.9 ms, p=0.44, p_Holm=0.44) 761.2 ± 28.0 ms 766.2 ms

Repository Layout

  • smart-turn-v3.2-interview.onnx: Drop-in FP32 ONNX checkpoint (seed29, highest Tier 1 dev N=100 300 ms ROC-AUC of 0.8892 vs. 0.8844 [seed17] and 0.8738 [seed7] on exact released v1.0.0 WAVs). Directly compatible with Pipecat / ONNX Runtime Smart Turn v3 pipelines.
  • model.safetensors: Exact PyTorch safetensors state dict (79 tensors, 8,000,386 parameters) corresponding to smart-turn-v3.2-interview.onnx.
  • seeds/seed{7,17,29}/: All three random-seed checkpoints (.onnx and .safetensors) for exact reproduction of 3-seed mean ± SD and ensemble results.
  • config.json: Architecture metadata, provenance breakdown, and unified nextafter frozen Tier 1 dev thresholds.
  • inference.py: Minimal standalone ONNX Runtime scorer for 16 kHz mono WAV files.
  • SHA256SUMS: SHA256 digests for all repository artifacts.

Quickstart (Drop-In ONNX Runtime)

from huggingface_hub import hf_hub_download
from inference import score_onnx

onnx_path = hf_hub_download(
    repo_id="pikaia-ai/smart-turn-v3.2-interview",
    filename="smart-turn-v3.2-interview.onnx",
)
p_eot = score_onnx("utterance_300ms.wav", onnx_path=onnx_path)
print(f"Response-ready probability: {p_eot:.4f}")

Training Provenance, Selection History & Scope Boundaries

  • Base Checkpoint: Losslessly restored from pipecat-ai/smart-turn-v3 (smart-turn-v3.2-gpu.onnx, rev. f766f81d, SHA256 ab8dc64b...) with zero head reset and zero architectural changes.
  • Training Corpus (N = 3,891, 262 Distinct Participants across 264 Interviews; Private Audio): Fine-tuned for 1 epoch (487 updates; AdamW, lr=1e-5, weight_decay=0.01, gradient accumulation 8, gradient norm clip 1.0, within-language inverse-class-frequency binary cross-entropy) on 3,891 causally truncated bilingual spoken-interview decision points (2,757 Mandarin [2,047 EOT / 710 HOLD] + 1,134 English [773 EOT / 361 HOLD]). Raw training audio is retained privately under participant privacy governance; verified participant/interview overlap and raw-WAV / float32 PCM SHA256 overlap with all 2,338 Pikaia Interview EOU Benchmark v1.0.0 items and all 2,116 LiveKit eot-bench clips are 0.
  • Exact Four-Part Label Provenance (238 + 129 + 2,354 + 1,170 = 3,891): Comprises (i) 238 manually reviewed interview pauses, (ii) 129 legacy bilingual 3-LLM consensus pauses with strictly pre-cutoff causal re-judgments, (iii) 2,354 retained legacy bilingual 3-LLM consensus pauses where post-cutoff continuation was visible to the upstream judge (731 with non-empty continuation), and (iv) 1,170 Mandarin pauses retained under unanimous 3-LLM voting (deepseek-v4-1-flash, native-audio gemini-3.8-flash, gpt-6.1-terra) from 300 ms-truncated audio/transcripts with pause duration in judge metadata.
  • Selection Chronology: Predictions in the training batch were logged across 12 checkpoints (2 expansion variants [N=3,891 and N=4,070] × 3 seeds × {375 updates, 1 epoch}); because a pre-evaluation lock timestamp was not archived separately, corpus/epoch choice is documented retrospectively, while all 12 candidates consistently improve paired 600 ms Tier 2 AUC from 0.8482 to 0.8864–0.9087 (48.25%–60.07% R@5% vs. 42.76% upstream). Because fine-tuning jointly updates interview domain acoustics, class weighting, and response_ready labels at fixed 8.0M scale, it establishes substantial practical headroom (49.5% wait-gap closure) without isolating supervision semantics alone.

Attribution & Citation

Upstream architecture and pretrained initialization originate from pipecat-ai/smart-turn-v3 (Pipecat / Daily, BSD-2-Clause).

@misc{lin2026pikaiaeou,
  author        = {Lin, Xuanxuan},
  title         = {Pikaia Interview EOU Benchmark: Evaluating Response Readiness in Bilingual Spoken Interviews},
  year          = {2026},
  howpublished  = {arXiv preprint},
  url           = {https://huggingface.co/datasets/pikaia-ai/interview-eou-benchmark}
}
Downloads last month
43
Safetensors
Model size
8M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for pikaia-ai/smart-turn-v3.2-interview

Quantized
(10)
this model

Datasets used to train pikaia-ai/smart-turn-v3.2-interview