voxturn

An open audio turn-taking model that answers several questions about the moment in one forward pass: has the user finished their turn, how soon will they go on, is this a backchannel or an interruption, is the sound speech at all. It hears the last 8 seconds of user audio (optionally with a transcript or the dialog so far), runs on one GPU in ~8 ms, and is measured on all 14 languages of eot-bench.

▶ Interactive demo and film: replay real user turns in six languages and watch Pipecat's default turn model cut the user off while voxturn waits for the end of the turn.

English	11.5% false cut-offs at a 300 ms latency budget — ahead of the commercial Baton (12.3%) and Deepgram Flux (12.9%)
Multilingual	19.9% mean over 14 languages — the best published mean among open models (LiveKit v1-mini 28.5%, Smart Turn v3.2 39.8%)
One pass, many questions	4 questions in 9.0 ms vs 33.2 ms asked one by one (3.7×), answers identical up to rounding
Size	one 0.8B model; 8.3 ms per decision, 1.9 GB GPU memory
Drop-in for voice agents	turn analyzers for Pipecat (16.7% of turns cut off vs 45.3% with Smart Turn v3) and LiveKit Agents (9.3% vs 18.0% with LiveKit's local turn detector, at lower latency)

Quick start

pip install "transformers>=5.19" peft safetensors huggingface_hub scipy torch

from huggingface_hub import snapshot_download
import sys, soundfile as sf

path = snapshot_download("AustinXiao/voxturn-v1")
sys.path.insert(0, path)
from voxturn import VoxTurn

model = VoxTurn.from_pretrained(path)                 # also fetches Qwen/Qwen3-ASR-0.6B-hf at a pinned revision
audio, sr = sf.read("user_turn_so_far.wav")           # mono; the last 8 s are used
model.predict(audio, sample_rate=sr, questions=["eot", "resume", "invalid"], lang="en")
# illustrative output:
# {'eot':     {'yes': 0.93, 'no': 0.07},
#  'resume':  {'soon': 0.04, 'later': 0.05, 'no': 0.91},
#  'invalid': {'yes': 0.00, 'no': 1.00}}

Call it when your VAD sees the user stop (and again every 100–200 ms while the silence lasts); respond once eot.yes passes your threshold, and keep a timeout as a fallback. predict_batch scores many clips at once. Without a GPU, VoxTurn.from_pretrained(path, device="cpu") runs in fp32 (about 1 s per decision on two cores).

Use with Pipecat

pipecat_voxturn.py is a drop-in replacement for Pipecat's Smart Turn analyzer (Pipecat ≥ 1.12):

import sys
from huggingface_hub import snapshot_download
sys.path.insert(0, snapshot_download("AustinXiao/voxturn-v1"))
from pipecat_voxturn import VoxTurnAnalyzer

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import LLMContextAggregatorPair, LLMUserAggregatorParams
from pipecat.turns.user_stop import TurnAnalyzerUserTurnStopStrategy
from pipecat.turns.user_turn_strategies import UserTurnStrategies

aggregators = LLMContextAggregatorPair(LLMContext(), user_params=LLMUserAggregatorParams(
    vad_analyzer=SileroVADAnalyzer(),
    user_turn_strategies=UserTurnStrategies(stop=[TurnAnalyzerUserTurnStopStrategy(turn_analyzer=VoxTurnAnalyzer())]),
))

Pipecat asks a turn model once, when VAD reports the user stopped; on "not finished" it waits for its 3 s silence timeout. voxturn grows more confident as the silence goes on, so VoxTurnAnalyzer also asks again every 0.25 s while the user stays silent and ends the turn as soon as P(end of turn) reaches the threshold (VoxTurnParams(threshold=..., recheck_secs=...)).

150 eot-bench English turns streamed through Pipecat's Silero VAD (a turn counts as cut off if any of its pauses was):

Turn analyzer	Turns cut off mid-turn	Turn end after the final silence starts (median / mean / p90)
Smart Turn v3 (Pipecat default)	45.3%	0.14 / 1.32 / 3.12 s
voxturn, threshold 0.8 (default)	16.7%	0.30 / 0.36 / 0.84 s
voxturn, threshold 0.9	13.3%	0.38 / 0.67 / 1.60 s
voxturn, threshold 0.7	22.0%	0.26 / 0.30 / 0.61 s

Use with LiveKit Agents

livekit_voxturn.py is an audio turn detector for LiveKit Agents (≥ 1.8), used in place of LiveKit's TurnDetector:

import sys
from huggingface_hub import snapshot_download
sys.path.insert(0, snapshot_download("AustinXiao/voxturn-v1"))
from livekit_voxturn import VoxTurnDetector

from livekit.agents import AgentSession
from livekit.plugins import silero

session = AgentSession(
    vad=silero.VAD.load(min_silence_duration=0.3),
    turn_handling={"turn_detection": VoxTurnDetector()},
    # stt=..., llm=..., tts=...
)

The session streams the user's audio into the detector and asks it once the VAD has heard 200 ms of silence; below the threshold it waits max_delay (2.5 s) before ending the turn. When voxturn's first answer is below the threshold it asks again every 0.25 s and answers as soon as P(end of turn) reaches it (VoxTurnDetector(threshold=..., recheck_secs=...)). Create the detector once per worker process (e.g. in prewarm); its sessions share the model.

150 eot-bench English turns through LiveKit's Silero VAD, replaying AgentSession's endpointing (min_delay 0.3 s, max_delay 2.5 s):

Turn detector	VAD min_silence_duration	Turns cut off mid-turn	Turn end after the final silence starts (median / mean / p90)
LiveKit turn-detector-v1-mini (local)	0.55 s (default)	8.7%	0.58 / 0.98 / 2.50 s
voxturn	0.55 s (default)	6.7%	0.58 / 0.65 / 0.80 s
LiveKit turn-detector-v1-mini (local)	0.3 s	18.0%	0.32 / 0.73 / 2.50 s
voxturn (recommended)	0.3 s	9.3%	0.34 / 0.47 / 0.78 s
voxturn, threshold 0.9	0.3 s	6.7%	0.50 / 0.69 / 1.56 s

What it answers

Every question is a multiple-choice prompt; all of them share one prefix (audio, optional text, who is speaking) and are answered in the same forward pass.

Question	Options	Typical use
eot — has the user finished their turn?	yes / no	the main endpointing decision
resume — when will the user go on?	within half a second / later / not this turn	wait a little longer instead of answering
state4 — utterance state	complete / incomplete / backchannel / wait	Easy-Turn style dialog state
intent — while the agent speaks:	interrupt / backchannel	barge-in handling (best with text, see limits)
invalid — is this sound valid input?	yes / no	ignore coughs, laughter, noise, music

Inputs: audio only, audio + transcript, or text only (text=..., speaker="user"|"agent"). Prompts exist in English and Chinese (lang); the audio itself can be in any language.

How it works

Part
Backbone	Qwen3-ASR-0.6B (Apache-2.0)
Readout	no text is generated: each question is written out with its options, and a linear head scores the hidden state at each option's last token; softmax over the options
One pass	question blocks follow a shared prefix under a block-isolating attention mask, each at the positions it would have if asked alone — so one pass with k questions equals k separate passes
Files	voxturn-v1.safetensors (47 MB), voxturn.py (inference), pipecat_voxturn.py (Pipecat analyzer), livekit_voxturn.py (LiveKit turn detector), questions.json (prompts), config.json

Evaluation

eot-bench (validation, 100 ms minimum silence)

Metrics from eot-harness: false cut-offs at a latency budget (lower is better) and latency at a cut-off rate. eot-bench audio and labels were never used for training.

English

Model	Open	Size	Cut-offs @ 300 ms	@ 600 ms	Latency @ 5%	Latency @ 10%
LiveKit Turn Detector v1	no	—	9.9%	—	543 ms	—
p99lab turn-1	yes	~2.8B (two encoders)	10.6%	4.5%	553 ms	322 ms
voxturn v1	yes	0.8B	11.5%	6.0%	669 ms	335 ms
JoinIn AI Baton	no	—	12.3%	—	577 ms	—
Deepgram Flux	no	—	12.9%	—	1151 ms	—
LiveKit v1-mini	yes	—	27.8%	—	1070 ms	—
Smart Turn v3.2	yes	8M	35.2%	—	1051 ms	—

turn-1 numbers are from its model card; the others from the official eot-bench results.

All 14 languages — cut-offs @ 300 ms (%)

	ar	de	en	es	fr	hi	id	it	ja	ko	nl	pt	tr	zh	mean
LiveKit v1 (closed)	16.2	11.8	9.9	15.5	12.7	10.5	22.3	16.9	5.6	5.5	12.5	20.1	15.7	20.6	14.0
voxturn v1	21.1	15.4	11.5	21.4	13.4	37.5	30.8	21.0	11.1	12.8	15.9	24.1	20.1	22.9	19.9
LiveKit v1-mini	24.4	24.8	27.8	26.0	22.4	27.4	43.5	33.7	23.5	12.3	32.9	36.9	35.3	27.5	28.5
ultraVAD	45.5	35.5	27.7	41.6	34.4	39.1	45.0	40.4	12.0	33.8	42.3	42.8	44.9	38.8	37.4
Smart Turn v3.2	39.2	35.5	35.2	38.7	38.9	42.6	46.2	44.1	34.9	30.1	42.3	42.7	43.3	42.9	39.8

Among open models with multilingual results (LiveKit v1-mini, ultraVAD, Smart Turn v3.2), voxturn v1 is best in 12 of the 14 languages and has the best mean.

Other tests

Test set	Input	voxturn v1
SLR99 non-verbal sounds (coughs, laughter, sighs, …) → invalid	audio	99.6% recall (TurnSense: 90.4%)
Real speech wrongly judged invalid (Easy-Turn + Smart Turn clips, 1,786)	audio	0.06%
Easy-Turn test (zh), complete / incomplete / backchannel / wait	audio	88.3% overall

Speed (NVIDIA L40S, bf16, 8 s audio)

	p50	p95
1 question (eot)	8.3 ms	8.4 ms
4 questions in one pass	9.0 ms	9.1 ms
the same 4 questions as 4 passes	33.2 ms	33.3 ms
4 clips in one batch, 1 question each	16.0 ms	16.1 ms

End-to-end inference time through predict, including feature extraction (not end-of-turn response latency); GPU memory 1.9 GB. The default fast=True merges the adapter weights into the backbone and replays CUDA graphs captured on first use of each input shape (the very first call takes approximately 0.5 s, each new shape approximately 0.1 s). VoxTurn.from_pretrained(path, fast=False) runs the unmerged weights eagerly (approximately 38 ms) and reproduces the evaluation computation exactly.

Training details will be described in a technical report.

Licence

Weights and code: CC BY-NC 4.0 — several training sets (SpokenWOZ, Seamless Interaction, MagicData-RAMC) are non-commercial. The backbone, Qwen3-ASR-0.6B, is Apache-2.0 and is downloaded from Qwen's repository.

Citation

@misc{voxturn2026,
  title  = {voxturn: an open multilingual turn-taking model answering several questions in one pass},
  author = {Xiao, Yang},
  year   = {2026},
  url    = {https://huggingface.co/AustinXiao/voxturn}
}

When reporting eot-bench numbers, please also cite eot-bench.