OpenYap Turn: Learning when to speak from real conversations

OpenYap Turn is a turn-taking model trained on conversations between friends and family. It ranks no. 1 for end of turn and no. 2 for interruption on TurnBench.

Originally published on theagenticdatacompany.com, where the figures are interactive.

Conversation is a complex social dance.¹ We begin learning it before we are born and practice it for the rest of our lives. By the time we are adults we have danced it so many thousands of times that we no longer notice the steps. We know what a good conversation feels like, and we feel it the instant someone misses a step.

With the people closest to us, talking takes no effort. Nobody hunts for the next thing to say when a pause opens up. We cut in, finish each other’s sentences, talk over each other and let silences sit, and the conversation carries itself, two people in sync. With a stranger it is different. We let the other person finish. We are polite and accommodating. We fill every gap with speech so that no silence turns awkward.

Most of the “natural” conversational speech data available today is the second kind: strangers, paired up and given a topic. We believe data like this cannot teach a model how a conversation is structured or how people actually talk to each other, because what a model most needs to learn is exactly what two strangers hold back.

To test this, we trained a turn-taking model only on real conversations between friends and family, from the corpus behind Open Yap 1K. That model, OpenYap Turn, ranks no. 1 of 24 entries on end-of-turn prediction and no. 2 of 23 on interruption prediction on Sesame’s TurnBench.²

Even so, we do not think small turn-taking models like this one are the answer, for reasons we come to at the end.

What is a turn-taking model?

A turn-taking model decides, moment by moment, whose turn it is to speak. A voice agent relies on it to know when to keep listening, when to answer, when to stay quiet and when it is being interrupted. Most agents today get this from a separate component built from one or more of these: a silence timer on top of voice activity detection, rules or a text model that judge from the transcript whether a sentence is finished, or a small audio model that listens to pitch, speech rate and other acoustic cues.

Speech-to-speech models such as OpenAI’s GPT-Live-1.1 and Google’s Gemini 3.8 Live have no separate turn-taking model bolted on. They learn when to speak the way they learn everything else, from their training data.

OpenYap Turn is a small audio model of 72.4 million parameters. It hears sound and timing, not words.

Figure 1. Four ways a voice agent decides when to speak. The first three run as a separate model beside the agent and listen for silence, for words or for sound. In a speech-to-speech model, turn-taking is learned along with everything else. OpenYap Turn is an audio model.

Why turn-taking is hard

Knowing when to speak, listen or cut in is one of the most basic skills in conversation. We react to what the other person says and to the cues they give. We also predict. Most of the time we know how their sentence will end before they finish it. We do both at once, with millisecond precision, and we learned it through years of practice.

Here are four moments from real conversations, in two pairs that a model must tell apart.

Figure 2. A pause, and an end of turn. Above, a speaker thinks aloud and pauses four times, for up to 1.34 s, without finishing. Below, a speaker finishes a question and the other answers 0.58 s later. Every pause is longer than that gap, so a silence timer patient enough to sit through the pauses would answer the question late.

Thinking aloud:

A question, and its answer:

Figure 3. A listener sound, and an interruption. Above, one speaker says “mhmm” during the other’s turn without trying to take it. Below, a listener starts while the other is still talking, keeps going, and takes the turn. Both begin the same way, as a second voice over the first. Excerpts from Open Yap 1K, CC BY 4.0, with machine transcripts; the marks are ours, not the model’s output.

A listener’s mhmm:

Cutting in to agree:

Has the speaker finished, or are they pausing to find a word? Is the listener’s “mhmm” agreement, or the start of a turn of their own? People settle these questions so fast that the gap between one turn and the next is typically about 200 milliseconds. That is quicker than anyone can plan a reply from scratch, so we start preparing ours while the other person is still talking.³ A voice agent that wants to feel natural has to decide at the same speed.

How OpenYap Turn works

OpenYap Turn listens to a conversation between two people, one speaker per channel, and predicts two kinds of moments as they happen. An end of turn is the moment a speaker has finished and it is the other person’s turn to speak. An interruption is a listener starting while the other is still talking and claiming the turn to speak.

The model works in three steps. A voice activity detector on each track marks when that speaker is making sound. One network forecasts, every 20 ms, how much each speaker will talk over the next two seconds. A few rules read that forecast and decide when a turn has ended and when someone has cut in.

Figure 4. The three steps, as an illustration. Each speaker’s audio arrives on its own track; the voice activity detector marks who is speaking; the network forecasts who will speak over the next two seconds; and the rules turn that forecast into a decision.

Everything the model learned is in that forecast. When it says the current speaker will stay silent over the coming stretch, the turn is ending. When it says the speaker will carry on while the listener stays quiet, the pause is only a pause and the turn is held. From the same activity, the network also learns how likely a listener is to take the turn and how likely a speaker is to hand it over.

Here is that forecast at one moment in a made-up conversation.

Figure 5. One moment in a made-up conversation, with illustrative values, not model output. Left of the line is what the model has heard; the words are there for you, as the model hears only the sound. A has stopped mid-sentence, and the model gives each way the next two seconds could go a probability. It expects A to carry on, so the pause is only a pause.

Training dataset

The network was trained on 1,984 hours of The Agentic Data Company’s corpus, 2,500 conversations in all:

• Conversations between people who know each other. Each speaker invites a friend or a family member, and the two of them talk.
• No topic, no prompt and no script. They talk about whatever comes up.
• Each speaker in their own place, on their own microphone, so every call arrives as two tracks with one voice on each.
• Self-supervised. The training target is the recording’s own speech activity over the next two seconds.
• 1,000 hours of the same corpus are public as Open Yap 1K, free for commercial and research use.

Training used no benchmark audio, no human labels and no transcripts. The benchmark’s public development set was used only to choose the operating point, the settings that decide how sure the model must be before it makes a prediction.

How it is scored

TurnBench tests a model on a held-out set of recorded conversations it has never heard, in which every moment a turn ends and every interruption has been marked by hand. The model listens to each conversation and predicts those moments as it goes. A prediction counts if it lands between a quarter of a second before a marked moment and three seconds after it.

Each task is scored on three numbers:

• Recall is the share of marked moments the model caught.
• Latency is how long after the marked moment the prediction came. Shorter is better.
• The false-positive rate is how often the model predicted a moment where there was none.

A good model catches nearly every moment, rarely predicts one that isn’t there, and is quick. The three pull against each other, because waiting longer makes a model surer but slower. TurnBench ranks entries by recall, among those under its false-positive limit.

Results

On TurnBench’s hidden test set, 116 conversations whose labels are not public, OpenYap Turn ranks no. 1 of 24 entries on end of turn. It caught 97.8% of the marked turn ends, with a median latency of 298 ms. On interruption it ranks no. 2 of 23, with a recall of 0.983, 0.001 behind the leader. Both results are within the benchmark’s false-positive limit.

End of turn

Interruption

Figure 6. The top five of each task on the TurnBench test set, as listed on 17 October 2026. Latency is the median.

The full leaderboard is on TurnBench’s site,⁴ under the entry OpenYap Turn from The Agentic Data Company.

Development set results

The operating point was chosen on all 38 conversations, so those figures are in-sample. The held-out estimate repeats the same choice on part of the set and scores it on the conversations it did not see, over 200 random splits.

Task	Measured on	Recall	False-positive rate	Median latency
End of turn	All 38 conversations	0.985	0.077	309 ms
End of turn	Held-out estimate	0.982–0.985	0.085–0.089	350–352 ms
Interruption	All 38 conversations	0.997	0.071	522 ms
Interruption	Held-out estimate	0.991	0.069–0.074	542–565 ms

More data, bigger network

If turn-taking is learned from conversation, more conversation should teach a model more. We tested that two ways, scoring each network on 28 hand-labelled conversations it never trained on.

First, more conversation. The same 18.4 M-parameter network was trained on five slices of the corpus, from 89 to 1,984 hours. With every slice up to about 1,000 hours it got better at telling an end of turn from a pause, the confusion in Figure 2. Doubling that to 1,984 hours, with the same length of training, added nothing.

Second, a bigger network. On the same 1,984 hours, a network four times larger, 72.4 M parameters, got better at telling a takeover from a listener sound, the confusion in Figure 3, where more conversation had stopped helping the small network past about 500 hours. A network ten times larger, 178.9 M parameters, did no better than 72.4 M.

Figure 7. More conversation: the 18.4 M-parameter network trained on five amounts of conversation. Bigger network: three network sizes trained on the same 1,984 hours; the blue point is the size OpenYap Turn uses. Both are scored on 28 hand-labelled conversations never trained on, as AUC: the chance that the model rates a real case above a look-alike, from 0.5, a coin toss, to 1, always.

Both curves climb and then flatten. The more real conversation the model heard, the more it learned, and a bigger network learned more from the same hours, but both gains stopped well short of perfect, at about 0.83 and 0.88. Past that point, neither more conversation nor more parameters helped a model that hears only sound and timing. We think what is left are the moments that turn on meaning, such as whether a question was asked or a story is finished, and that is where a small turn-taking model runs out.

Turn-taking models are a dead end

We do not think turn detectors, ours included, are the destination. Justin Uberti, who leads realtime AI at OpenAI, called turn detectors and any other low-parameter endpointing mechanism a dead end, and when asked what should do the job instead wrote:

“It needs to be done by a large model that has significant conversational context and strong priors about human interactions.”⁵

We share that view. A detector beside the agent sees only a sliver of the conversation, a few seconds of sound or the last thing that was said. It does not know that a question was asked, that a story is halfway through, or that this person tends to cut in and that one waits. The confusions in Figures 2 and 3, a pause that sounds like an ending and a “mhmm” that sounds like a takeover, are settled by meaning and by knowing how people behave. Only a model that holds the whole conversation, and has learned from a great many of them, can make the right call reliably.

So why build one? Because those priors have to be learned from somewhere, and a small detector is a clean test of whether a set of recordings holds them. OpenYap Turn was given no turn labels, no transcripts and no benchmark audio, only two-channel recordings of people talking with their friends and family. From those alone it learned to tell a pause from an ending well enough to rank no. 1 on TurnBench, and a listener sound from a takeover well enough to rank no. 2.

What the result shows is that real conversations between people who know each other carry the dynamics of conversation, turn-taking among them, and that a model can learn those dynamics from the recordings alone. We believe the models that truly understand human conversation will be full-duplex models trained on large corpora of real conversations, learning its dynamics the way each of us did, from thousands of them.

Limitations

Two channels, one voice each. The model reads two tracks, one speaker on each. A mixed recording of a room, a meeting or a phone line with both voices on one track is outside what it was built or measured for.

Sound only. It does not transcribe, does not know what was said, and does not identify speakers. A turn that ends on a word it would need to understand is called from timing and voice alone.

Latency is a chosen trade. On the test set the median wait is 602 ms for an interruption and 298 ms for an end of turn. We chose the operating points under a latency cap; uncapped, on the development set, the same model waited 871 and 475 ms for 0.26 percentage points more end-of-turn recall. Calling earlier means more false calls inside the same budget.

More than one prediction per moment. The benchmark counts a burst of predictions inside a moment’s window as one, and the submitted setting can make more than one prediction for a single moment. A product that needs exactly one decision per moment needs a stricter rule; that is our next step.

Tuned on the development set. The operating point was chosen on the benchmark’s 38 public development conversations. The results above come from its hidden test set, whose labels we have not seen.

One kind of recording. Training audio is calls between two people in separate locations on their own microphones. Meetings, in-person rooms, noisy environments and single-channel telephone audio were not measured.

Open Yap 1K

Everything above was learned from the corpus alone. 1,000 hours of it are public as Open Yap 1K, free for commercial and research use.

A sample is on Hugging Face at TheAgenticDataCompany/open-yap-1k. For the full dataset, request access or read about Open Yap 1K.

Notes

1. Sefik Emre Eskimez, Ankit Kumar, Françoise Beaufays, et al. TurnBench: measuring the timing of conversation. Sesame, 28 August 2026. www.sesame.com/journal/turnbench
2. Freeman Jiang, Ramon Sanabria, Soham Deshmukh, et al. TurnBench: A Multi-Domain Benchmark for Turn-Taking Dynamics in Spoken Dialogue. arXiv:2608.25218, 2026. arxiv.org/abs/2608.25218
3. Stephen C. Levinson and Francisco Torreira. Timing in turn-taking and its implications for processing models of language. Frontiers in Psychology 6:731, 2015. doi.org/10.3389/fpsyg.2015.00731
4. TurnBench leaderboard. turnbench.sesame.com
5. Justin Uberti. Post and reply on X, 13 October 2026. x.com/juberti