adventists-ai/Sophon-Realtime-v1

Sophon-Realtime v1

An open full-duplex voice model that hears your tone and speaks with one.
Adventists AI · Apache-2.0

GitHub · ModelScope · 中文

Highlights

• VoiceBench 82.8 and Big Bench Audio 81.8%: the strongest brain among open full-duplex models. #1 among open full-duplex models on both; level with GPT-4o-mini-Audio on VoiceBench.
• Paralinguistic understanding 48.6, #1 overall: it hears your tone. Ahead of Gemini 3.8 Flash (46.9) and Qwen3-Omni (44.6) on LISTEN, StressTest and MSPB. It hears emotion, emphasis, sarcasm, laughter and sighs straight from your voice.
• NVV-SuperBench 0.74 (Chinese) and 0.68 (English): it speaks with tone. The baseline without expression scores 0.37 / 0.27. The model decides from the conversation when to laugh, when to sigh, and when to clear a dry throat.
• Starts talking about 0.4 s after you stop: it's fast. It listens the whole time you talk and drafts its reply as you go. It picks up as soon as you stop, and when you cut in it stops and answers your new question.
• Full-Duplex-Bench v3 tool calling: Pass@1 48%, ahead of Grok (43%), and 100% response rate: it gets work done. Live mode lets it work together with a larger backend model, and it is compatible with the OpenAI GPT-Live interface format. Text and voice stream together, so every reply comes with an auditable transcript.
• Runs on a single GPU: yours to deploy. A quantized build runs on a single consumer GPU. Apache-2.0 code and weights, free for commercial use.

Benchmark details

Conversation: VoiceBench (official evaluators, gpt-4o-mini judge)

model	full-duplex	open weights	overall
GPT-4o-Audio	—	—	86.8
Sophon-Realtime v1	✅	✅	82.8
GPT-4o-mini-Audio	—	—	82.8
MiniCPM-o 4.5	✅	✅	76.1
MiniCPM-o 2.6	✅	✅	71.2
Freeze-Omni	✅	✅	55.2
NemotronLabs VoiceChat	✅	✅	55.1
PersonaPlex	✅	✅	30.6
Moshi	✅	✅	29.5

_{AlpacaEval uses the 199-question standard subset. Other scores come from the VoiceBench leaderboard and from Table 3 of the NemotronLabs VoiceChat paper (arXiv:2609.21967); the MiniCPM-o 4.5 score there is from an independent evaluation.}

Spoken reasoning: Big Bench Audio

1,000 spoken reasoning questions, graded on what the model actually says.

model	open weights	accuracy
GPT-Live-1	—	90%
Sophon-Realtime v1	✅	81.8%
Raon SpeechChat	✅	58%
Freeze-Omni	✅	33%
PersonaPlex	✅	19%
Moshi	✅	4%

_{Sophon: self-run, with spoken answers transcribed and judged by gpt-4.1. Others: Artificial Analysis leaderboard.}

Tool calling: Full-Duplex-Bench v3

Disfluent real human speech with multi-step tool calls. 100 tasks, judged by GPT-4o.

model	open weights	Pass@1 ↑	responds when it should ↑
GPT-Realtime	—	60%	96%
Gemini Live 3.1	—	54%	—
Sophon-Realtime	✅	48%	100%
cascaded pipeline	—	45%	—
Grok	—	43%	—
Ultravox v0.7	✅	41%	—

_{Sophon: best of several runs (internal baseline configuration). Others: official Full-Duplex-Bench v3 results.}

Hearing tone: paralinguistic understanding

Sophon-Realtime listens with DuplexJev. These benchmarks test emotion, sarcasm, stress and intonation.

model	LISTEN (4 tasks) + StressTest + MSPB, combined
DuplexJev (Sophon-Realtime's ears)	48.6
Gemini 3.8 Flash	46.9
Qwen3-Omni	44.6

Speaking with tone: NVV-SuperBench

Checks whether the laughs, coughs, sighs and breaths a reply calls for actually show up in the audio. Official scripts, 50 items per category, judged by Gemini 3 Flash; higher F1 is better.

model	Chinese	English
Sophon-Realtime v1	0.74	0.68
internal baseline without expression	0.37	0.27

How it works

• A vector language hidden inside the model. We found that the stream of continuous vectors a speech LLM builds while it listens is a complete language in its own right. A frozen LLM reads it as easily as text, and it carries both what was said and how it was said: intonation, emphasis and emotion. Sophon-Realtime v1's voice writes this vector language directly, then turns it back into sound. Hearing and speaking share one representation, so the tone the model can hear is the tone it can speak.
• Listens continuously. At each pause, the same model decides in a single token whether you've finished, are still talking, are back-channeling or are thinking.
• Drafts early, speaks on time. A reply is drafted from your first pause and released the moment the model decides it's its turn. If you keep talking, the draft is dropped before you hear it.
• Remembers what you actually heard. History keeps the real audio of both sides. When you interrupt, only the part you heard is kept.

Deployment

Runs on a single GPU; a quantized build runs on a single consumer GPU.

pip install sophon-realtime
hf download adventists-ai/Sophon-Realtime-v1 --local-dir Sophon-Realtime-v1
# launch commands: see deploy/ in the GitHub repository

The server is compatible with the OpenAI GPT-Live interface format, so existing realtime voice clients can connect directly.

License

Code and weights are released under Apache-2.0. Third-party components keep their own licences; see NOTICE.

Contact

Business enquiries: jiejin@adventists.ai