Text Classification
Safetensors
GGUF
English
end-of-turn
turn-taking
endpointing
barge-in
voice-agents
decision-model
eot-bench
interruptions
conversational
Instructions to use Pype/cue-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Pype/cue-v1 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Pype/cue-v1:Q8_0 # Run inference directly in the terminal: llama cli -hf Pype/cue-v1:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Pype/cue-v1:Q8_0 # Run inference directly in the terminal: llama cli -hf Pype/cue-v1:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Pype/cue-v1:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf Pype/cue-v1:Q8_0
Use Docker
docker model run hf.co/Pype/cue-v1:Q8_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use Pype/cue-v1 with Ollama:
ollama run hf.co/Pype/cue-v1:Q8_0
- Unsloth Desktop
- Pi
How to use Pype/cue-v1 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Pype/cue-v1:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Pype/cue-v1:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Pype/cue-v1 with Docker Model Runner:
docker model run hf.co/Pype/cue-v1:Q8_0
- Lemonade
How to use Pype/cue-v1 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Pype/cue-v1:Q8_0
Run and chat with the model
lemonade run user.cue-v1-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use Pype/cue-v1 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Pype/cue-v1:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default Pype/cue-v1:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Pype/cue-v1 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Pype/cue-v1:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "Pype/cue-v1:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download README.md from Pype/cue-v1: direct link, hf CLI and curl.
- Browser
- Download file 24.3 kB
-
https://huggingface.co/Pype/cue-v1/resolve/main/README.md
- Command line
-
hf download hf://Pype/cue-v1/README.md
-
curl -L -o README.md https://huggingface.co/Pype/cue-v1/resolve/main/README.md
24.3 kB
| license: apache-2.0 | |
| base_model: google/gemma-4-E4B-it | |
| language: | |
| - en | |
| tags: | |
| - end-of-turn | |
| - turn-taking | |
| - endpointing | |
| - barge-in | |
| - voice-agents | |
| - decision-model | |
| - gguf | |
| - eot-bench | |
| - interruptions | |
| pipeline_tag: text-classification | |
| # Cue v1 | |
| **[π» Code on GitHub](https://github.com/PYPE-AI-MAIN/cue-v1)** Β· **[π Full results and analysis](https://github.com/PYPE-AI-MAIN/cue-v1/blob/main/RESULTS.md)** Β· **[β Check our numbers](https://github.com/PYPE-AI-MAIN/cue-v1#check-our-numbers)** | |
| **Cue v1 is turn detection for voice agents that combines context and audio:** | |
| > **Cue v1 = the fine-tuned Gemma 4 E4B's logits (the words and context) Γ βSmart Turn v3.2 (the voice)** | |
| Combining the two gives better and faster results than either one alone. On LiveKit's eot-bench, with one model and one English prompt across 14 languages: | |
| **4th in English Β· 1st in Portuguese Β· about 13 decisions a second on one T4** | |
| Cue v1 works best with [Shortwire](#part-of-shortwire); we'll share more details soon. | |
| **Code, every benchmark prediction and a one-command check of our numbers:** [github.com/PYPE-AI-MAIN/cue-v1](https://github.com/PYPE-AI-MAIN/cue-v1) | |
| <video controls playsinline width="100%" src="https://huggingface.co/Pype/cue-v1/resolve/main/assets/cue-v1-thought-hold.mp4"></video> | |
| *Thought Hold on a real call: the caller pauses mid-sentence while giving their details, the end-of-turn decision | |
| answers MID_THOUGHT (0.68), and the agent holds instead of cutting in. Recorded in Shortwire with the base Gemma | |
| 4 E4B asking this same question, before the fine-tune; Cue v1's Gemma part is that decision, trained. The phone number is masked.* | |
| ## What it decides | |
| This repository holds Cue v1's Gemma part: Gemma 4 E4B, fine-tuned to answer two typed questions: *when the caller pauses, are they done, and if not, why not?* and *when the caller speaks over the agent, | |
| should the agent stop?* Each answer is read from the logits, as probabilities over named states instead of a single | |
| score, so the agent knows not only *whether* to act but *how long to wait* and *whether it was really interrupted*: | |
| | | Answer | What it means | What the agent does | | |
| |---|---|---|---| | |
| | **End of turn** (the caller pauses) | **DONE** | A complete question, answer, request or statement | Reply now | | |
| | | **LIKELY_DONE** | Probably finished, but it trails off | A short grace, then reply | | |
| | | **MID_THOUGHT** | The sentence is left hanging: a trailing "and", "so", "my number is", half a number | Wait: more is coming | | |
| | | **HOLDING** | Telling a story or explaining something: the pause is thinking | Wait longer, don't cut in | | |
| | **Interruptions** (the caller speaks while the agent talks) | **STOP** | A real interruption: a question, a correction, an objection, "wait", "hello?", or a full answer to the agent's question | Stop talking and listen | | |
| | | **CONTINUE** | Only listening: "hmm", "okay", "haan", "achha", laughter, noise, a fragment | Keep talking | | |
| ## Context and audio together give the best results | |
| The Gemma part brings the **context**: what the agent asked, what the caller has said so far, whether a sentence, a | |
| list or a phone number has been left hanging. It does not hear the voice. Smart Turn, an **audio turn model**, hears | |
| the voice (a falling tone, a breath) but not the meaning. Together they beat either one alone: | |
| | eot-bench, English | Latency @5% | Latency @10% | Cut-offs @300 ms | Cut-offs @600 ms | | |
| |---|---|---|---|---| | |
| | Smart Turn v3.2 only (audio) | 1051 ms | 739 ms | 35.2% | 14.8% | | |
| | Gemma part only (words and context) | 750 ms | 552 ms | 34.0% | 7.8% | | |
| | **Cue v1 (both)** | **719 ms** | **532 ms** | **26.8%** | **7.5%** | | |
| Across all 14 eot-bench languages, adding the voice to the Gemma part makes it faster in **12 of 14 languages** at | |
| both the 5% and 10% budgets. The combination is one line, `p_end Γ βp_voice` (see | |
| [Using the probabilities](#using-the-probabilities)). Smart Turn v3.2 is free (BSD-2), from Pipecat, and is not | |
| included here: download it from [pipecat-ai/smart-turn-v3](https://huggingface.co/pipecat-ai/smart-turn-v3). | |
| In the results below, **Cue v1** is always the combination; **Gemma part only** is the fine-tune on its own. | |
| ## How it decides: logits, not text | |
| Cue v1 never writes an answer. At every pause: | |
| 1. **Ask one question.** The decision prompt lists the options as letters, followed by what was said so far. | |
| - **End of turn:** A = DONE, B = LIKELY_DONE, C = MID_THOUGHT, D = HOLDING; with what the other side last said, | |
| what the speaker has said in this turn, and how the transcript ends. | |
| - **Interruptions:** A = STOP, B = CONTINUE; with what the agent is saying, the question it last asked, and what | |
| the caller just said over it. | |
| 2. **Let the model compute one token.** `max_tokens: 1`, with the top 20 log-probabilities returned | |
| (`logprobs: true`). llama.cpp returns log-probabilities, the normalised form of the logits; for choosing between | |
| options they are the same thing. | |
| 3. **Read the probability of each letter** and renormalise them so they sum to 1: | |
| ``` | |
| Agent asked: What's the best number to reach you on? | |
| Caller said: Sure, it's nine eight four five, | |
| MID_THOUGHT 0.999 DONE 0.001 LIKELY_DONE 0.000 HOLDING 0.000 | |
| ``` | |
| 4. **Turn the probabilities into an action**, as described in [Using the probabilities](#using-the-probabilities): | |
| - **Caller and agent:** p(DONE) + p(LIKELY_DONE) is the end-of-turn score, multiplied by the square root of an | |
| audio turn model's score. | |
| - **Two people talking:** the probabilities set how long to wait in silence. | |
| - **Interruptions:** p(STOP) decides whether the agent stops talking. | |
| Why logits instead of an answer in words or JSON: | |
| - **Typed by construction.** The answer is always one of the options. There is nothing to parse and nothing | |
| that can come back malformed. | |
| - **A full distribution, not a label.** On clear cases the answer is near-certain (0.999 above). On unclear ones | |
| the probability splits between options, and that split carries into how long to wait. | |
| - **Fast.** One forward pass and one token, about 100 to 300 ms on a T4. | |
| - **The fine-tune moves the logits, nothing else.** Training put the loss on the answer letter only, so the model | |
| learned where to put probability (the base Gemma almost never put it on HOLDING), while staying an ordinary | |
| Gemma 4 E4B that any llama.cpp, Ollama or vLLM server can host. | |
| ## Quick start (the Gemma part) | |
| Serve the GGUF with llama.cpp: | |
| ```bash | |
| llama-server -m cue-v1-q8_0.gguf -c 8192 -np 4 -ngl 999 --jinja --port 8090 | |
| ``` | |
| Ask it about a pause. The answer is the first token's probability mass on each option letter; nothing is generated | |
| beyond one token. | |
| ```python | |
| import json, math, httpx | |
| END_OF_TURN = json.load(open("prompts/end_of_turn_phone.json")) | |
| INTERRUPTION = json.load(open("prompts/barge_in.json")) | |
| LETTERS = "ABCDEFGH" | |
| LABELS = {"agent_question": "Agent asked", "caller_earlier": "Caller earlier", "caller_text": "Caller said", | |
| "agent_said": "Agent is saying"} | |
| def block(decision, inputs): | |
| return "\n".join(f"{LABELS.get(k, k)}: {inputs.get(k) or '(nothing)'}" for k in decision["inputs"]) | |
| def system(decision): | |
| ids = [o["id"] for o in decision["options"]] | |
| lines = [decision["instruction"], "", "Options:"] | |
| lines += [f"{LETTERS[i]}: {o['describe']}" for i, o in enumerate(decision["options"])] | |
| lines += ["", "Answer with exactly one letter."] | |
| if decision["examples"]: | |
| lines += ["", "Examples:"] | |
| for ex in decision["examples"]: | |
| lines += ["", block(decision, ex["inputs"]), f"Answer: {LETTERS[ids.index(ex['answer'])]}"] | |
| return "\n".join(lines) | |
| def ask(decision, inputs): | |
| r = httpx.post("http://localhost:8090/v1/chat/completions", json={ | |
| "messages": [{"role": "system", "content": system(decision)}, | |
| {"role": "user", "content": block(decision, inputs) + "\nAnswer:"}], | |
| "max_tokens": 1, "temperature": 0, "logprobs": True, "top_logprobs": 20, | |
| "reasoning_effort": "none", "chat_template_kwargs": {"enable_thinking": False}}).json() | |
| raw = {} | |
| for tp in r["choices"][0]["logprobs"]["content"][0]["top_logprobs"]: | |
| t = tp["token"].strip() | |
| if len(t) == 1 and t in LETTERS[:len(decision["options"])]: | |
| opt = decision["options"][LETTERS.index(t)]["id"] | |
| raw[opt] = raw.get(opt, 0.0) + math.exp(tp["logprob"]) | |
| total = sum(raw.values()) or 1.0 | |
| return {k: v / total for k, v in raw.items()} | |
| probs = ask(END_OF_TURN, {"agent_question": "What's the best number to reach you on?", | |
| "caller_earlier": "", "caller_text": "Sure, it's nine eight four five,", | |
| "transcript_end": "no sentence end"}) | |
| print(probs) # {'MID_THOUGHT': 0.999, 'DONE': 0.001, 'LIKELY_DONE': 0.0002, 'HOLDING': 0.0001}: half a number | |
| ``` | |
| ## Using the probabilities | |
| Ask again whenever the transcript changes, and re-check the score as the silence grows. Until the first words of the | |
| turn are transcribed, treat the end-of-turn score as 0.5 (or the wait as 1.0 s). | |
| ### Caller and agent (voice agents, phone calls) | |
| The words decide most; an audio turn model breaks ties with what the words can't carry, such as a falling tone. | |
| We use Smart Turn v3.2 (BSD-2), rescored as the pause goes on. | |
| ```python | |
| p_end = probs.get("DONE", 0) + probs.get("LIKELY_DONE", 0) | |
| score = p_end * p_voice ** 0.5 # p_voice: the audio model's end-of-turn probability for this moment | |
| if score >= THETA: | |
| reply() | |
| ``` | |
| THETA sets the trade between replying fast and cutting callers off. Choose it on your own calls for the false | |
| cut-off rate you can accept. This is the setup behind the eot-bench numbers below: at a 5% false cut-off rate, Cue v1 | |
| replies after 719 ms on average in English. | |
| ### Two people talking | |
| Here the answer sets how long to wait. Give each option a silence budget and blend by probability: | |
| ```python | |
| WAITS = {"DONE": 0.5, "LIKELY_DONE": 1.0, "MID_THOUGHT": 1.75, "HOLDING": 2.25} # seconds of silence | |
| wait = sum(p * WAITS[o] for o, p in probs.items()) | |
| if silence_so_far >= wait: | |
| take_the_turn() | |
| ``` | |
| A confident DONE ends the turn after half a second; a likely HOLDING waits over two seconds, so someone thinking | |
| mid-story isn't cut off. Here use the Gemma part on its own, without Smart Turn: in our two-person tests, adding | |
| Smart Turn's score made it slightly worse. | |
| ### Interruptions | |
| When the caller says something while the agent is talking, ask the interruption question with what the agent is | |
| saying, the question it last asked, and the caller's words so far: | |
| ```python | |
| probs = ask(INTERRUPTION, {"agent_said": "...so the appointment is on Tuesday at", "agent_question": "", | |
| "caller_text": "wait, Tuesday doesn't work"}) | |
| # {'STOP': 0.999, 'CONTINUE': 0.001}. The same question with "hmm" gives {'CONTINUE': 1.0, 'STOP': 0.0} | |
| if probs.get("STOP", 0) >= 0.5: | |
| stop_speaking() # a real interruption: listen | |
| # else: a backchannel ("hmm", "okay"), keep talking | |
| ``` | |
| Ask again as more of the caller's words arrive: a fragment can read like a backchannel until the rest of it comes in. | |
| The threshold trades stopping on noise against talking over a caller; 0.5 is a sensible start. | |
| ## Evaluation: eot-bench, English | |
| [LiveKit's eot-bench](https://livekit.com/benchmarks/eot-bench), all 400 English turns, scored with the harness's | |
| own metrics. Latency at a 5% and 10% false cut-off rate, and false cut-offs at a 300 ms and 600 ms latency budget. | |
|  | |
| | Model | Latency @5% | Latency @10% | Cut-offs @300 ms | Cut-offs @600 ms | | |
| |---|---|---|---|---| | |
| | LiveKit turn-detector v1 (hosted) | 543 ms | 295 ms | 9.9% | 4.5% | | |
| | Baton v1 | 577 ms | 350 ms | 12.3% | 4.8% | | |
| | Soniox stt-rt-preview | 647 ms | 512 ms | β | 5.5% | | |
| | **Cue v1** | **719 ms** | **532 ms** | **26.8%** | **7.5%** | | |
| | Cue v1, Gemma part only | 750 ms | 552 ms | 34.0% | 7.8% | | |
| | Deepgram flux-general-multi | 1151 ms | 548 ms | 12.9% | 9.9% | | |
| | ultraVAD | 899 ms | 663 ms | 27.7% | 11.9% | | |
| | LiveKit turn-detector v1-mini (local) | 1070 ms | 698 ms | 27.8% | 12.1% | | |
| | Smart Turn v3.2 | 1051 ms | 739 ms | 35.2% | 14.8% | | |
| Cue v1 is **4th on all four views** (AUC 0.961), behind LiveKit v1, Baton v1 and Soniox, and ahead of Deepgram Flux, | |
| ultraVAD, LiveKit's local model and Smart Turn v3.2 on its own. The Gemma part on its own (AUC 0.920) is also 4th at | |
| the 5% budget and on cut-offs at 600 ms. | |
| **Honest notes on this table:** | |
| - With the base Gemma in place of the fine-tune (same prompt, same Smart Turn), the result is 723 / 502 ms: in English | |
| the fine-tune is a tie with a well-prompted base model. Its gains show on holds (below) and on two-person conversation. | |
| - These latencies measure *when* Cue decides, not how fast the hardware is; see [Speed and cost](#speed-and-cost). | |
| ## 14 languages, one model, one English prompt | |
| Cue v1 and its Gemma part on all 14 eot-bench languages, with **one model and the same English prompt, | |
| English examples included**: no translation, no per-language examples, no per-language tuning. The fine-tune itself only | |
| ever saw English. The one language-aware step is automatic. Where most caller turns end without punctuation in the | |
| transcript (Arabic, Hindi, Indonesian, Japanese, Korean, Chinese), the "how the transcript ends" input is set to | |
| neutral, because a missing full stop means nothing there. | |
| Each cell is **Gemma part only β Cue v1**, with Cue v1's rank on the board in brackets. | |
| | Language | Latency @5% (ms) | Latency @10% (ms) | Cut-offs @300 ms | Cut-offs @600 ms | | |
| |---|---|---|---|---| | |
| | English | 750 β 719 (4th of 13) | 552 β 532 (4th) | 34.0% β 26.8% (4th) | 7.8% β 7.5% (4th) | | |
| | Portuguese | 800 β **681** (1st of 11) | 583 β 498 (3rd) | 27.4% β 27.7% (3rd) | 9.4% β **6.5%** (2nd) | | |
| | Spanish | 884 β **792** (2nd of 11) | 690 β 603 (3rd) | 40.2% β 29.9% (4th) | 13.5% β 10.1% (3rd) | | |
| | French | 738 β 647 (3rd of 11) | 524 β 449 (3rd) | 23.7% β 24.4% (4th) | 8.1% β **5.2%** (2nd) | | |
| | German | 802 β 786 (3rd of 11) | 548 β 559 (4th) | 22.7% β 22.7% (3rd) | 8.5% β 9.2% (4th) | | |
| | Italian | 924 β 912 (3rd of 11) | 620 β 595 (3rd) | 34.6% β 31.6% (3rd) | 10.4% β 9.9% (3rd) | | |
| | Dutch | 730 β 770 (3rd of 10) | 590 β 554 (4th) | 23.7% β 24.4% (3rd) | 9.6% β 9.4% (4th) | | |
| | Japanese | 567 β 557 (3rd of 10) | 374 β 406 (3rd) | 13.1% β 15.1% (3rd) | 3.9% β 4.6% (3rd) | | |
| | Korean | 593 β **608** (2nd of 9) | 462 β 406 (3rd) | 19.6% β 21.2% (3rd) | 4.7% β **5.5%** (2nd) | | |
| | Chinese | 862 β **838** (2nd of 9) | 645 β 637 (3rd) | 40.1% β 35.5% (3rd) | 11.8% β 11.7% (3rd) | | |
| | Indonesian | 881 β **871** (2nd of 9) | 673 β **598** (2nd) | 41.1% β **34.4%** (2nd) | 13.4% β **9.9%** (2nd) | | |
| | Hindi | 628 β 607 (4th of 10) | 513 β 494 (3rd) | 37.5% β 30.8% (4th) | 6.5% β 5.3% (4th) | | |
| | Turkish | 1015 β 1012 (3rd of 9) | 743 β 692 (3rd) | 41.3% β **33.9%** (2nd) | 14.9% β 11.6% (3rd) | | |
| | Arabic | 829 β 815 (6th of 9) | 685 β 640 (5th) | 51.5% β 39.9% (4th) | 14.8% β 12.4% (4th) | | |
| Ranks are among the models the benchmark reports for that language. Not every model covers every language. | |
| **Adding the voice helps in most languages:** Cue v1 is faster than its Gemma part alone in 12 of 14 languages at both | |
| budgets, and has fewer cut-offs at 600 ms in 11 of 14. In Dutch and Korean, the Gemma part alone is slightly faster at | |
| the 5% budget. | |
| **In Portuguese, Cue v1 is 1st at a 5% false cut-off rate**, ahead of LiveKit v1 and Soniox (both 702 ms). | |
| **Fine-tuning on English made the other languages better too.** Against Cue v1 built on the base Gemma (same prompt, | |
| same Smart Turn), the fine-tune is faster at the 5% budget in 11 of 14 languages and at the 10% budget in 11 of 14. The biggest moves: | |
| Hindi from 7th to 4th, Portuguese from 4th to 1st, French 698 to 647 ms, Korean 485 to 406 ms at 10%. It is slower | |
| in Dutch (688 to 770 ms at 5%), and at the 10% budget in English (502 to 532 ms) and Chinese (606 to 637 ms). | |
| The whole board, with Cue v1 in the outlined column. The small rank under each Cue value counts every model eot-bench | |
| reports for that language; the outlined cell in each row is the best. Grey cells are languages a model does not support. | |
|  | |
|  | |
| <details> | |
| <summary>The other two views: latency at 10% cut-offs, and cut-offs at a 600 ms budget</summary> | |
|  | |
|  | |
| </details> | |
| Across the 14 languages, counting the 5% and 10% budgets: | |
| | Against | Cue v1 is faster in | | |
| |---|---| | |
| | OpenAI GPT Realtime 2 (semantic VAD) | 28 of 28 | | |
| | VAP | 28 of 28 | | |
| | Gradium | 28 of 28 | | |
| | Smart Turn v3.2 | 26 of 28 | | |
| | ultraVAD | 25 of 28 | | |
| | LiveKit turn-detector v1-mini (local) | 25 of 28 | | |
| | Deepgram Flux | 12 of 18 | | |
| | Soniox | 12 of 28 | | |
| | LiveKit turn-detector v1 (hosted) | 1 of 28: still the one to beat | | |
| Arabic is the weak spot (6th at 5%). | |
| ## Speed and cost | |
| **The eot-bench latencies are not hardware speed.** They measure *when* Cue v1 decides that the caller is done, | |
| counted from the moment the caller actually stopped. They come out the same on any hardware, as long as each decision | |
| is ready in time: the benchmark assumes words arrive 0.5 s after they're spoken, covering the ASR and the decision. | |
| On slow hardware (a CPU-only machine, a busy laptop), a late decision makes the real-world wait longer than these | |
| numbers. | |
| **Turn detection runs on Gemma logits: one token, no generation.** Every decision is one forward pass that reads the | |
| option letters' probabilities, with the long prompt shared and cached across calls. Measured on one NVIDIA T4 | |
| (llama.cpp, q8_0, this repo's phone prompt, about 70 new tokens per question): | |
| | Questions in flight | Decisions per second | Time per decision (median / 95th percentile) | | |
| |---|---|---| | |
| | 1 | 12.5 | 79 / 96 ms | | |
| | 2 | 13.6 | 145 / 173 ms | | |
| | 4 | 10.5 | 364 / 419 ms | | |
| | 8 | 12.2 | 649 / 718 ms | | |
| One T4 serves about 13 decisions a second; beyond two at once, questions queue. On eot-bench, Cue asked about six | |
| questions per caller turn. At an assumed six caller turns a minute, that is roughly **20 concurrent calls per T4 with | |
| every decision under 200 ms**, or about $0.0005 per call-minute for turn detection on a T4 at around $0.58 an hour | |
| (an estimate: measure your own call pattern). Smart Turn v3.2, the audio part, is a small ONNX model that runs on the | |
| CPU. A faster GPU serves more. | |
| ## Training data: simulated conversations | |
| The fine-tune never saw a real recording or a benchmark transcript. | |
| - **310 conversations written by an LLM** to a written specification, each on a different topic: 150 phone calls | |
| between a caller and an agent, and 160 two-person conversations (casual, storytelling, instructional, | |
| argumentative, collaborative, transactional). | |
| - **About 10,600 labelled pauses**, plus 600 backchannels and 430 real interruptions for the barge-in decision. | |
| - **Speech textures written in:** fillers and false starts, repeated words, trailing conjunctions, unpunctuated | |
| endings, and numbers and addresses read out in pieces. A few Indian-English backchannels ("haan", "achha") appear, | |
| but the data is English. | |
| - **Shaped by a few aggregate statistics** from TurnBench's public dev split: the share of pauses that are holds | |
| (about 35%), the share of holds that follow a complete sentence (about 70%), and the share of turn ends without | |
| punctuation (about 25%). No dev audio, transcript or label was used as a training example. The TurnBench test split | |
| and otoSpeech were not used at all. | |
| - **The loss is on the answer letter only**, so the model learns where to put probability, not how to write. | |
| On held-out simulated conversations (10% of the conversations, never trained on): | |
| | | Base Gemma 4 E4B | Cue v1's Gemma part | | |
| |---|---|---| | |
| | Exact answer, every held-out question | 38.3% | **83.2%** | | |
| | End of turn: done vs not done | 58.2% | **86.9%** | | |
| | Interruptions: STOP vs CONTINUE (90 questions) | 93.3% | **98.9%** | | |
| The base model almost never answers HOLDING. That is the main thing the fine-tune teaches. | |
| ## Files | |
| - **Cue v1's Gemma part:** Gemma 4 E4B instruction-tuned, fine-tuned with a LoRA adapter (rank 16) using Unsloth, merged and | |
| quantised to q8_0 GGUF. | |
| - **Input:** text. What the other side last said, what the speaker has said so far in this turn, and how the | |
| transcript ends (a question mark, a full stop, or no sentence end). Audio is read by your ASR and, optionally, an | |
| audio turn model. | |
| - **Output:** a probability for each option, from one forward pass. No JSON, no generated text, no parsing. | |
| - **Prompts are part of the model.** The options, their descriptions and any worked examples live in | |
| `prompts/*.json`. Changing a description or adding examples in another language changes behaviour without | |
| retraining. | |
| | File | What | | |
| |---|---| | |
| | `cue-v1-q8_0.gguf` | Cue v1's Gemma part: the fine-tune merged and quantised to q8_0, for llama.cpp | | |
| | `adapter/` | The LoRA adapter (safetensors) for `google/gemma-4-E4B-it` | | |
| | `prompts/end_of_turn_phone.json` | End of turn, caller and agent on a phone call | | |
| | `prompts/end_of_turn_dialogue.json` | End of turn, two people talking | | |
| | `prompts/barge_in.json` | Interruptions: STOP or CONTINUE while the agent is speaking | | |
| | Code, benchmark predictions, verification | [github.com/PYPE-AI-MAIN/cue-v1](https://github.com/PYPE-AI-MAIN/cue-v1): the `cue` client, every eot-bench prediction, `verify.py`, `RESULTS.md` | | |
| | `assets/` | The English table, the 14-language heatmaps and the script that draws them; the Thought Hold demo clip | | |
| ## Intended use | |
| Turn-taking inside voice agents and other spoken-dialogue systems that already have an ASR: deciding when to | |
| reply, when to keep listening, and whether something the caller says over the agent should stop it. | |
| ## Limits | |
| - **The Gemma part reads words, not prosody.** On its own it cannot hear a falling tone or a breath; that is what | |
| Smart Turn adds in Cue v1. Using the Gemma part alone gives the slower "Gemma part only" numbers above. | |
| - **Short objections can pass as backchannels.** "Hmm, actually no" said over the agent comes back CONTINUE (0.94). | |
| Clear interruptions ("wait, Tuesday doesn't work") are STOP at 0.999, but a lone "no" or "actually" can be missed. | |
| - **Needs an ASR and a GPU.** It is not an on-device model. | |
| - **Trained on English only.** It carries over to 13 other languages with the English prompt, but every | |
| language except Portuguese still has a faster hosted model, and Arabic is behind most of the board. | |
| - **Trained on simulated conversations.** Real speech is messier than any written transcript. Measure it on your | |
| own calls before relying on its probabilities. | |
| ## Part of Shortwire | |
| Cue is the turn engine in **Shortwire**, Pype's open source voice AI stack. Shortwire splits a call in two: | |
| - **Background decision models handle how the call moves:** when the caller is done, when to hold, when to stop | |
| talking, what to guard against, when a number is being dictated, which tool to call. | |
| - **The foreground LLM focuses on the talking:** what the agent says. | |
| Every decision the cascade makes is auditable: each one is logged with its answer and probability, and can be | |
| replayed after the call. In Shortwire, MID_THOUGHT and HOLDING are what we call **Thought Hold**, and the record of | |
| every decision is the **Decision Trace**. | |
| *Knows when to wait. Knows when to stop.* | |
| **Cue v1 works best with Shortwire.** We'll share more details soon. | |
| ## Licence | |
| Apache 2.0, the same as the base model, Gemma 4 E4B. | |
| ## Citation | |
| ```bibtex | |
| @misc{pype2026cue, | |
| title = {Cue v1: turn detection from context and audio, with a fine-tuned decision model}, | |
| author = {Pype}, | |
| year = {2026}, | |
| url = {https://huggingface.co/Pype/cue-v1} | |
| } | |
| ``` | |