Text Classification
Transformers
ONNX
Safetensors
bert
barge-in
backchannel
multilingual
experimental
text-embeddings-inference
Instructions to use HashNuke/barge-in-detector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use HashNuke/barge-in-detector with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="HashNuke/barge-in-detector")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("HashNuke/barge-in-detector") model = AutoModelForSequenceClassification.from_pretrained("HashNuke/barge-in-detector", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Akash Manohar commited on
Commit ·
a768911
1
Parent(s): e5336e8
Add a shared project introduction and model quick start
Browse filesLink source and dataset releases with common badges, demonstrate CPU inference, and retain evaluation and timing limitations.
README.md
CHANGED
|
@@ -7,54 +7,80 @@ base_model: intfloat/multilingual-e5-small
|
|
| 7 |
datasets: [HashNuke/barge-in-detector-data]
|
| 8 |
tags: [barge-in, backchannel, multilingual, experimental]
|
| 9 |
---
|
| 10 |
-
# Barge-in detector: experimental model release
|
| 11 |
|
| 12 |
-
|
| 13 |
-
backchannel (continue) or barge_in (yield). Exactly two labels; no audio,
|
| 14 |
-
preceding agent context or language hint is required. Answers after the agent
|
| 15 |
-
finishes are handled by the runtime. Keep the first 64 tokenizer tokens.
|
| 16 |
|
| 17 |
-
|
| 18 |
-
evaluation and benchmark code, tests, reports and labnotes live in the
|
| 19 |
-
[GitHub source](https://github.com/vxpipe/barge-in-detector).
|
| 20 |
-
[Dataset releases](https://huggingface.co/datasets/HashNuke/barge-in-detector-data)
|
| 21 |
-
are separate. release.json pins the exact code and dataset revisions and records
|
| 22 |
-
artifact hashes and original training provenance.
|
| 23 |
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
|
|
|
| 27 |
|
| 28 |
-
|
|
|
|
|
|
|
| 29 |
|
| 30 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
```python
|
| 33 |
import torch
|
| 34 |
-
from transformers import
|
| 35 |
|
| 36 |
repo = "HashNuke/barge-in-detector"
|
| 37 |
tokenizer = AutoTokenizer.from_pretrained(repo)
|
| 38 |
model = AutoModelForSequenceClassification.from_pretrained(repo).cpu().eval()
|
|
|
|
|
|
|
| 39 |
with torch.inference_mode():
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
```
|
| 45 |
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
|
| 51 |
-
|
|
|
|
|
|
|
| 52 |
|
| 53 |
-
|
| 54 |
-
2,640 synthetic train / 333 validation / 331 test; five epochs, batch 32,
|
| 55 |
-
weighted cross-entropy, AdamW, 3e-5 learning rate, 10% warm-up, seed 20261115,
|
| 56 |
-
bf16 on one RTX 5090. Epoch 5 selected on validation macro F1 (0.99324).
|
| 57 |
-
Training scripts and complete reports are in the pinned GitHub source.
|
| 58 |
|
| 59 |
| Evaluation | Accuracy | Missed interruptions | False interruptions |
|
| 60 |
| --- | --- | --- | --- |
|
|
@@ -63,17 +89,17 @@ Training scripts and complete reports are in the pinned GitHub source.
|
|
| 63 |
| Reviewed English challenge (98) | 73.5% | 54.2% | 0% |
|
| 64 |
| Challenge unseen in any synthetic split (93) | 72.0% | 54.2% | 0% |
|
| 65 |
|
| 66 |
-
The
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
all languages
|
| 73 |
|
| 74 |
-
## CPU
|
| 75 |
|
| 76 |
-
Four virtual AMD EPYC Genoa cores, PyTorch 2.15.0+cpu eager fp32,
|
| 77 |
batch one, 100 warm-ups and 1,000 measured calls per scenario:
|
| 78 |
|
| 79 |
| Warm text through label | p50 | p95 | p99 |
|
|
@@ -81,8 +107,40 @@ batch one, 100 warm-ups and 1,000 measured calls per scenario:
|
|
| 81 |
| Typical multilingual test input | 13.1 ms | 19.4 ms | 25.3 ms |
|
| 82 |
| Full 64-token budget | 26.8 ms | 33.3 ms | 37.9 ms |
|
| 83 |
|
| 84 |
-
|
| 85 |
-
seconds over ten runs.
|
| 86 |
-
be warm. Peak warm process RSS approximately 1,164 MiB.
|
| 87 |
-
and Vxpipe boundary are excluded. Reference-CPU
|
| 88 |
-
ONNX/int8 is unmeasured.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
datasets: [HashNuke/barge-in-detector-data]
|
| 8 |
tags: [barge-in, backchannel, multilingual, experimental]
|
| 9 |
---
|
|
|
|
| 10 |
|
| 11 |
+
# Barge-in detector
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
+
**Give acknowledgments room. Give interruptions the floor.**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
+
[](https://github.com/vxpipe/barge-in-detector)
|
| 16 |
+
[](https://huggingface.co/HashNuke/barge-in-detector)
|
| 17 |
+
[](https://huggingface.co/datasets/HashNuke/barge-in-detector-data)
|
| 18 |
+
[](https://github.com/vxpipe/barge-in-detector/blob/main/LICENSE)
|
| 19 |
|
| 20 |
+
Voice agents hear callers say “uh-huh”, “yes”, and “okay” while they are still
|
| 21 |
+
speaking. Those acknowledgments should let the agent continue. Corrections,
|
| 22 |
+
questions, and requests to stop should give the caller the floor.
|
| 23 |
|
| 24 |
+
Barge-in detector is an open-source, multilingual, text-only project exploring
|
| 25 |
+
that distinction with synthetic data and a learned classifier that runs on CPU.
|
| 26 |
+
|
| 27 |
+
This repository is the **model release**: weights, tokenizer, configuration,
|
| 28 |
+
licence, and provenance. Training, inference scripts, evaluation, benchmarks,
|
| 29 |
+
and labnotes are maintained in the [GitHub source](https://github.com/vxpipe/barge-in-detector).
|
| 30 |
+
|
| 31 |
+
> **Experimental checkpoint.** The first model misses 25 of 30 short
|
| 32 |
+
> interruptions in the reviewed English challenge. Use it for research and
|
| 33 |
+
> evaluation; its probabilities are not calibrated.
|
| 34 |
+
|
| 35 |
+
## Run the model
|
| 36 |
+
|
| 37 |
+
The CPU example below was tested on Linux with Python 3.14 and uv:
|
| 38 |
+
|
| 39 |
+
```shell
|
| 40 |
+
uv venv --python 3.14
|
| 41 |
+
uv pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/cpu
|
| 42 |
+
uv pip install transformers==5.23.0 huggingface-hub==2.4.0
|
| 43 |
+
source .venv/bin/activate
|
| 44 |
+
```
|
| 45 |
+
|
| 46 |
+
Run this in Python (`python`) or a notebook using that environment:
|
| 47 |
|
| 48 |
```python
|
| 49 |
import torch
|
| 50 |
+
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
| 51 |
|
| 52 |
repo = "HashNuke/barge-in-detector"
|
| 53 |
tokenizer = AutoTokenizer.from_pretrained(repo)
|
| 54 |
model = AutoModelForSequenceClassification.from_pretrained(repo).cpu().eval()
|
| 55 |
+
|
| 56 |
+
inputs = tokenizer("please stop for a moment", return_tensors="pt", truncation=True, max_length=64)
|
| 57 |
with torch.inference_mode():
|
| 58 |
+
logits = model(**inputs).logits
|
| 59 |
+
p_barge_in = logits.softmax(-1)[0, model.config.label2id["barge_in"]].item()
|
| 60 |
+
|
| 61 |
+
print({"p_barge_in": p_barge_in, "label": "barge_in" if p_barge_in >= 0.5 else "backchannel"})
|
| 62 |
```
|
| 63 |
|
| 64 |
+
Load once and reuse the model/tokenizer in a server. For a ready-to-run CLI and
|
| 65 |
+
training workflow, follow the [GitHub quick start](https://github.com/vxpipe/barge-in-detector#try-the-model-on-your-cpu).
|
| 66 |
+
Pin `revision=MODEL_COMMIT` when deploying a specific release.
|
| 67 |
+
|
| 68 |
+
## Input and output
|
| 69 |
+
|
| 70 |
+
| Contract | Value |
|
| 71 |
+
| --- | --- |
|
| 72 |
+
| Input | Caller text spoken over an ongoing agent turn |
|
| 73 |
+
| Cleaning | Lowercase, punctuation removed; native scripts may mix with English |
|
| 74 |
+
| Budget | First 64 tokenizer tokens |
|
| 75 |
+
| Labels | `0 = backchannel`, `1 = barge_in` |
|
| 76 |
+
| Output | `p_barge_in` in [0, 1]; current probabilities are uncalibrated |
|
| 77 |
+
| Context | No audio, preceding agent line, or language hint is required |
|
| 78 |
|
| 79 |
+
The runtime handles speech after the agent finishes and owns decision thresholds.
|
| 80 |
+
See the [source cleaner contract](https://github.com/vxpipe/barge-in-detector/blob/main/docs/transcript-format.md).
|
| 81 |
+
The 0.5 threshold in the example is diagnostic.
|
| 82 |
|
| 83 |
+
## Evaluation
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
|
| 85 |
| Evaluation | Accuracy | Missed interruptions | False interruptions |
|
| 86 |
| --- | --- | --- | --- |
|
|
|
|
| 89 |
| Reviewed English challenge (98) | 73.5% | 54.2% | 0% |
|
| 90 |
| Challenge unseen in any synthetic split (93) | 72.0% | 54.2% | 0% |
|
| 91 |
|
| 92 |
+
The short-interruption slice is 5/30 correct. The challenge was authored before
|
| 93 |
+
inference, then owner-reviewed and reconciled with authorization. Its final
|
| 94 |
+
binary labels match the frozen expectations. It includes repeated prefixes and
|
| 95 |
+
is now a development/regression set. Five challenge examples appear in synthetic
|
| 96 |
+
train/validation. These results are not a representative independent accuracy
|
| 97 |
+
estimate. Synthetic splits share criteria; template/semantic leakage remains
|
| 98 |
+
unaudited. Human-verified accuracy across all languages is not established.
|
| 99 |
|
| 100 |
+
## CPU performance
|
| 101 |
|
| 102 |
+
Four virtual AMD EPYC Genoa cores, four threads, PyTorch 2.15.0+cpu eager fp32,
|
| 103 |
batch one, 100 warm-ups and 1,000 measured calls per scenario:
|
| 104 |
|
| 105 |
| Warm text through label | p50 | p95 | p99 |
|
|
|
|
| 107 |
| Typical multilingual test input | 13.1 ms | 19.4 ms | 25.3 ms |
|
| 108 |
| Full 64-token budget | 26.8 ms | 33.3 ms | 37.9 ms |
|
| 109 |
|
| 110 |
+
A fresh Python process took **4.22 seconds median** to return its first label
|
| 111 |
+
(range 3.85–4.81 seconds over ten runs). That includes imports and local loading;
|
| 112 |
+
OS file cache may be warm. Peak warm process RSS was approximately 1,164 MiB.
|
| 113 |
+
HTTP, network, queueing, and Vxpipe boundary time are excluded. Reference-CPU
|
| 114 |
+
gates are unestablished; ONNX/int8 is unmeasured.
|
| 115 |
+
|
| 116 |
+
## Languages and provenance
|
| 117 |
+
|
| 118 |
+
The training/evaluation data spans English (US, India, Singapore), Hindi,
|
| 119 |
+
Kannada, Tamil, Telugu, Malayalam, Mandarin, Japanese, Indonesian, Filipino,
|
| 120 |
+
French, German, Spanish, and Italian. Results across these 14 languages are
|
| 121 |
+
synthetic; the reviewed challenge is English. See the
|
| 122 |
+
[dataset card](https://huggingface.co/datasets/HashNuke/barge-in-detector-data) for verification limits.
|
| 123 |
+
|
| 124 |
+
The checkpoint fine-tunes [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small)
|
| 125 |
+
with a two-label head. First run: 2,640 train / 333 validation / 331 test examples;
|
| 126 |
+
five epochs, batch 32, weighted cross-entropy, AdamW, learning rate 3e-5, 10% warm-up,
|
| 127 |
+
seed 20261115, bf16 on one RTX 5090. Epoch 5 was selected using validation macro F1
|
| 128 |
+
(0.99324). Released weights are fp32. See LICENSE and NOTICE for MIT attribution.
|
| 129 |
+
|
| 130 |
+
[release.json](release.json) pins the code and dataset revisions, artifact hashes,
|
| 131 |
+
original training-source manifest, and full training metrics. Reports and
|
| 132 |
+
labnotes live in the linked GitHub source. This repository contains no training
|
| 133 |
+
data or development scripts.
|
| 134 |
+
|
| 135 |
+
## One project, three repositories
|
| 136 |
+
|
| 137 |
+
| What you want to do | Start here |
|
| 138 |
+
| --- | --- |
|
| 139 |
+
| Generate data, train, evaluate, benchmark, or contribute | [GitHub source](https://github.com/vxpipe/barge-in-detector) |
|
| 140 |
+
| Download a released model and tokenizer | [Hugging Face model](https://huggingface.co/HashNuke/barge-in-detector) |
|
| 141 |
+
| Load a versioned dataset | [Hugging Face dataset](https://huggingface.co/datasets/HashNuke/barge-in-detector-data) |
|
| 142 |
+
|
| 143 |
+
GitHub owns all code, tests, docs, labnotes, and milestones, plus a dataset copy.
|
| 144 |
+
Hugging Face hosts model and dataset releases. Release metadata links artifacts
|
| 145 |
+
to exact source and dataset revisions. The project is available under the
|
| 146 |
+
[MIT licence](https://github.com/vxpipe/barge-in-detector/blob/main/LICENSE).
|