Audio Classification
Transformers
Safetensors
qwen3_omni_moe
text-to-audio
audio
turn-taking
end-of-turn-detection
interruption-detection
lora
ms-swift
custom-inference
Instructions to use Beagledata/Turn-Detection-Jev with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Beagledata/Turn-Detection-Jev with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("audio-classification", model="Beagledata/Turn-Detection-Jev")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("Beagledata/Turn-Detection-Jev") model = AutoModelForMultimodalLM.from_pretrained("Beagledata/Turn-Detection-Jev", device_map="auto") - Notebooks
- Google Colab
- Kaggle
add readme
#1
by Beagledata - opened
- .gitattributes +5 -34
- EVALUATION.md +100 -0
- LICENSE +201 -0
- NOTICE +46 -0
- README.md +296 -0
- README_zh.md +296 -0
- calibration.json +12 -0
- image/01_metric_comparison.png +3 -0
- image/02_latency_comparison.png +3 -0
- image/效果对比.png +3 -0
- release_manifest.json +21 -0
.gitattributes
CHANGED
|
@@ -1,36 +1,7 @@
|
|
| 1 |
-
*.
|
| 2 |
-
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
-
*.
|
| 5 |
-
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
-
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
-
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
-
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
-
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
-
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
-
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
-
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
-
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
-
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
-
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
-
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
-
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
-
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
-
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
-
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
-
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
-
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
-
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
-
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
-
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
-
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
-
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
-
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
-
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
-
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
-
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 1 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
|
|
|
| 2 |
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.wav filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
image/01_metric_comparison.png filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
image/02_latency_comparison.png filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
image/效果对比.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
EVALUATION.md
ADDED
|
@@ -0,0 +1,100 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Turn-Detection-Jev:训练与评估详情
|
| 2 |
+
|
| 3 |
+
[返回模型主页](README.md)
|
| 4 |
+
|
| 5 |
+
开源发布名为 **Turn-Detection-Jev**。
|
| 6 |
+
|
| 7 |
+
## 训练数据与配置
|
| 8 |
+
|
| 9 |
+
该模型 共使用 **156,925 条**处理后的样本,来源为 Easy-Turn、Smart-Turn 和 MUSAN noise([OpenSLR 17](https://www.openslr.org/17/) 的 noise 子集)。
|
| 10 |
+
|
| 11 |
+
| 来源 | 纳入样本数 |
|
| 12 |
+
| --- | ---: |
|
| 13 |
+
| Easy-Turn | 115,995 |
|
| 14 |
+
| Smart-Turn | 40,000 |
|
| 15 |
+
| MUSAN noise | 930 |
|
| 16 |
+
| **合计** | **156,925** |
|
| 17 |
+
|
| 18 |
+
| 最终标签 | train | validation | calibration | test |
|
| 19 |
+
| --- | ---: | ---: | ---: | ---: |
|
| 20 |
+
| `listen_complete` | 45,000 | 2,500 | 1,250 | 1,250 |
|
| 21 |
+
| `listen_incomplete` | 22,500 | 1,250 | 625 | 625 |
|
| 22 |
+
| `listen_background` | 419 | 24 | 12 | 12 |
|
| 23 |
+
| `speak_interrupt` | 27,000 | 1,500 | 750 | 750 |
|
| 24 |
+
| `speak_backchannel` | 23,395 | 1,300 | 650 | 650 |
|
| 25 |
+
| `speak_incomplete` | 22,500 | 1,250 | 625 | 625 |
|
| 26 |
+
| `speak_background` | 418 | 23 | 11 | 11 |
|
| 27 |
+
| **合计** | **141,232** | **7,847** | **3,923** | **3,923** |
|
| 28 |
+
|
| 29 |
+
训练过程中将五种语义按照助手实际状态拆分为七小类。
|
| 30 |
+
|
| 31 |
+
| 训练参数 | 值 |
|
| 32 |
+
| --- | --- |
|
| 33 |
+
| LoRA rank / alpha / dropout | 32 / 64 / 0.05 |
|
| 34 |
+
| 学习率 / 训练预算 | 5e-5 / 3 epochs |
|
| 35 |
+
| 单卡 train / eval batch size | 4 / 4 |
|
| 36 |
+
| 梯度累积 / 有效训练 batch | 64 / 256(本次单卡训练) |
|
| 37 |
+
| 精度 / 最大输入 token 数 | bfloat16 / 4096 |
|
| 38 |
+
| 损失 | `audio_jev_ce_brier`,Brier 权重 1.0 |
|
| 39 |
+
| warmup ratio / weight decay | 0.03 / 0 |
|
| 40 |
+
| 随机种子 | 0 |
|
| 41 |
+
| 检查点选择 | 每 100 步验证,以最低验证 NLL 选择 best |
|
| 42 |
+
|
| 43 |
+
## 评估结果
|
| 44 |
+
|
| 45 |
+
### 公开测试集:完整评估范围
|
| 46 |
+
|
| 47 |
+
结果来自 2026-11-11 完整原生 Transformers 运行 `merged-20261111_051726-1672084`,prompt 为 `audio-jev-labels-v4`、评分 head 为 `lm_head`、温度 T=1,未启用 calibration。两套报告均为 `is_full_run=true`。该运行替换主页中的历史 ms-swift 成绩,历史原始记录仍保留在代码仓库 `results/reference/`。
|
| 48 |
+
|
| 49 |
+
代码仓库 `docs/public-benchmark.json` 保存完整/共同样本的精确指标、预测文件 SHA256 和来源路径;路径均为仓库相对路径。
|
| 50 |
+
|
| 51 |
+
| 数据集 | N | 正确数 | Accuracy | 状态标签 Macro-F1 | 语义 Macro-F1 |
|
| 52 |
+
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 53 |
+
| Easy-Turn | 800 | 789 | 98.625% | 98.83% | 99.00% |
|
| 54 |
+
| Smart-Test-4000 | 4,000 | 3,871 | 96.775% | 96.78% | 96.81% |
|
| 55 |
+
|
| 56 |
+
公开集宏平均只覆盖有真值样本的类别。语义映射将两种 incomplete 合并为 `incomplete`,两种 background 合并为 `invalid`,`speak_interrupt` 对应 Easy-Turn 原始 `wait`。零支持类别不代表已经获得完整评估。
|
| 57 |
+
|
| 58 |
+
| 状态标签 | Easy-Turn 正确 / N | Recall | Smart-Test 正确 / N | Recall |
|
| 59 |
+
| --- | ---: | ---: | ---: | ---: |
|
| 60 |
+
| `listen_complete` | 296 / 300 | 98.67% | 1,958 / 2,000 | 97.90% |
|
| 61 |
+
| `listen_incomplete` | 144 / 150 | 96.00% | 914 / 1,000 | 91.40% |
|
| 62 |
+
| `listen_background` | 0 / 0 | 未评估 | 0 / 0 | 未评估 |
|
| 63 |
+
| `speak_interrupt` | 100 / 100 | 100.00% | 0 / 0 | 未评估 |
|
| 64 |
+
| `speak_backchannel` | 100 / 100 | 100.00% | 0 / 0 | 未评估 |
|
| 65 |
+
| `speak_incomplete` | 149 / 150 | 99.33% | 999 / 1,000 | 99.90% |
|
| 66 |
+
| `speak_background` | 0 / 0 | 未评估 | 0 / 0 | 未评估 |
|
| 67 |
+
|
| 68 |
+
Easy-Turn 的 listening / speaking 准确率分别为 **97.78%(440/450)**、**99.71%(349/350)**;Smart-Test 分别为 **95.73%(2872/3000)**、**99.90%(999/1000)**。
|
| 69 |
+
|
| 70 |
+
### 共同样本上的模型参照
|
| 71 |
+
|
| 72 |
+
Easy-Turn 下表只取 complete/incomplete 的 **600 条**共同样本;Turn-Detection-Jev 为 **589/600**,与全量 800 条的 789/800 是不同范围。Smart-Test 使用固定 4,000 条子集。Turn-Detection-Jev 使用本次结果,其他模型保留历史结果。
|
| 73 |
+
|
| 74 |
+
| 数据集 | 模型 | N | 准确率 | 精准率(宏平均) | 召回率(宏平均) | F1(宏平均) |
|
| 75 |
+
| --- | --- | --- | --- | --- | --- | --- |
|
| 76 |
+
| easyturn | TurnSense 1.1 GPU | 600 | 96.17% | 96.19% | 96.17% | 96.17% |
|
| 77 |
+
| easyturn | Smart-Turn 3.2 CPU | 600 | 76.00% | 78.21% | 76.00% | 75.52% |
|
| 78 |
+
| easyturn | Smart-Turn 3.2 GPU | 600 | 77.17% | 78.26% | 77.17% | 76.94% |
|
| 79 |
+
| easyturn | Qwen3-Omni-30B-A3B-Instruct | 600 | 92.33% | 93.40% | 92.33% | 92.53% |
|
| 80 |
+
| easyturn | Turn-Detection-Jev | 600 | 98.17% | 98.33% | 98.17% | 98.25% |
|
| 81 |
+
| smart-test | TurnSense 1.1 GPU | 4000 | 26.28% | 75.97% | 26.28% | 38.86% |
|
| 82 |
+
| smart-test | Smart-Turn 3.2 CPU | 4000 | 92.50% | 92.59% | 92.50% | 92.50% |
|
| 83 |
+
| smart-test | Smart-Turn 3.2 GPU | 4000 | 93.88% | 93.89% | 93.88% | 93.87% |
|
| 84 |
+
| smart-test | Qwen3-Omni-30B-A3B-Instruct | 4000 | 69.40% | 80.43% | 69.40% | 66.35% |
|
| 85 |
+
| smart-test | Turn-Detection-Jev | 4000 | 96.78% | 96.87% | 96.78% | 96.81% |
|
| 86 |
+
|
| 87 |
+
Turn-Detection-Jev 获得了按标签规则构造的助手状态,其他音频模型未获得该状态,且服务和环境不同。因此表格只作描述性参照,不能据此将差值全部归因于微调或声称同等输入条件下的优势。
|
| 88 |
+
|
| 89 |
+
### 评估协议与耗时
|
| 90 |
+
|
| 91 |
+
公开集使用单声道、16 kHz、末尾最多 8 秒的 PCM16 WAV 音频;短音频保留实际长度。逐条串行预测,complete 分配到 listening,interrupt/backchannel 分配到 speaking;incomplete 按样本 ID 哈希分配一半到每个状态,状态种子为 4。助手状态为构造的场景元数据,未向麦克风音频混入助手播放声音。
|
| 92 |
+
|
| 93 |
+
| 完整公开集 | 平均逐样本耗时 | P50 | P90 | 串行 samples/s |
|
| 94 |
+
| --- | ---: | ---: | ---: | ---: |
|
| 95 |
+
| Easy-Turn(800) | 111.22 ms | 91.69 ms | 157.55 ms | 8.99 |
|
| 96 |
+
| Smart-Test(4,000) | 97.50 ms | 96.29 ms | 110.90 ms | 10.26 |
|
| 97 |
+
|
| 98 |
+
本次计时包含本地文件读取/解码、音频预处理和模型推理,不经 HTTP;未剔除首次调用开销。samples/s=样本数×1000/累计逐样本耗时,不是整轮墙钟吞吐。分位数在排序后的 (N−1)×q 位置线性插值。未测量独立 GPU kernel 耗时或真实在线打断延迟。
|
| 99 |
+
|
| 100 |
+
主页对比图中的 Easy-Turn 耗时仅对应 600 条共同样本,因此与上表 800 条的均值、分位数不同。其他模型的历史计时排除文件读取/解码,硬件和计时边界均不一致,不能作为同硬件速度排名。
|
LICENSE
ADDED
|
@@ -0,0 +1,201 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Apache License
|
| 2 |
+
Version 2.0, January 2004
|
| 3 |
+
http://www.apache.org/licenses/
|
| 4 |
+
|
| 5 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 6 |
+
|
| 7 |
+
1. Definitions.
|
| 8 |
+
|
| 9 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 10 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 11 |
+
|
| 12 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 13 |
+
the copyright owner that is granting the License.
|
| 14 |
+
|
| 15 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 16 |
+
other entities that control, are controlled by, or are under common
|
| 17 |
+
control with that entity. For the purposes of this definition,
|
| 18 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 19 |
+
direction or management of such entity, whether by contract or
|
| 20 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 21 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 22 |
+
|
| 23 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 24 |
+
exercising permissions granted by this License.
|
| 25 |
+
|
| 26 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 27 |
+
including but not limited to software source code, documentation
|
| 28 |
+
source, and configuration files.
|
| 29 |
+
|
| 30 |
+
"Object" form shall mean any form resulting from mechanical
|
| 31 |
+
transformation or translation of a Source form, including but
|
| 32 |
+
not limited to compiled object code, generated documentation,
|
| 33 |
+
and conversions to other media types.
|
| 34 |
+
|
| 35 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 36 |
+
Object form, made available under the License, as indicated by a
|
| 37 |
+
copyright notice that is included in or attached to the work
|
| 38 |
+
(an example is provided in the Appendix below).
|
| 39 |
+
|
| 40 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 41 |
+
form, that is based on (or derived from) the Work and for which the
|
| 42 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 43 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 44 |
+
of this License, Derivative Works shall not include works that remain
|
| 45 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 46 |
+
the Work and Derivative Works thereof.
|
| 47 |
+
|
| 48 |
+
"Contribution" shall mean any work of authorship, including
|
| 49 |
+
the original version of the Work and any modifications or additions
|
| 50 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 51 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 52 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 53 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 54 |
+
means any form of electronic, verbal, or written communication sent
|
| 55 |
+
to the Licensor or its representatives, including but not limited to
|
| 56 |
+
communication on electronic mailing lists, source code control systems,
|
| 57 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 58 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 59 |
+
excluding communication that is conspicuously marked or otherwise
|
| 60 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 61 |
+
|
| 62 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 63 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 64 |
+
subsequently incorporated within the Work.
|
| 65 |
+
|
| 66 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 67 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 68 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 69 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 70 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 71 |
+
Work and such Derivative Works in Source or Object form.
|
| 72 |
+
|
| 73 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 74 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 75 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 76 |
+
(except as stated in this section) patent license to make, have made,
|
| 77 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 78 |
+
where such license applies only to those patent claims licensable
|
| 79 |
+
by such Contributor that are necessarily infringed by their
|
| 80 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 81 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 82 |
+
institute patent litigation against any entity (including a
|
| 83 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 84 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 85 |
+
or contributory patent infringement, then any patent licenses
|
| 86 |
+
granted to You under this License for that Work shall terminate
|
| 87 |
+
as of the date such litigation is filed.
|
| 88 |
+
|
| 89 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 90 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 91 |
+
modifications, and in Source or Object form, provided that You
|
| 92 |
+
meet the following conditions:
|
| 93 |
+
|
| 94 |
+
(a) You must give any other recipients of the Work or
|
| 95 |
+
Derivative Works a copy of this License; and
|
| 96 |
+
|
| 97 |
+
(b) You must cause any modified files to carry prominent notices
|
| 98 |
+
stating that You changed the files; and
|
| 99 |
+
|
| 100 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 101 |
+
that You distribute, all copyright, patent, trademark, and
|
| 102 |
+
attribution notices from the Source form of the Work,
|
| 103 |
+
excluding those notices that do not pertain to any part of
|
| 104 |
+
the Derivative Works; and
|
| 105 |
+
|
| 106 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 107 |
+
distribution, then any Derivative Works that You distribute must
|
| 108 |
+
include a readable copy of the attribution notices contained
|
| 109 |
+
within such NOTICE file, excluding those notices that do not
|
| 110 |
+
pertain to any part of the Derivative Works, in at least one
|
| 111 |
+
of the following places: within a NOTICE text file distributed
|
| 112 |
+
as part of the Derivative Works; within the Source form or
|
| 113 |
+
documentation, if provided along with the Derivative Works; or,
|
| 114 |
+
within a display generated by the Derivative Works, if and
|
| 115 |
+
wherever such third-party notices normally appear. The contents
|
| 116 |
+
of the NOTICE file are for informational purposes only and
|
| 117 |
+
do not modify the License. You may add Your own attribution
|
| 118 |
+
notices within Derivative Works that You distribute, alongside
|
| 119 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 120 |
+
that such additional attribution notices cannot be construed
|
| 121 |
+
as modifying the License.
|
| 122 |
+
|
| 123 |
+
You may add Your own copyright statement to Your modifications and
|
| 124 |
+
may provide additional or different license terms and conditions
|
| 125 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 126 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 127 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 128 |
+
the conditions stated in this License.
|
| 129 |
+
|
| 130 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 131 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 132 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 133 |
+
this License, without any additional terms or conditions.
|
| 134 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 135 |
+
the terms of any separate license agreement you may have executed
|
| 136 |
+
with Licensor regarding such Contributions.
|
| 137 |
+
|
| 138 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 139 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 140 |
+
except as required for reasonable and customary use in describing the
|
| 141 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 142 |
+
|
| 143 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 144 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 145 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 146 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 147 |
+
implied, including, without limitation, any warranties or conditions
|
| 148 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 149 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 150 |
+
appropriateness of using or redistributing the Work and assume any
|
| 151 |
+
risks associated with Your exercise of permissions under this License.
|
| 152 |
+
|
| 153 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 154 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 155 |
+
unless required by applicable law (such as deliberate and grossly
|
| 156 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 157 |
+
liable to You for damages, including any direct, indirect, special,
|
| 158 |
+
incidental, or consequential damages of any character arising as a
|
| 159 |
+
result of this License or out of the use or inability to use the
|
| 160 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 161 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 162 |
+
other commercial damages or losses), even if such Contributor
|
| 163 |
+
has been advised of the possibility of such damages.
|
| 164 |
+
|
| 165 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 166 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 167 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 168 |
+
or other liability obligations and/or rights consistent with this
|
| 169 |
+
License. However, in accepting such obligations, You may act only
|
| 170 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 171 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 172 |
+
defend, and hold each Contributor harmless for any liability
|
| 173 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 174 |
+
of your accepting any such warranty or additional liability.
|
| 175 |
+
|
| 176 |
+
END OF TERMS AND CONDITIONS
|
| 177 |
+
|
| 178 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 179 |
+
|
| 180 |
+
To apply the Apache License to your work, attach the following
|
| 181 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 182 |
+
replaced with your own identifying information. (Don't include
|
| 183 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 184 |
+
comment syntax for the file format. We also recommend that a
|
| 185 |
+
file or class name and description of purpose be included on the
|
| 186 |
+
same "printed page" as the copyright notice for easier
|
| 187 |
+
identification within third-party archives.
|
| 188 |
+
|
| 189 |
+
Copyright [yyyy] [name of copyright owner]
|
| 190 |
+
|
| 191 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 192 |
+
you may not use this file except in compliance with the License.
|
| 193 |
+
You may obtain a copy of the License at
|
| 194 |
+
|
| 195 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 196 |
+
|
| 197 |
+
Unless required by applicable law or agreed to in writing, software
|
| 198 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 199 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 200 |
+
See the License for the specific language governing permissions and
|
| 201 |
+
limitations under the License.
|
NOTICE
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Turn-Detection-Jev
|
| 2 |
+
|
| 3 |
+
This release is a fine-tuned derivative of Qwen3-Omni-30B-A3B-Instruct,
|
| 4 |
+
developed and released by the Qwen team. Rights in the original model
|
| 5 |
+
remain with their respective holders.
|
| 6 |
+
|
| 7 |
+
Original model and upstream license declaration:
|
| 8 |
+
https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct
|
| 9 |
+
|
| 10 |
+
Official Qwen3-Omni source repository and Apache-2.0 license:
|
| 11 |
+
https://github.com/QwenLM/Qwen3-Omni
|
| 12 |
+
https://github.com/QwenLM/Qwen3-Omni/blob/main/LICENSE
|
| 13 |
+
|
| 14 |
+
Modification notice:
|
| 15 |
+
The base model was fine-tuned with LoRA for state-conditioned audio
|
| 16 |
+
turn completion, incomplete speech, interruption, backchannel and
|
| 17 |
+
background classification. Inference scores candidate answer tokens
|
| 18 |
+
through the language-model head. Historical experiment identifier:
|
| 19 |
+
Audio-Jev v3, checkpoint-1100, evaluated on 2026-11-02.
|
| 20 |
+
|
| 21 |
+
The fine-tuned weights in this release are distributed under the
|
| 22 |
+
Apache License, Version 2.0. See the accompanying LICENSE file.
|
| 23 |
+
Retain applicable original copyright and attribution notices when
|
| 24 |
+
redistributing base-model files or derived model files.
|
| 25 |
+
|
| 26 |
+
The Audio-Jev implementation uses ms-swift and includes work adapted
|
| 27 |
+
from OmniJev/OneJev (Apache-2.0):
|
| 28 |
+
https://github.com/modelscope/ms-swift
|
| 29 |
+
https://github.com/OmniJev/OneJev
|
| 30 |
+
If implementation code is distributed with the weights, retain its
|
| 31 |
+
applicable license and original notices as well.
|
| 32 |
+
|
| 33 |
+
Qwen and other project names are used only to identify the upstream
|
| 34 |
+
sources. This independent derivative release is not endorsed by the
|
| 35 |
+
upstream authors. Training datasets and third-party software retain
|
| 36 |
+
their respective licenses; this notice does not grant rights to
|
| 37 |
+
redistribute the original training audio.
|
| 38 |
+
|
| 39 |
+
Standalone packaging changes:
|
| 40 |
+
The frozen v3 prompt/schema/audio processing helpers were extracted from
|
| 41 |
+
ms-swift commit 96e8f8bdfe19b8ae3a2480797f09f69e4b33222b. Training-only
|
| 42 |
+
turn-policy code was omitted. The native Transformers/PEFT loader, CLI,
|
| 43 |
+
HTTP API, portable benchmark and release tooling were added for this
|
| 44 |
+
independent project. No ms-swift runtime dependency is required.
|
| 45 |
+
Dataset readers and the public-audio preprocessing profile were adapted
|
| 46 |
+
from the local audio-eval project. See SOURCES.json for file hashes.
|
README.md
ADDED
|
@@ -0,0 +1,296 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3-Omni-30B-A3B-Instruct
|
| 4 |
+
base_model_relation: finetune
|
| 5 |
+
library_name: transformers
|
| 6 |
+
pipeline_tag: audio-classification
|
| 7 |
+
tags:
|
| 8 |
+
- audio
|
| 9 |
+
- turn-taking
|
| 10 |
+
- end-of-turn-detection
|
| 11 |
+
- interruption-detection
|
| 12 |
+
- lora
|
| 13 |
+
- ms-swift
|
| 14 |
+
- custom-inference
|
| 15 |
+
datasets:
|
| 16 |
+
- ASLP-lab/Easy-Turn-Trainset
|
| 17 |
+
- pipecat-ai/smart-turn-data-v3.2-train
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# Turn-Detection-Jev
|
| 21 |
+
|
| 22 |
+
### Audio turn completion, interruptions, and backchannels
|
| 23 |
+
|
| 24 |
+
**Fine-tuned Qwen3-Omni-30B-A3B-Instruct · Five semantic actions / seven state labels · Apache-2.0**
|
| 25 |
+
|
| 26 |
+
[GitHub code](https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev/tree/main) · [Model weights](https://huggingface.co/YOUR_HF_ORG/Turn-Detection-Jev) · [Quick start](#quick-start) · [License](LICENSE)
|
| 27 |
+
|
| 28 |
+
**Language:** English | [中文](README_zh.md)
|
| 29 |
+
|
| 30 |
+
Turn-Detection-Jev predicts when a speech assistant should consider responding, waiting, or handling a user interruption. It takes audio and the application's current assistant playback state, then scores the labels available in that state. It is fine-tuned from [Qwen3-Omni-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct).
|
| 31 |
+
|
| 32 |
+
## Contents
|
| 33 |
+
|
| 34 |
+
- [Core features](#core-features)
|
| 35 |
+
- [Labels](#labels)
|
| 36 |
+
- [Benchmark results](#benchmark-results)
|
| 37 |
+
- [Metric comparison](#metric-comparison)
|
| 38 |
+
- [Client request latency](#client-request-latency)
|
| 39 |
+
- [Quick start](#quick-start)
|
| 40 |
+
- [Training and evaluation](#training-and-evaluation)
|
| 41 |
+
- [Limitations](#limitations)
|
| 42 |
+
- [License and acknowledgements](#license-and-acknowledgements)
|
| 43 |
+
- [Source code and feedback](#source-code-and-feedback)
|
| 44 |
+
|
| 45 |
+
## Core features
|
| 46 |
+
|
| 47 |
+
| Capability | Description |
|
| 48 |
+
| --- | --- |
|
| 49 |
+
| Turn completion | Distinguishes finished speech, incomplete speech, and background when the assistant is silent |
|
| 50 |
+
| Interruption and backchannel | Distinguishes completed interruptions, brief feedback, unfinished speech, and background during playback |
|
| 51 |
+
| State-conditioned scoring | Uses three listening labels or four speaking labels according to caller-supplied playback state |
|
| 52 |
+
| Single-forward prediction | Uses Qwen3-Omni-30B-A3B-Instruct audio understanding and reads candidate answer-token logits in one forward pass, without generating explanations |
|
| 53 |
+
| Application integration | The GitHub runtime provides CLI inference and an HTTP API returning labels and probabilities |
|
| 54 |
+
| Open weights | Apache-2.0, retaining attribution to the Qwen3-Omni-30B-A3B-Instruct base model |
|
| 55 |
+
|
| 56 |
+
## Labels
|
| 57 |
+
|
| 58 |
+
| Assistant playback | Output label | Meaning | Illustrative example (not an observed prediction) |
|
| 59 |
+
| --- | --- | --- | --- |
|
| 60 |
+
| Off | `listen_complete` | The user has finished and yielded the floor | “Check tomorrow's weather in Beijing.” |
|
| 61 |
+
| Off | `listen_incomplete` | The user is still speaking or holding the floor | “I'd like to ask about, um…” |
|
| 62 |
+
| Off | `listen_background` | Background unrelated to the target interaction | Ambient noise, bystanders, or playback echo |
|
| 63 |
+
| On | `speak_interrupt` | A completed request to stop, correct, or take the floor | “Stop a moment. I meant tomorrow.” |
|
| 64 |
+
| On | `speak_backchannel` | Finished brief feedback without requesting the floor | “Mm-hmm, right.” |
|
| 65 |
+
| On | `speak_incomplete` | The user has started but not finished an utterance | “Wait, what I wanted was…” |
|
| 66 |
+
| On | `speak_background` | Background unrelated to the target interaction | Ambient noise, bystanders, or playback echo |
|
| 67 |
+
|
| 68 |
+
The application must provide the real `assistant_speaking` state. The same words can express different intentions in context; these examples are illustrations, not measured model predictions.
|
| 69 |
+
|
| 70 |
+
## Benchmark results
|
| 71 |
+
|
| 72 |
+
### Public evaluation sets
|
| 73 |
+
|
| 74 |
+
Turn-Detection-Jev scores come from the complete native Transformers run `merged-20261111_051726-1672084` on 2026-11-11, using temperature T=1 with calibration disabled. Other models retain their historical results.
|
| 75 |
+
|
| 76 |
+
| Dataset | Samples | Correct | Accuracy | Semantic Macro-F1 |
|
| 77 |
+
| --- | ---: | ---: | ---: | ---: |
|
| 78 |
+
| Easy-Turn | 800 | 789 | **98.625%** | **99.00%** |
|
| 79 |
+
| Smart-Test-4000 | 4,000 | 3,871 | **96.775%** | **96.81%** |
|
| 80 |
+
|
| 81 |
+
**Evaluation conditions:** These are **state-conditioned single-window** results. Playback state was constructed using label rules and sample-ID hashes. Input audio was mono 16 kHz PCM16, retaining at most the final 8 seconds. Macro averages cover ground-truth-supported labels only. Easy-Turn covers five state labels and Smart-Test three; neither has background ground truth.
|
| 82 |
+
|
| 83 |
+
### Comparison on shared samples
|
| 84 |
+
|
| 85 |
+
The main comparison uses original semantic labels: listen_complete→complete; listen_incomplete/speak_incomplete→incomplete; speak_interrupt→wait; speak_backchannel→backchannel; both background labels→invalid. Only labels are merged; predictions are retained, and options are not renormalized based on ground truth.
|
| 86 |
+
|
| 87 |
+
## Metric comparison
|
| 88 |
+
|
| 89 |
+
| Dataset | Model | N | Accuracy | Macro precision | Macro recall | Macro-F1 |
|
| 90 |
+
| --- | --- | --- | --- | --- | --- | --- |
|
| 91 |
+
| easyturn | TurnSense 1.1 GPU | 600 | 96.17% | 96.19% | 96.17% | 96.17% |
|
| 92 |
+
| easyturn | Smart-Turn 3.2 CPU | 600 | 76.00% | 78.21% | 76.00% | 75.52% |
|
| 93 |
+
| easyturn | Smart-Turn 3.2 GPU | 600 | 77.17% | 78.26% | 77.17% | 76.94% |
|
| 94 |
+
| easyturn | Qwen3-Omni-30B-A3B-Instruct | 600 | 92.33% | 93.40% | 92.33% | 92.53% |
|
| 95 |
+
| easyturn | Turn-Detection-Jev | 600 | 98.17% | 98.33% | 98.17% | 98.25% |
|
| 96 |
+
| smart-test | TurnSense 1.1 GPU | 4000 | 26.28% | 75.97% | 26.28% | 38.86% |
|
| 97 |
+
| smart-test | Smart-Turn 3.2 CPU | 4000 | 92.50% | 92.59% | 92.50% | 92.50% |
|
| 98 |
+
| smart-test | Smart-Turn 3.2 GPU | 4000 | 93.88% | 93.89% | 93.88% | 93.87% |
|
| 99 |
+
| smart-test | Qwen3-Omni-30B-A3B-Instruct | 4000 | 69.40% | 80.43% | 69.40% | 66.35% |
|
| 100 |
+
| smart-test | Turn-Detection-Jev | 4000 | 96.78% | 96.87% | 96.78% | 96.81% |
|
| 101 |
+
|
| 102 |
+

|
| 103 |
+
|
| 104 |
+
Accuracy is computed over all shared samples. Precision, recall, and F1 are equally averaged over the two ground-truth target classes, complete and incomplete. Predictions outside those targets remain errors and are not renormalized.
|
| 105 |
+
|
| 106 |
+
Turn-Detection-Jev received constructed assistant-state metadata that the other audio models did not receive. Services and execution environments also differ. These figures compare the current Jev run with historical baselines; they are not a controlled comparison with identical information. The TurnSense figures here come from our evaluation, not its official model card.
|
| 107 |
+
|
| 108 |
+
### Client request latency
|
| 109 |
+
|
| 110 |
+
| Dataset | Model | N | Mean ms | P50 ms | P90 ms | Serial samples/s |
|
| 111 |
+
| --- | --- | --- | --- | --- | --- | --- |
|
| 112 |
+
| easyturn | TurnSense 1.1 GPU | 600 | 638.60 | 648.29 | 657.59 | 1.57 |
|
| 113 |
+
| easyturn | Smart-Turn 3.2 CPU | 600 | 64.91 | 47.66 | 54.51 | 15.41 |
|
| 114 |
+
| easyturn | Smart-Turn 3.2 GPU | 600 | 5.36 | 4.18 | 5.26 | 186.68 |
|
| 115 |
+
| easyturn | Qwen3-Omni-30B-A3B-Instruct | 600 | 222.09 | 226.87 | 247.68 | 4.50 |
|
| 116 |
+
| easyturn | Turn-Detection-Jev | 600 | 120.38 | 94.68 | 160.56 | 8.31 |
|
| 117 |
+
| smart-test | TurnSense 1.1 GPU | 4000 | 543.53 | 649.01 | 659.98 | 1.84 |
|
| 118 |
+
| smart-test | Smart-Turn 3.2 CPU | 4000 | 165.56 | 192.63 | 296.12 | 6.04 |
|
| 119 |
+
| smart-test | Smart-Turn 3.2 GPU | 4000 | 4.07 | 3.42 | 4.63 | 245.65 |
|
| 120 |
+
| smart-test | Qwen3-Omni-30B-A3B-Instruct | 4000 | 229.76 | 221.62 | 275.89 | 4.35 |
|
| 121 |
+
| smart-test | Turn-Detection-Jev | 4000 | 97.50 | 96.29 | 110.90 | 10.26 |
|
| 122 |
+
|
| 123 |
+

|
| 124 |
+
|
| 125 |
+
Turn-Detection-Jev latency comes from the native Transformers run above and includes local audio reading/decoding, preprocessing and inference. First-call overhead is retained; independent GPU kernel time was not measured. Other models retain historical timing.
|
| 126 |
+
|
| 127 |
+
The chart uses logarithmic axes. Historical ONNX runs used warmup=20 and remote runs warmup=0. ONNX timing includes mixing, resampling, features and inference; remote timing additionally includes encoding, HTTP, queueing and parsing. Both historical paths exclude file reading/decoding, whereas this Jev run includes it. Hardware, models and execution environments also differ, so this is not a controlled speed ranking.
|
| 128 |
+
|
| 129 |
+
Table samples/s equals N × 1000 / cumulative per-sample latency; it is neither whole-run wall-clock throughput nor concurrent capacity. Independent `gpu_latency_ms` and `inference_latency_ms` were not recorded in this run.
|
| 130 |
+
|
| 131 |
+
## Quick start
|
| 132 |
+
|
| 133 |
+
This Hugging Face repository contains model artifacts and documentation. Inference, serving, evaluation tools, and test-set instructions are maintained in the linked GitHub repository.
|
| 134 |
+
|
| 135 |
+
Copy the complete project to the server, or obtain its source first (Git does not include the test WAV files):
|
| 136 |
+
|
| 137 |
+
```bash
|
| 138 |
+
git clone https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev.git
|
| 139 |
+
cd Turn-Detection-Jev
|
| 140 |
+
git checkout main
|
| 141 |
+
```
|
| 142 |
+
|
| 143 |
+
### 1. Install dependencies
|
| 144 |
+
|
| 145 |
+
Activate an isolated Python >= 3.10 environment and install CUDA-enabled PyTorch suitable for the server first. From the project root, run:
|
| 146 |
+
|
| 147 |
+
```bash
|
| 148 |
+
bash scripts/01_install.sh
|
| 149 |
+
```
|
| 150 |
+
|
| 151 |
+
Alternatively, run the equivalent commands directly (choose either the script or these commands):
|
| 152 |
+
|
| 153 |
+
```bash
|
| 154 |
+
python -m pip install -e '.[datasets]'
|
| 155 |
+
```
|
| 156 |
+
|
| 157 |
+
This runs `python -m pip install -e '.[datasets]'` and prints the PyTorch version and CUDA availability. Dependencies are declared in `pyproject.toml`; the datasets extra installs PyArrow for raw Parquet conversion in step 2; merged weights do not require the PEFT/adapter extra. Confirm `CUDA available: True` before continuing. All three scripts use the current environment's `python`, falling back to `python3`, or an explicit `PYTHON=/absolute/path/to/python`. Use the same environment for all steps.
|
| 158 |
+
|
| 159 |
+
### 2. Convert and validate the original test sets
|
| 160 |
+
|
| 161 |
+
Before step 2, keep this layout in the code repository when using default paths:
|
| 162 |
+
|
| 163 |
+
```text
|
| 164 |
+
Turn-Detection-Jev/
|
| 165 |
+
├── datasets/
|
| 166 |
+
│ ├── README.md # Dataset instructions; recommended
|
| 167 |
+
│ ├── Easy-Turn-Testset/ # Complete original Easy-Turn test set
|
| 168 |
+
│ ├── smart-turn-data-v3.2-test/ # Complete original Smart-Turn v3.2 test set
|
| 169 |
+
│ ├── easyturn/ # Documentation/licenses copied during export
|
| 170 |
+
│ │ ├── LICENSE
|
| 171 |
+
│ │ ├── README.md
|
| 172 |
+
│ │ └── SOURCE_README.md
|
| 173 |
+
│ ├── smart-test-4000/ # Documentation/licenses copied during export
|
| 174 |
+
│ │ ├── LICENSE
|
| 175 |
+
│ │ ├── README.md
|
| 176 |
+
│ │ └── SOURCE_README.md
|
| 177 |
+
│ └── v3-public/ # Generated by step 2; do not create an empty directory
|
| 178 |
+
│ ├── easyturn/ # audio/, test.jsonl, summary.json, docs/licenses
|
| 179 |
+
│ └── smart-test-4000/ # audio/, test.jsonl, summary.json, docs/licenses
|
| 180 |
+
└── results/
|
| 181 |
+
└── reference/ # Frozen IDs, labels, states and audio hashes
|
| 182 |
+
├── easyturn.predictions.jsonl
|
| 183 |
+
└── smart-test-4000.predictions.jsonl
|
| 184 |
+
```
|
| 185 |
+
|
| 186 |
+
Keep the original upstream layouts and audio. See the code repository's `datasets/README.md` for required lists and Parquet shards. Old `audio/`, `test.jsonl` and `summary.json` files directly under `datasets/easyturn/` and `datasets/smart-test-4000/` are not inputs to the current steps 2 and 3 and do not need uploading; retain the three documentation/license files shown in each directory.
|
| 187 |
+
|
| 188 |
+
Step 3 reads the generated `datasets/v3-public/`; keep its audio, manifests and accompanying files together. Step 2 validates and reuses a complete existing export without requiring the original datasets. Rebuilding requires the originals, documentation/licenses and `results/reference/` again. Keep the remaining source code and configuration files as usual.
|
| 189 |
+
|
| 190 |
+
With the default layout above, run:
|
| 191 |
+
|
| 192 |
+
```bash
|
| 193 |
+
bash scripts/02_validate_data.sh
|
| 194 |
+
```
|
| 195 |
+
|
| 196 |
+
Use the original datasets on the evaluation server (paths may contain spaces):
|
| 197 |
+
|
| 198 |
+
```bash
|
| 199 |
+
EASYTURN_DIR=/path/to/Easy-Turn-Testset \
|
| 200 |
+
SMART_TEST_DIR=/path/to/smart-turn-data-v3.2-test \
|
| 201 |
+
bash scripts/02_validate_data.sh
|
| 202 |
+
```
|
| 203 |
+
|
| 204 |
+
Without these variables, sources default to `datasets/Easy-Turn-Testset` and `datasets/smart-turn-data-v3.2-test` inside this project. The default input is the full original Smart-Turn v3.2 test snapshot; conversion selects the frozen historical 4000 IDs. You can also point `SMART_TEST_DIR` to an existing 4000-row raw Parquet subset.
|
| 205 |
+
|
| 206 |
+
Step 2 converts to `datasets/v3-public/` and validates paths, SHA256 hashes, audio format and label/state consistency for **800** Easy-Turn and **4000** Smart-Test samples. Audio is mono 16 kHz PCM16 WAV, keeping at most the last 8 seconds. No model is loaded and sources are unchanged. Later runs fully validate and reuse existing output. Damaged or incomplete output causes an error; use a new `DATASETS_DIR` to rebuild.
|
| 207 |
+
|
| 208 |
+
Git excludes `datasets/**/audio/`. Convert from the originals on the server without uploading a local audio export. Step 2 does not download the originals automatically.
|
| 209 |
+
|
| 210 |
+
To change the output directory, set the same `DATASETS_DIR` for steps 2 and 3:
|
| 211 |
+
|
| 212 |
+
```bash
|
| 213 |
+
EASYTURN_DIR=/path/to/Easy-Turn-Testset \
|
| 214 |
+
SMART_TEST_DIR=/path/to/smart-turn-data-v3.2-test \
|
| 215 |
+
DATASETS_DIR=datasets/v3-public-new bash scripts/02_validate_data.sh
|
| 216 |
+
DATASETS_DIR=datasets/v3-public-new bash scripts/03_evaluate.sh
|
| 217 |
+
```
|
| 218 |
+
|
| 219 |
+
Equivalent direct commands (choose either these or the script):
|
| 220 |
+
|
| 221 |
+
```bash
|
| 222 |
+
python scripts/prepare_v3_testsets.py \
|
| 223 |
+
--easyturn /path/to/Easy-Turn-Testset \
|
| 224 |
+
--smart-test /path/to/smart-turn-data-v3.2-test \
|
| 225 |
+
--output datasets/v3-public --reuse-existing
|
| 226 |
+
python evaluate.py --manifest datasets/v3-public/easyturn/test.jsonl --validate-only
|
| 227 |
+
python evaluate.py --manifest datasets/v3-public/smart-test-4000/test.jsonl --validate-only
|
| 228 |
+
```
|
| 229 |
+
|
| 230 |
+
### 3. Evaluate merged weights
|
| 231 |
+
|
| 232 |
+
Place the full model in `models/Turn-Detection-Jev/`. Include `config.json`, all weight shards and their index, processor/tokenizer files and the original `scoring_head.json`. Do not leave an adapter's `adapter_config.json` in this directory: it selects the adapter loading path.
|
| 233 |
+
|
| 234 |
+
```bash
|
| 235 |
+
bash scripts/03_evaluate.sh
|
| 236 |
+
|
| 237 |
+
# Optional: specify a model directory and a new output directory.
|
| 238 |
+
# Relative paths are resolved from the project root.
|
| 239 |
+
bash scripts/03_evaluate.sh models/Turn-Detection-Jev results/runs/merged-check
|
| 240 |
+
```
|
| 241 |
+
|
| 242 |
+
Alternatively, run the equivalent commands directly (choose either the script or these commands):
|
| 243 |
+
|
| 244 |
+
```bash
|
| 245 |
+
RUN_OUTPUT="results/runs/merged-$(date +%Y%m%d_%H%M%S)-$$"
|
| 246 |
+
python evaluate.py --model models/Turn-Detection-Jev \
|
| 247 |
+
--device cuda:0 --dtype bfloat16 --attn-implementation sdpa \
|
| 248 |
+
--calibration hf/calibration.json \
|
| 249 |
+
--manifest datasets/v3-public/easyturn/test.jsonl --out "$RUN_OUTPUT/easyturn" \
|
| 250 |
+
--manifest datasets/v3-public/smart-test-4000/test.jsonl --out "$RUN_OUTPUT/smart-test-4000"
|
| 251 |
+
```
|
| 252 |
+
|
| 253 |
+
By default this reads the converted data from `datasets/v3-public/` validated in step 2. This starts one `evaluate.py` process, loads the model once and sequentially evaluates all 800 Easy-Turn and 4000 Smart-Test samples, without `--limit`. It uses native Transformers/PyTorch, one GPU (`cuda:0`), BF16, SDPA and temperature calibration from `hf/calibration.json`. No HTTP server, vLLM, separate base-model download or LoRA weights are needed. Select another GPU with `DEVICE=cuda:1 bash scripts/03_evaluate.sh`; the selected GPU must have enough memory for inference.
|
| 254 |
+
|
| 255 |
+
The script now loads `hf/calibration.json` by default: listening uses `choice:3` (T≈1.01754) and speaking uses `choice:4` (T≈1.37385). The report records `provenance.calibration_enabled=true`. To select another file:
|
| 256 |
+
|
| 257 |
+
```bash
|
| 258 |
+
CALIBRATION=/path/to/calibration.json bash scripts/03_evaluate.sh /path/to/model
|
| 259 |
+
```
|
| 260 |
+
|
| 261 |
+
The published 2026-11-11 scores above came from a run with calibration disabled and have not been replaced with a calibrated rerun. Use this command to match that T=1 setting:
|
| 262 |
+
|
| 263 |
+
```bash
|
| 264 |
+
CALIBRATION=none bash scripts/03_evaluate.sh /path/to/model
|
| 265 |
+
```
|
| 266 |
+
|
| 267 |
+
Temperature scaling adjusts probabilities and confidence. Positive temperature preserves label ordering mathematically and does not guarantee an improvement in Accuracy/F1.
|
| 268 |
+
|
| 269 |
+
Each run defaults to a new `results/runs/merged-TIMESTAMP-PID/` directory with per-dataset `predictions.jsonl` and `report.json` files. Inspect `classification.accuracy`, `classification.macro_f1`, `semantic.macro_f1` and `is_full_run=true`. The main benchmark table reports semantic Macro-F1. A specified output directory must not already exist; scripts stop on errors.
|
| 270 |
+
|
| 271 |
+
These are single-window results conditioned on constructed `assistant_speaking` state. Published Jev scores now use the complete native Transformers run above; historical ms-swift results remain in the code repository under `results/reference/`.
|
| 272 |
+
|
| 273 |
+
|
| 274 |
+
## Training and evaluation
|
| 275 |
+
|
| 276 |
+
The model was fine-tuned with LoRA rank 32, alpha 64, learning rate 5e-5, and a three-epoch budget. The processed pool contains 156,925 samples from Easy-Turn, Smart-Turn and MUSAN noise, with 141,232 assigned to training. On the separate internal test set of 3,923 samples, accuracy was 97.04% and seven-label Macro-F1 was 97.87%.
|
| 277 |
+
|
| 278 |
+
See [EVALUATION.md](EVALUATION.md) for data distributions, label recalls, experiment provenance and timing boundaries.
|
| 279 |
+
|
| 280 |
+
## Limitations
|
| 281 |
+
|
| 282 |
+
- Public sets do not assess background recall. Smart-Test's speaking group contains incomplete speech only, so its 99.90% accuracy does not establish interruption/backchannel discrimination.
|
| 283 |
+
- Performance is conditional on correct external playback state. These results do not establish streaming event latency or robustness to deployment devices, echo, noise and accents.
|
| 284 |
+
- This release requires a Qwen3-Omni-30B-A3B-Instruct GPU environment. CPU, ONNX and quantized deployments have not been benchmarked.
|
| 285 |
+
|
| 286 |
+
## License and acknowledgements
|
| 287 |
+
|
| 288 |
+
Fine-tuned weights are released under [Apache License 2.0](LICENSE), permitting no-charge research, personal and commercial use, modification and redistribution subject to its terms.
|
| 289 |
+
|
| 290 |
+
The base model was developed and released by the Qwen team: [Qwen3-Omni-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct). Its official model page declares Apache-2.0; the official license text is available in [Qwen3-Omni/LICENSE](https://github.com/QwenLM/Qwen3-Omni/blob/main/LICENSE). Base-model attribution and fine-tuning modifications are recorded in [NOTICE](NOTICE).
|
| 291 |
+
|
| 292 |
+
Thanks to Qwen, ms-swift, OmniJev/OneJev, Easy-Turn, Smart-Turn and MUSAN. Datasets and dependencies retain their own licenses. This independent derivative is not endorsed by the upstream authors.
|
| 293 |
+
|
| 294 |
+
## Source code and feedback
|
| 295 |
+
|
| 296 |
+
[GitHub source](https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev/tree/main) · [GitHub Issues](https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev/issues)
|
README_zh.md
ADDED
|
@@ -0,0 +1,296 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3-Omni-30B-A3B-Instruct
|
| 4 |
+
base_model_relation: finetune
|
| 5 |
+
library_name: transformers
|
| 6 |
+
pipeline_tag: audio-classification
|
| 7 |
+
tags:
|
| 8 |
+
- audio
|
| 9 |
+
- turn-taking
|
| 10 |
+
- end-of-turn-detection
|
| 11 |
+
- interruption-detection
|
| 12 |
+
- lora
|
| 13 |
+
- ms-swift
|
| 14 |
+
- custom-inference
|
| 15 |
+
datasets:
|
| 16 |
+
- ASLP-lab/Easy-Turn-Trainset
|
| 17 |
+
- pipecat-ai/smart-turn-data-v3.2-train
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# Turn-Detection-Jev
|
| 21 |
+
|
| 22 |
+
### 面向语音对话的轮次结束、打断与附和判断
|
| 23 |
+
|
| 24 |
+
**Qwen3-Omni-30B-A3B-Instruct 微调 · 五种语义 / 七个状态标签 · Apache-2.0**
|
| 25 |
+
|
| 26 |
+
[GitHub 源码](https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev/tree/main) · [模型权重](https://huggingface.co/YOUR_HF_ORG/Turn-Detection-Jev) · [快速开始](#快速开始) · [许可证](LICENSE)
|
| 27 |
+
|
| 28 |
+
**语言:** [English](README.md) | 中文
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
**Turn-Detection-Jev** 用于判断语音对话中何时接话、何时等待,以及助手播放过程中用户是否希望打断。模型同时接收音频与应用提供的助手播放状态,输出对应场景的类别及概率,可用于语音助手、客服和语音对话系统的轮次控制研究与集成。
|
| 32 |
+
|
| 33 |
+
Turn-Detection-Jev is a state-conditioned audio model for turn completion, interruptions, backchannels, incomplete speech, and background audio, fine-tuned from Qwen3-Omni-30B-A3B-Instruct.
|
| 34 |
+
|
| 35 |
+
## 目录
|
| 36 |
+
|
| 37 |
+
- [核心特性](#核心特性)
|
| 38 |
+
- [模型如何判断](#模型如何判断)
|
| 39 |
+
- [基准测试结果](#基准测试结果)
|
| 40 |
+
- [快速开始](#快速开始)
|
| 41 |
+
- [训练与评测说明](#训练与评测说明)
|
| 42 |
+
- [使用边界](#使用边界)
|
| 43 |
+
- [许可证与致谢](#许可证与致谢)
|
| 44 |
+
|
| 45 |
+
## 核心特性
|
| 46 |
+
|
| 47 |
+
| 能力 | 说明 |
|
| 48 |
+
| --- | --- |
|
| 49 |
+
| 轮次结束判断 | 在助手未播放时,区分用户说完、尚未说完和背景声音 |
|
| 50 |
+
| 打断与附和判断 | 在助手播放时,区分已完成的打断请求、简短附和、未完成表达和背景声音 |
|
| 51 |
+
| 状态条件决策 | 依据应用提供的 `assistant_speaking`,在三种或四种候选标签中评分 |
|
| 52 |
+
| 基于音频评分 | 使用 Qwen3-Omni-30B-A3B-Instruct 的音频理解能力,通过一次前向计算读取选项 logits,无需生成解释文本 |
|
| 53 |
+
| 可接入应用 | 通过 HTTP 返回预测类别与候选概率,供应用的轮次控制逻辑使用 |
|
| 54 |
+
| 免费开放使用 | 微调权重采用 Apache-2.0,保留 Qwen3-Omni-30B-A3B-Instruct 基座归属与许可声明 |
|
| 55 |
+
|
| 56 |
+
## 模型如何判断
|
| 57 |
+
|
| 58 |
+
模型包含 complete、incomplete、interrupt、backchannel、background 五种语义。结合助手是否正在播放,形成七个监督标签。
|
| 59 |
+
|
| 60 |
+
| 助手状态 | 输出标签 | 含义 | 场景示例(仅用于说明) |
|
| 61 |
+
| --- | --- | --- | --- |
|
| 62 |
+
| 未播放 | `listen_complete` | 用户完成表达,交出话轮 | “帮我查一下明天北京的天气。” |
|
| 63 |
+
| 未播放 | `listen_incomplete` | 用户尚未完成,仍持有话轮 | “我想问一下,那个……” |
|
| 64 |
+
| 未播放 | `listen_background` | 与当前交互无关的背景声音 | 环境噪声、旁人声音或播放回声 |
|
| 65 |
+
| 播放中 | `speak_interrupt` | 用户完成了停止、纠正或接管话轮的表达 | “先停一下,我说的是明天。” |
|
| 66 |
+
| 播放中 | `speak_backchannel` | 用户完成简短反馈,无意接管话轮 | “嗯,对。” |
|
| 67 |
+
| 播放中 | `speak_incomplete` | 用户已经开始表达,但尚未完成 | “等一下,我是想……” |
|
| 68 |
+
| 播放中 | `speak_background` | 与当前交互无关的背景声音 | 环境噪声、旁人声音或播放回声 |
|
| 69 |
+
|
| 70 |
+
`assistant_speaking` 应由应用的真实播放状态提供。同一句话在不同语境中可能有不同意图;上述例子不代表实测预测结果。
|
| 71 |
+
|
| 72 |
+
## 基准测试结果
|
| 73 |
+
|
| 74 |
+
### 公开测试集
|
| 75 |
+
|
| 76 |
+
Turn-Detection-Jev 成绩来自 2026-11-11 的完整原生 Transformers 评估。
|
| 77 |
+
|
| 78 |
+
| 数据集 | 样本数 | 正确数 | Accuracy | 语义 Macro-F1 |
|
| 79 |
+
| --- | ---: | ---: | ---: | ---: |
|
| 80 |
+
| Easy-Turn | 800 | 789 | **98.625%** | **99.00%** |
|
| 81 |
+
| Smart-Test-4000 | 4,000 | 3,871 | **96.775%** | **96.81%** |
|
| 82 |
+
|
| 83 |
+
**评估条件**:以上为已知助手状态的单窗口分类结果。状态由标签规则与样本 ID 构造,输入为单声道 16 kHz、末尾最多 8 秒音频;不代表纯音频分类或在线系统表现。Easy-Turn 覆盖五个状态标签,Smart-Test 覆盖三个,均没有 background 真值。语义 Macro-F1 只计有真值支持的类别。
|
| 84 |
+
|
| 85 |
+
### 共同样本上的模型参照
|
| 86 |
+
|
| 87 |
+
主比较统一使用原语义标签:listen_complete→complete,listen_incomplete/speak_incomplete→incomplete,speak_interrupt→wait,speak_backchannel→backchannel,两种 background→invalid。只合并标签,不删除预测或按真值重新选项归一化。
|
| 88 |
+
|
| 89 |
+
## 指标对比
|
| 90 |
+
|
| 91 |
+
| 数据集 | 模型 | N | 准确率 | 精准率(宏平均) | 召回率(宏平均) | F1(宏平均) |
|
| 92 |
+
| --- | --- | --- | --- | --- | --- | --- |
|
| 93 |
+
| easyturn | TurnSense 1.1 GPU | 600 | 96.17% | 96.19% | 96.17% | 96.17% |
|
| 94 |
+
| easyturn | Smart-Turn 3.2 CPU | 600 | 76.00% | 78.21% | 76.00% | 75.52% |
|
| 95 |
+
| easyturn | Smart-Turn 3.2 GPU | 600 | 77.17% | 78.26% | 77.17% | 76.94% |
|
| 96 |
+
| easyturn | Qwen3-Omni-30B-A3B-Instruct | 600 | 92.33% | 93.40% | 92.33% | 92.53% |
|
| 97 |
+
| easyturn | Turn-Detection-Jev | 600 | 98.17% | 98.33% | 98.17% | 98.25% |
|
| 98 |
+
| smart-test | TurnSense 1.1 GPU | 4000 | 26.28% | 75.97% | 26.28% | 38.86% |
|
| 99 |
+
| smart-test | Smart-Turn 3.2 CPU | 4000 | 92.50% | 92.59% | 92.50% | 92.50% |
|
| 100 |
+
| smart-test | Smart-Turn 3.2 GPU | 4000 | 93.88% | 93.89% | 93.88% | 93.87% |
|
| 101 |
+
| smart-test | Qwen3-Omni-30B-A3B-Instruct | 4000 | 69.40% | 80.43% | 69.40% | 66.35% |
|
| 102 |
+
| smart-test | Turn-Detection-Jev | 4000 | 96.78% | 96.87% | 96.78% | 96.81% |
|
| 103 |
+
|
| 104 |
+

|
| 105 |
+
|
| 106 |
+
准确率按全部共同样本计算;精准率、召回率、F1 均对 complete 和 incomplete 两个真实目标类等权平均。所有目标外预测仍计入错误,不重新归一化。
|
| 107 |
+
|
| 108 |
+
Turn-Detection-Jev 额外获得了构造的助手状态,其他音频模型未获得该输入;各服务和运行环境也不同。因此这些数值为本次 Jev 与其他模型历史实现的描述性参照,不能据此声称同等输入下的领先或将差值全部归因于微调。这里的 TurnSense 数据来自本项目的评估,不是 TurnSense 官方页面公布的成绩。
|
| 109 |
+
|
| 110 |
+
### 调用耗时
|
| 111 |
+
|
| 112 |
+
| 数据集 | 模型 | N | 均值 ms | P50 ms | P90 ms | 串行 samples/s |
|
| 113 |
+
| --- | --- | --- | --- | --- | --- | --- |
|
| 114 |
+
| easyturn | TurnSense 1.1 GPU | 600 | 638.60 | 648.29 | 657.59 | 1.57 |
|
| 115 |
+
| easyturn | Smart-Turn 3.2 CPU | 600 | 64.91 | 47.66 | 54.51 | 15.41 |
|
| 116 |
+
| easyturn | Smart-Turn 3.2 GPU | 600 | 5.36 | 4.18 | 5.26 | 186.68 |
|
| 117 |
+
| easyturn | Qwen3-Omni-30B-A3B-Instruct | 600 | 222.09 | 226.87 | 247.68 | 4.50 |
|
| 118 |
+
| easyturn | Turn-Detection-Jev | 600 | 120.38 | 94.68 | 160.56 | 8.31 |
|
| 119 |
+
| smart-test | TurnSense 1.1 GPU | 4000 | 543.53 | 649.01 | 659.98 | 1.84 |
|
| 120 |
+
| smart-test | Smart-Turn 3.2 CPU | 4000 | 165.56 | 192.63 | 296.12 | 6.04 |
|
| 121 |
+
| smart-test | Smart-Turn 3.2 GPU | 4000 | 4.07 | 3.42 | 4.63 | 245.65 |
|
| 122 |
+
| smart-test | Qwen3-Omni-30B-A3B-Instruct | 4000 | 229.76 | 221.62 | 275.89 | 4.35 |
|
| 123 |
+
| smart-test | Turn-Detection-Jev | 4000 | 97.50 | 96.29 | 110.90 | 10.26 |
|
| 124 |
+
|
| 125 |
+

|
| 126 |
+
|
| 127 |
+
Turn-Detection-Jev 耗时来自上述原生 Transformers 运行,含本地音频读取/解码、预处理和推理;未剔除首次调用开销,未测量独立 GPU kernel 耗时。其他模型仍为历史计时。
|
| 128 |
+
|
| 129 |
+
图表使用对数轴。其他模型的 ONNX 运行 warmup=20,历史远端运行 warmup=0;历史 ONNX 计时含混音、重采样、特征和推理,远端还含编码、HTTP、排队及响应解析,均不含文件读取/解码。Turn-Detection-Jev 本次计时包含读取/解码,且硬件、模型和运行环境不同,本图不能用于同硬件速度排名。
|
| 130 |
+
|
| 131 |
+
表中 samples/s=样本数×1000/累计逐样本耗时,不是整轮墙钟吞吐,也不表示并发容量。未在本次输出中记录独立 `gpu_latency_ms` 或 `inference_latency_ms`。
|
| 132 |
+
|
| 133 |
+
## 快速开始
|
| 134 |
+
|
| 135 |
+
本 Hugging Face 仓库提供模型权重、模型卡和配套评分/校准文件。推理、HTTP 服务、评估工具和测试集说明在 [GitHub 源码仓库](https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev/tree/main)。
|
| 136 |
+
|
| 137 |
+
在远程服务器复制完整项目,或先获取源码(代码仓库不包含测试 WAV):
|
| 138 |
+
|
| 139 |
+
```bash
|
| 140 |
+
git clone https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev.git
|
| 141 |
+
cd Turn-Detection-Jev
|
| 142 |
+
git checkout main
|
| 143 |
+
```
|
| 144 |
+
|
| 145 |
+
### 1. 安装依赖
|
| 146 |
+
|
| 147 |
+
先激活 Python >= 3.10 的独立环境,并按服务器 GPU/CUDA 环境安装相应的 CUDA 版 PyTorch。在项目根目录执行:
|
| 148 |
+
|
| 149 |
+
```bash
|
| 150 |
+
bash scripts/01_install.sh
|
| 151 |
+
```
|
| 152 |
+
|
| 153 |
+
也可直接执行以下等价命令(与脚本二选一):
|
| 154 |
+
|
| 155 |
+
```bash
|
| 156 |
+
python -m pip install -e '.[datasets]'
|
| 157 |
+
```
|
| 158 |
+
|
| 159 |
+
脚本执行 `python -m pip install -e '.[datasets]'`,随后显示 PyTorch 版本和 CUDA 是否可用。依赖由 `pyproject.toml` 管理;`datasets` 扩展安装原始 Parquet 转换所需的 PyArrow;完整合并模型不需要 PEFT/adapter 扩展。确认 `CUDA available: True` 后继续。三个脚本均使用当前环境的 `python`(不存在时用 `python3`),也可通过 `PYTHON=/绝对路径/python` 指定解释器;后续步骤应使用同一环境。
|
| 160 |
+
|
| 161 |
+
### 2. 转换原始数据并校验
|
| 162 |
+
|
| 163 |
+
按默认路径执行第二步前,项目中应保留以下文件和目录:
|
| 164 |
+
|
| 165 |
+
```text
|
| 166 |
+
Turn-Detection-Jev/
|
| 167 |
+
├── datasets/
|
| 168 |
+
│ ├── README.md # 数据说明,建议保留
|
| 169 |
+
│ ├── Easy-Turn-Testset/ # 完整原始 Easy-Turn 测试集
|
| 170 |
+
│ ├── smart-turn-data-v3.2-test/ # 完整原始 Smart-Turn v3.2 测试集
|
| 171 |
+
│ ├── easyturn/ # 转换时复制的说明和许可文件
|
| 172 |
+
│ │ ├── LICENSE
|
| 173 |
+
│ │ ├── README.md
|
| 174 |
+
│ │ └── SOURCE_README.md
|
| 175 |
+
│ ├── smart-test-4000/ # 转换时复制的说明和许可文件
|
| 176 |
+
│ │ ├── LICENSE
|
| 177 |
+
│ │ ├── README.md
|
| 178 |
+
│ │ └── SOURCE_README.md
|
| 179 |
+
│ └── v3-public/ # 第二步成功后自动生成,首次不要创建空目录
|
| 180 |
+
│ ├── easyturn/ # audio/、test.jsonl、summary.json 和说明/许可
|
| 181 |
+
│ └── smart-test-4000/ # audio/、test.jsonl、summary.json 和说明/许可
|
| 182 |
+
└── results/
|
| 183 |
+
└── reference/ # 固定样本 ID、标签、状态和音频哈希所需
|
| 184 |
+
├── easyturn.predictions.jsonl
|
| 185 |
+
└── smart-test-4000.predictions.jsonl
|
| 186 |
+
```
|
| 187 |
+
|
| 188 |
+
两个原始数据集须保留上游目录结构及音频内容;具体清单和 Parquet 分片要求见代码仓库的 `datasets/README.md`。顶层 `datasets/easyturn/`、`datasets/smart-test-4000/` 中的旧 `audio/`、`test.jsonl`、`summary.json` 不是当前第二、三步的输入,无需上传;上述各自的三个说明/许可文件仍需保留。
|
| 189 |
+
|
| 190 |
+
第三步默认只读取新生成的 `datasets/v3-public/`,需完整保留其中的音频、清单和配套文件。已有完整转换结果时,第二步校验并复用该目录,不再要求原始数据目录存在;重新转换时仍需原始数据、说明/许可文件和 `results/reference/`。其余项目源码和配置文件照常保留。
|
| 191 |
+
|
| 192 |
+
按上述默认目录放置后,直接执行:
|
| 193 |
+
|
| 194 |
+
```bash
|
| 195 |
+
bash scripts/02_validate_data.sh
|
| 196 |
+
```
|
| 197 |
+
|
| 198 |
+
服务器上已有原始测试集时,指定实际目录执行(路径可包含空格):
|
| 199 |
+
|
| 200 |
+
```bash
|
| 201 |
+
EASYTURN_DIR=/path/to/Easy-Turn-Testset \
|
| 202 |
+
SMART_TEST_DIR=/path/to/smart-turn-data-v3.2-test \
|
| 203 |
+
bash scripts/02_validate_data.sh
|
| 204 |
+
```
|
| 205 |
+
|
| 206 |
+
不设置环境变量时,默认从本项目的 `datasets/Easy-Turn-Testset` 和 `datasets/smart-turn-data-v3.2-test` 读取原始数据。默认读取完整的原始 Smart-Turn v3.2 测试快照,转换时仅选取冻结的历史 4000 个 ID;也可通过 `SMART_TEST_DIR` 指定已有的 4000 条原始 Parquet 子集。
|
| 207 |
+
|
| 208 |
+
第二步自动转换到 `datasets/v3-public/`,校验 Easy-Turn **800** 条和 Smart-Test **4000** 条的路径、SHA256、音频格式及标签/状态。音频转换为单声道 16 kHz、末尾最多 8 秒 PCM16 WAV,不加载模型、不修改原始数据。后续运行会完整校验并复用已有输出,无需再次转换;已有输出损坏或不完整时会报错,请用新的 `DATASETS_DIR` 重新转换。
|
| 209 |
+
|
| 210 |
+
Git 不包含 `datasets/**/audio/`;无需从本地上传导出的测试音频,使用服务器上的原始数据转换即可。第二步不会自动下载原始数据。
|
| 211 |
+
|
| 212 |
+
自定义输出目录时,第二步和第三步使用相同的 `DATASETS_DIR`:
|
| 213 |
+
|
| 214 |
+
```bash
|
| 215 |
+
EASYTURN_DIR=/path/to/Easy-Turn-Testset \
|
| 216 |
+
SMART_TEST_DIR=/path/to/smart-turn-data-v3.2-test \
|
| 217 |
+
DATASETS_DIR=datasets/v3-public-new bash scripts/02_validate_data.sh
|
| 218 |
+
DATASETS_DIR=datasets/v3-public-new bash scripts/03_evaluate.sh
|
| 219 |
+
```
|
| 220 |
+
|
| 221 |
+
也可直接执行以下等价命令(与脚本二选一):
|
| 222 |
+
|
| 223 |
+
```bash
|
| 224 |
+
python scripts/prepare_v3_testsets.py \
|
| 225 |
+
--easyturn /path/to/Easy-Turn-Testset \
|
| 226 |
+
--smart-test /path/to/smart-turn-data-v3.2-test \
|
| 227 |
+
--output datasets/v3-public --reuse-existing
|
| 228 |
+
python evaluate.py --manifest datasets/v3-public/easyturn/test.jsonl --validate-only
|
| 229 |
+
python evaluate.py --manifest datasets/v3-public/smart-test-4000/test.jsonl --validate-only
|
| 230 |
+
```
|
| 231 |
+
|
| 232 |
+
### 3. 评估完整合并模型
|
| 233 |
+
|
| 234 |
+
经 LoRA 合并后的完整模型(或从本 HF 仓库下载的完整模型)放入 `models/Turn-Detection-Jev/`,包含 `config.json`、全部权重分片及索引、processor/tokenizer 配套文件,并保留原始 `scoring_head.json`。不要混入 adapter 的 `adapter_config.json`,否则加载器会进入 adapter 分支。
|
| 235 |
+
|
| 236 |
+
```bash
|
| 237 |
+
bash scripts/03_evaluate.sh
|
| 238 |
+
|
| 239 |
+
# 可选:指定模型目录及一个尚不存在的结果目录(相对路径以项目根目录为准)。
|
| 240 |
+
bash scripts/03_evaluate.sh models/Turn-Detection-Jev results/runs/merged-check
|
| 241 |
+
```
|
| 242 |
+
|
| 243 |
+
也可直接执行以下等价命令(与脚本二选一):
|
| 244 |
+
|
| 245 |
+
```bash
|
| 246 |
+
RUN_OUTPUT="results/runs/merged-$(date +%Y%m%d_%H%M%S)-$$"
|
| 247 |
+
python evaluate.py --model models/Turn-Detection-Jev \
|
| 248 |
+
--device cuda:0 --dtype bfloat16 --attn-implementation sdpa \
|
| 249 |
+
--calibration hf/calibration.json \
|
| 250 |
+
--manifest datasets/v3-public/easyturn/test.jsonl --out "$RUN_OUTPUT/easyturn" \
|
| 251 |
+
--manifest datasets/v3-public/smart-test-4000/test.jsonl --out "$RUN_OUTPUT/smart-test-4000"
|
| 252 |
+
```
|
| 253 |
+
|
| 254 |
+
脚本默认读取第二步生成并校验的 `datasets/v3-public/`,在同一个 `evaluate.py` 进程中只加载一次模型,依次完整评估 Easy-Turn 800 条与 Smart-Test 4000 条,不设置 `--limit`。使用原生 Transformers/PyTorch、单卡 `cuda:0`、BF16、SDPA,默认加载 `hf/calibration.json` 进行温度校准,无需启动 HTTP 服务或 vLLM,也无需另行下载基座/LoRA 权重。可用 `DEVICE=cuda:1 bash scripts/03_evaluate.sh` 指定 GPU;该卡需要容纳完整推理所需显存。
|
| 255 |
+
|
| 256 |
+
脚本现默认加载 `hf/calibration.json`:listening 使用 `choice:3`(T≈1.01754),speaking 使用 `choice:4`(T≈1.37385)。报告记录 `provenance.calibration_enabled=true`。也可指定其他校准文件:
|
| 257 |
+
|
| 258 |
+
```bash
|
| 259 |
+
CALIBRATION=/path/to/calibration.json bash scripts/03_evaluate.sh /path/to/model
|
| 260 |
+
```
|
| 261 |
+
|
| 262 |
+
上述 2026-11-11 公布成绩来自未启用 calibration 的完整运行,尚未替换为校准后的重跑结果。若要使用相同的 T=1 设置,显式关闭校准:
|
| 263 |
+
|
| 264 |
+
```bash
|
| 265 |
+
CALIBRATION=none bash scripts/03_evaluate.sh /path/to/model
|
| 266 |
+
```
|
| 267 |
+
|
| 268 |
+
温度校准调整概率和置信度;正温度在数学上保持类别排序,不保证提高 Accuracy/F1。
|
| 269 |
+
|
| 270 |
+
默认结果保存到自动生成的 `results/runs/merged-时间戳-进程号/`,两个数据集各有 `predictions.jsonl` 和 `report.json`。查看 `classification.accuracy`、`classification.macro_f1`、`semantic.macro_f1`,并确认 `is_full_run=true`;主页主表使用语义 Macro-F1。指定的结果目录必须不存在,脚本遇到错误会停止。
|
| 271 |
+
|
| 272 |
+
以上是已构造 `assistant_speaking` 状态条件下的单窗口评估。主页 Jev 成绩已更新为上述原生 Transformers 完整运行;历史 ms-swift 成绩保留于代码仓库的 `results/reference/`。
|
| 273 |
+
|
| 274 |
+
## 训练与评测说明
|
| 275 |
+
|
| 276 |
+
模型基于 Qwen3-Omni-30B-A3B-Instruct,经 LoRA 微调:rank 32、alpha 64、学习率 5e-5、训练预算 3 epochs。数据池共 156,925 条,来自 Easy-Turn、Smart-Turn 和 MUSAN noise;实际训练划分为 141,232 条。内部 3923 条 test 的 Accuracy 为 97.04%,七类 Macro-F1 为 97.87%。
|
| 277 |
+
|
| 278 |
+
详细标签召回率、来源、实验版本和计时边界见 [EVALUATION.md](EVALUATION.md)。
|
| 279 |
+
|
| 280 |
+
## 使用边界
|
| 281 |
+
|
| 282 |
+
- 公开集无 background 真值;Smart-Test speaking 组只有 incomplete,不能验证打断和附和的完整区分能力。
|
| 283 |
+
- 性能依赖正确的外部播放状态;当前成绩不代表流式在线打断延迟或跨设备、噪声、回声下的可靠性。
|
| 284 |
+
- 需要可运行 Qwen3-Omni-30B-A3B-Instruct 的 GPU 环境;当前未提供 CPU、ONNX 或量化后端的验证结果。
|
| 285 |
+
|
| 286 |
+
## 许可证与致谢
|
| 287 |
+
|
| 288 |
+
微调权重采用 [Apache License 2.0](LICENSE),可在遵守条款的前提下免费用于研究、个人和商业应用,以及修改和再分发。
|
| 289 |
+
|
| 290 |
+
基座由 Qwen 团队发布:[Qwen3-Omni-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct) 的官方模型页标识为 Apache-2.0,许可全文参见 [Qwen3-Omni 官方 LICENSE](https://github.com/QwenLM/Qwen3-Omni/blob/main/LICENSE)。本发布保留基座归属与微调修改说明,见 [NOTICE](NOTICE)。
|
| 291 |
+
|
| 292 |
+
感谢 Qwen、ms-swift、OmniJev/OneJev、Easy-Turn、Smart-Turn 和 MUSAN。各数据集和依赖保留其自身许可;独立衍生模型不代表上游作者背书。
|
| 293 |
+
|
| 294 |
+
## 源码与反馈
|
| 295 |
+
|
| 296 |
+
推理代码、测试集准备与评估:[GitHub](https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev/tree/main)。问题反馈:[GitHub Issues](https://github.com/YOUR_GITHUB_ORG/Turn-Detection-Jev/issues)。
|
calibration.json
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"temperatures": {
|
| 3 |
+
"choice:3": 1.0175427143279028,
|
| 4 |
+
"choice:4": 1.3738458063981829
|
| 5 |
+
},
|
| 6 |
+
"source": "historical v3 calibration split; 3923 samples",
|
| 7 |
+
"group_counts": {
|
| 8 |
+
"choice:3": 1887,
|
| 9 |
+
"choice:4": 2036
|
| 10 |
+
},
|
| 11 |
+
"prompt_version": "audio-jev-labels-v4"
|
| 12 |
+
}
|
image/01_metric_comparison.png
ADDED
|
Git LFS Details
|
image/02_latency_comparison.png
ADDED
|
Git LFS Details
|
image/效果对比.png
ADDED
|
Git LFS Details
|
release_manifest.json
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"complete_model": false,
|
| 3 |
+
"weight_files": [],
|
| 4 |
+
"links_are_placeholders": true,
|
| 5 |
+
"hf_repo_id": "YOUR_HF_ORG/Turn-Detection-Jev",
|
| 6 |
+
"github_repo": "YOUR_GITHUB_ORG/Turn-Detection-Jev",
|
| 7 |
+
"github_ref": "main",
|
| 8 |
+
"sha256": {
|
| 9 |
+
".gitattributes": "77db011f87feff0d3ed5a3d8935c8df0ccdf1f3f0e37ad39de66ef56aa3dc130",
|
| 10 |
+
"EVALUATION.md": "12cbe3a42811b407d6589b72d635dfd2be1ed0926e3401f1c75b515257b91905",
|
| 11 |
+
"LICENSE": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4",
|
| 12 |
+
"NOTICE": "0fa613f92457ef19139f198d3ea35e9f2879ea552f886ddf82977bd485102d62",
|
| 13 |
+
"README.md": "5d2e3912da2d40ba9f9abf37e49aee159fa8ed1f671635837ba437943c2856ae",
|
| 14 |
+
"README_zh.md": "736c25f1530929eb8b048b88eb7972ac98b09da71c0a91738427ada9e286fdd4",
|
| 15 |
+
"calibration.json": "e0bdf32ab8aae2d0750b9d50ace497cf5cd2d1729ad346063f3ec13996eb3a3e",
|
| 16 |
+
"image/01_metric_comparison.png": "797025024168f1c8e6cd26e0b8b4c3286ef1f1214bbe7e7efe57a781bbbfec12",
|
| 17 |
+
"image/02_latency_comparison.png": "1beeaa8bde006079f467590a6f014c68498ebfad67eb678b3734294db2e41b46",
|
| 18 |
+
"image/效果对比.png": "797025024168f1c8e6cd26e0b8b4c3286ef1f1214bbe7e7efe57a781bbbfec12"
|
| 19 |
+
},
|
| 20 |
+
"gpu_validation": "not performed by packaging; actual checkpoint validation remains required"
|
| 21 |
+
}
|