Release Aria v0.2.3.3
Browse filesOverwrite the v0.1-pre mirror with the v0.2.3.3 package (content of Project-Pitchee/Aria main HEAD, model_card frontmatter license: apache-2.0). v0.1-pre content preserved under the v0.1-pre tag.
- .gitignore +8 -0
- KNOWN_ISSUES.md +37 -0
- README.md +44 -19
- SHA256SUMS +36 -0
- config.json +134 -31
- history/v0.1-pre/KNOWN_ISSUES.md +7 -0
- history/v0.1-pre/README.md +36 -0
- history/v0.1-pre/config.json +44 -0
- history/v0.1-pre/inference.py +144 -0
- history/v0.1-pre/model_card.md +26 -0
- {weights → history/v0.1-pre/weights}/harmonization.json +0 -0
- {weights → history/v0.1-pre/weights}/imputation.json +0 -0
- {weights → history/v0.1-pre/weights}/linear_head.json +0 -0
- {weights → history/v0.1-pre/weights}/scaler.json +0 -0
- inference/__init__.py +22 -0
- inference/native_features/__init__.py +0 -0
- inference/native_features/abi_native.py +133 -0
- inference/native_features/gne_port.py +134 -0
- inference/native_features/praat_port.py +834 -0
- inference/native_features/voice_port.py +554 -0
- inference/predict.py +439 -0
- inference/stage0/__init__.py +18 -0
- inference/stage0/audio.py +97 -0
- inference/stage0/bypass.py +117 -0
- inference/stage0/constants.py +156 -0
- inference/stage0/gates.py +78 -0
- inference/stage0/missing.py +122 -0
- inference/stage0/pipeline.py +163 -0
- inference/stage0/quality.py +303 -0
- inference/stage0/report.py +147 -0
- inference/stage0/run_corpus.py +202 -0
- inference/stage0/run_requests.py +143 -0
- inference/stage0/segment.py +228 -0
- inference/stage0/vad.py +139 -0
- model_card.md +142 -16
- weights/g_table.json +295 -0
- weights/model.json +149 -0
.gitignore
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
*.py[cod]
|
| 3 |
+
.venv/
|
| 4 |
+
.DS_Store
|
| 5 |
+
*.npz
|
| 6 |
+
*.npy
|
| 7 |
+
*.csv
|
| 8 |
+
*.log
|
KNOWN_ISSUES.md
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Aria v0.2.3.3 已知部署问题
|
| 2 |
+
|
| 3 |
+
此页只记录已知问题,不在 v0.2.3.3 中修复。模型数值、门结论与发布状态不受影响。
|
| 4 |
+
|
| 5 |
+
## 范围与接口
|
| 6 |
+
|
| 7 |
+
- **未验证任意录音**:本包只在 PVQD 协议复合录音经阶段 0 产出的段 A 上验证过。面向任意录音的元音切割前端尚未建立(切割器任务书仍是草案),本包不对任意录音的切分正确性、噪声鲁棒性或端到端输出作任何声明。
|
| 8 |
+
- **VAD 模型不随包分发**:`inference/` 的阶段 0 需要 Silero v5 ONNX(SHA-256 `1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3`)。哈希不符会被硬性拒绝。只走已切好段 A 的路径不需要 VAD。
|
| 9 |
+
- **两个门口径并存,可评分人数差别很大**:本包同时输出训练口径(本包自身数字所用口径,R1/W_窗数 不排除)与产品口径(七门全过)的 status。§2.5 锁定 58 人上分别是 57 与 22 可评分。产品侧接入时必须显式选择口径,不要混用。
|
| 10 |
+
- **16 kHz 单声道假设**:段 A 必须是 16 kHz mono。其他采样率/声道不做隐式转换(`--segment-a` 会把多声道降为单声道、但不会重采样)。
|
| 11 |
+
|
| 12 |
+
## 冷启动阈值与档位
|
| 13 |
+
|
| 14 |
+
- **T_diag 未标定**:规格原文 3.44 是 L0(ABI 原始口径)的诊断阈值;标签换到 CAPE-V 严重度、头换成 L3 之后它没有定义。S_diag = g(4.128) = 99.87、T_red^cold = 90、T_yellow^cold = 75 都只是占位。**冷启动红线不得用于任何产品结论。** 本包因此只输出 ABI′、y_hat_mm 与 S,不输出产品档位判定。待 Ryo 裁定 T_diag^(3)。
|
| 15 |
+
- **个人化参数未出**:σ_floor、σ_CVA、w_base 未定。
|
| 16 |
+
|
| 17 |
+
## 数值与平台
|
| 18 |
+
|
| 19 |
+
- **native 引擎的平台依赖**:复刻的是 arm64 上 parselmouth 0.4.7(Praat 6.1.38,FMA 舍入)的数值;对 x86 Praat 基准未验证,预期 CPPS 有约 1e-3 dB 量级的平台差。
|
| 20 |
+
- **重采样残余**:段 A 是 16 kHz,经 sinc 重采样到 44.1 kHz;Hfno-6000 的 6–10 kHz 分母只有重采样残余。
|
| 21 |
+
- **段 A 与原始录音之间的特征偏移**:12 人上 ABI r = 0.954。属阶段 0 范畴,已裁定不回炉,记为已知限制。
|
| 22 |
+
- **G6 修正器未激活**(ŵ = 0),ABI′_corr = ABI′_raw。
|
| 23 |
+
|
| 24 |
+
## 阶段 0 遗留
|
| 25 |
+
|
| 26 |
+
- WADA-SNR 的 G 统计表是按论文的 Gamma(0.4) 语音模型模拟重建的,不是原表。
|
| 27 |
+
- q_codec placeholder-inactive(2026-10-18 裁决),产品口径不变。
|
| 28 |
+
- 距离档为占位;T60_est 与元音样检测器(VOWEL_* 常数)都是占位实现,后者只产覆盖率。
|
| 29 |
+
- 距产品预期的「自由说约一分钟」输入,PVQD 句材只有 16–30 s;R1(T_voiced < 10 s)在短材料上会大量触发(锁定集产品口径 36/58)。
|
| 30 |
+
|
| 31 |
+
## 评估口径的边界
|
| 32 |
+
|
| 33 |
+
- **§2.5 未达显著**:锁定集上 v0.2.3.3 与 v1 冻结 CAPE-V severity 头的精度差异不显著(单侧 Wilcoxon p = 0.790005;median d = +0.693 mm,95% CI [−2.408, +3.012])。**不得写「显著优于」。**
|
| 34 |
+
- 分层(作者 Category)子样本很小(Severe 仅 4 人),只作描述,不构成分层结论。
|
| 35 |
+
- hold-out 只有 12 人,rs 的 CI 很宽;L3 与 L0 在 hold-out 上的差在噪声内。
|
| 36 |
+
- G3 按修订门 R1 通过;按原门(CV 极差 ≤ 0.10)不通过。两者并排披露,不覆盖原结论。
|
| 37 |
+
- 非医疗产品:输出为嗓音状态参考,不是诊断。
|
README.md
CHANGED
|
@@ -1,33 +1,58 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
| 3 |
model_type: aria_linear_head
|
| 4 |
-
#
|
|
|
|
| 5 |
---
|
| 6 |
|
| 7 |
-
# Aria v0.
|
| 8 |
|
| 9 |
-
本目录是可直接运行的本地 Hugging Face 模型包
|
| 10 |
|
| 11 |
-
> ⚠️ **
|
|
|
|
|
|
|
| 12 |
|
| 13 |
-
Dev:Spearman **0.7422**,MAE **10.98 mm**(n=36,speaker-grouped,seed=7)。完整方法、局限和已知部署问题见 [model_card.md](model_card.md)。
|
| 14 |
## Quickstart
|
| 15 |
|
| 16 |
-
依赖
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
```bash
|
| 19 |
-
python -
|
| 20 |
-
|
| 21 |
-
import numpy as np
|
| 22 |
-
from inference.inference import predict
|
| 23 |
-
|
| 24 |
-
# X 必须是 (n, 398):[14 acoustic | 192 embedding mean | 192 embedding std]
|
| 25 |
-
X = np.load("your_canonical_features.npy")
|
| 26 |
-
out = predict(X)
|
| 27 |
-
print(out["severity_mm"])
|
| 28 |
-
PY
|
| 29 |
```
|
| 30 |
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
-
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
model_type: aria_linear_head
|
| 4 |
+
# §2.5 locked benchmark has run; the result (not significant vs. the v1 frozen
|
| 5 |
+
# CAPE-V severity head) is in the "锁定 Benchmark 结果" section of model_card.md.
|
| 6 |
---
|
| 7 |
|
| 8 |
+
# Aria v0.2.3.3
|
| 9 |
|
| 10 |
+
本目录是可直接运行的本地 Hugging Face 模型包:**冷启动测量内核**。
|
| 11 |
|
| 12 |
+
> ⚠️ **§2.5 锁定 Benchmark 已运行(2026-10-18,一次性)。** 结论:**未达显著**——单侧 Wilcoxon p = 0.790005;主终点 median d = +0.693 mm,95% CI [−2.408, +3.012];v0.2.3.3 rs 0.7164 / MAE 11.971 mm,v1 同集 rs 0.8130 / MAE 10.951 mm(n = 57)。**不得写「v0.2.3.3 显著优于 v1 severity 头」。** 完整方法、分层、拒评披露与合规声明见 [model_card.md](model_card.md)。
|
| 13 |
+
|
| 14 |
+
Dev(训练口径,非锁定测试):hold-out rs **0.888**(12 人),CV rs 均值 **0.642**,G3 修订门 R1 PASS。完整方法、局限和已知部署问题见 [model_card.md](model_card.md)。
|
| 15 |
|
|
|
|
| 16 |
## Quickstart
|
| 17 |
|
| 18 |
+
依赖:Python 3.10+、numpy、scipy、soundfile(走原始录音再加 onnxruntime)。
|
| 19 |
+
|
| 20 |
+
```bash
|
| 21 |
+
python -m pip install numpy scipy soundfile onnxruntime
|
| 22 |
+
```
|
| 23 |
+
|
| 24 |
+
**已切好的段 A(本包验证过的输入口径)**:
|
| 25 |
+
|
| 26 |
+
```bash
|
| 27 |
+
python inference/predict.py --segment-a segment_a_16k.wav --out out.json
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
**原始录音**(阶段 0 → 段 A → 九特征 → ABI′ → S;需自备 Silero VAD ONNX,SHA-256 `1a153a22…78e3`):
|
| 31 |
+
|
| 32 |
+
```bash
|
| 33 |
+
python inference/predict.py --audio rec.wav --vad-model /path/SileroVAD.onnx --out out.json
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
**批量**:`requests.jsonl` 每行 `{"audio_path", "audio_sha256", "distance_cm", "speaker_id"}`
|
| 37 |
|
| 38 |
```bash
|
| 39 |
+
python inference/predict.py --requests requests.jsonl --dataset /path/PVQD \
|
| 40 |
+
--vad-model /path/SileroVAD.onnx --outdir out/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
```
|
| 42 |
|
| 43 |
+
输出:`abi_prime`(ABI′,0–10)、`y_hat_mm`(= 10 × ABI′,PVQD CAPE-V 总体严重度 mm)、`s`(量纲链 g,0–100)、九个特征、阶段 0 的 21 字段质量报告与诊断,以及两个口径的 status(`status` = 本包训练口径,`status_product_caliber` = 七门全过)。`predict` 不读取任何标签。
|
| 44 |
+
|
| 45 |
+
| 路径 | 内容 |
|
| 46 |
+
|---|---|
|
| 47 |
+
| `weights/model.json` | L3 系数、clip、g 版本、冷启动阈值、dev 指标、包身份 |
|
| 48 |
+
| `weights/g_table.json` | 量纲链 g 常数表与验收记录 |
|
| 49 |
+
| `inference/predict.py` | 入口(`--audio` / `--segment-a` / `--requests`) |
|
| 50 |
+
| `inference/stage0/` | 阶段 0,seg_ver `stage0-v0.2.1.2`(13 模块) |
|
| 51 |
+
| `inference/native_features/` | native 九特征引擎(5 模块) |
|
| 52 |
+
| `history/v0.1-pre/` | 上一代发布包 v0.1-pre 的卡、权重与入口(归档,含其 §2.5 结论) |
|
| 53 |
+
|
| 54 |
+
`inference/` 的 18 个模块与冻结预测所用的代码逐字节相同(哈希见 `SHA256SUMS` 与 model_card.md 的「发布包自洽门」)。训练、重拟合与锁定评估代码在独立的 `aria-training` 仓;本包不含训练代码与任何测试标签。
|
| 55 |
+
|
| 56 |
+
## 适用范围
|
| 57 |
|
| 58 |
+
**已验证范围仅限 PVQD 协议复合录音经阶段 0 产出的段 A。** 任意录音场景不在本包验证范围内——面向任意录音的元音切割前端尚未建立。详见 [model_card.md](model_card.md) 的「适用范围」。
|
SHA256SUMS
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
e43bffce7eb41af9f502b06c08f9ac680f14e9dd6a0136d4613b87c879f6c52c .gitignore
|
| 2 |
+
09799c668fb6d0efbfb4a8c5b1fa933abfd1394b5aafd0492df08030f6f82d87 KNOWN_ISSUES.md
|
| 3 |
+
c7ffd4e407504792fd6b7b1ce090caeb29d0ac79e48cc4b74850f53bf35ab6c5 README.md
|
| 4 |
+
4592a220ea9d9636f2c4bca8d61907729942455830796250a4e3d3ebf6257bf5 config.json
|
| 5 |
+
1f4f2bd11b9d4251474876b42fe648226b4619d8040cb87914079a7aef413dc4 history/v0.1-pre/KNOWN_ISSUES.md
|
| 6 |
+
7ab6dae3405132d64ad9e108ed5d238dfc99896ecf563f81d1dae76564cbc0db history/v0.1-pre/README.md
|
| 7 |
+
3c73e254caccad277bbd816e8d8261b3388548b8a680859332d615c0159ff15c history/v0.1-pre/config.json
|
| 8 |
+
f5d380b7647b20ac36f5e245798f5a17c8e6d683cafd8e235c5de5af93c9b7d5 history/v0.1-pre/inference.py
|
| 9 |
+
fc7e5254c8a2f21d03166533cf73ef4c2111cd161fc9a65fc5b71b5319ff501b history/v0.1-pre/model_card.md
|
| 10 |
+
c96181f5b6521076b33fe0669c8dd9108882cad5d4db3a67ac5d819c33f0deb8 history/v0.1-pre/weights/harmonization.json
|
| 11 |
+
a9dbe926b0f402aa281b4523565cef947711147ee4ef2a5571236f540ee96d84 history/v0.1-pre/weights/imputation.json
|
| 12 |
+
96df4a1b83b0924c60245924bd99e04c09b01a98e57b93c83b143610ecb6de66 history/v0.1-pre/weights/linear_head.json
|
| 13 |
+
8ef107f56a047cee916524ffa250f0a8a44f16575195c10cf8e736f493ae0007 history/v0.1-pre/weights/scaler.json
|
| 14 |
+
b87f53f518b3d8bf89a5bade55a35d5a29620307de37ec640dec42ada01d297c inference/__init__.py
|
| 15 |
+
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 inference/native_features/__init__.py
|
| 16 |
+
b6e704483175268259e6f45e3bc01a3968d1bb07b280d1f0f05e22c5c304080b inference/native_features/abi_native.py
|
| 17 |
+
b955c7cb8402baf2429dadfa1ff9bbf95dd57b4f2e6ff95dc0f5250b5873dc10 inference/native_features/gne_port.py
|
| 18 |
+
c66ee3ec20512209ec5ffb20aacc854f808d4dfb1dfa035415f49100a4ad5776 inference/native_features/praat_port.py
|
| 19 |
+
7478ccf7c54b35c21295a53107944b84a9d0f549f6299173ba6ebe642379315e inference/native_features/voice_port.py
|
| 20 |
+
e35c37e2f4f6abfb6fb0b6d144f4db83ba3feb64b9c2c62338ae5f6c82e7b16a inference/predict.py
|
| 21 |
+
130795620d3d087ee155efa00b475b1b6f696a4f98e114f6204679c70a6a022c inference/stage0/__init__.py
|
| 22 |
+
c2c5e314821adca2362fd9e88eefe6dffe9ede0e518f7dc5185db8894cd02438 inference/stage0/audio.py
|
| 23 |
+
0ce0f394c401712c6c18e320b554533ba1990aefa7f9168f6bb7369112021c14 inference/stage0/bypass.py
|
| 24 |
+
6285e758d761122e73366292723208140ed84d414131b2fafe79ca6239b3723a inference/stage0/constants.py
|
| 25 |
+
9608c4a15ed35237afe20b9c710e5dac98a20f3ad6b8e37189d1a46edf2b4914 inference/stage0/gates.py
|
| 26 |
+
b0aab8b0dec08a35851de3562ec88e870a57294039a6e22562cb5b88e12b8ec9 inference/stage0/missing.py
|
| 27 |
+
25361d5e4231e17f3351223c2351e6aaf3168587a1d4d1f67e6473f753ddaa99 inference/stage0/pipeline.py
|
| 28 |
+
5fab8377aff7f74e1472087071221853c4c38beb89c33ab86a14fce8fe92d7f5 inference/stage0/quality.py
|
| 29 |
+
99e8410722f19d14ae2afb498427824f0275e3754bf619e719638675d069a032 inference/stage0/report.py
|
| 30 |
+
8ccef988629394cbc7cb6007bf851910b4f0685103e23cc43d4d9f701feb27fd inference/stage0/run_corpus.py
|
| 31 |
+
365e8c27ba7baac75f6b73451cda00f69745627c9e64b1fcb64e0de12fbe813c inference/stage0/run_requests.py
|
| 32 |
+
a87ed8d48cdb8f2d40790c65ce6f3946f0acf9131d70c60d01686aa97f849721 inference/stage0/segment.py
|
| 33 |
+
805678eb774d1cd8b78c69b930834b8162b349ee9e1c5c6af2cbffa9a715b8c0 inference/stage0/vad.py
|
| 34 |
+
f5d7fbfd31e41da42c9def865c15d57917554eaccb4a46739fa472b25c8c0542 model_card.md
|
| 35 |
+
4cc7d521f6989c87b2355f49381081541860c6f60aa9e170f2393d727c8da385 weights/g_table.json
|
| 36 |
+
4415d3fd2addcb5fdc23b5eab2b60ef6b82183954d5d5378b9892050c8ae56c0 weights/model.json
|
config.json
CHANGED
|
@@ -1,40 +1,143 @@
|
|
| 1 |
{
|
| 2 |
"model_type": "aria_linear_head",
|
| 3 |
-
"model_version": "v0.
|
| 4 |
-
"status": "
|
| 5 |
-
"
|
| 6 |
-
"
|
| 7 |
-
"
|
| 8 |
-
"
|
| 9 |
-
"
|
| 10 |
-
"
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
"
|
| 14 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
},
|
| 16 |
"training_data": {
|
| 17 |
"corpus": "PVQD",
|
| 18 |
-
"
|
| 19 |
-
"
|
| 20 |
-
"
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
"test_labels_in_package": false,
|
| 26 |
-
"
|
| 27 |
-
},
|
| 28 |
-
"preprocessing": {
|
| 29 |
-
"sentinel_policy": "core_v1",
|
| 30 |
-
"imputation": "train-only median on trusted acoustic values",
|
| 31 |
-
"availability_indicators": "14 unscaled indicators appended after standardized F2; 1=available and task-trusted",
|
| 32 |
-
"harmonization": "PVQD CAPE-V train-only mean/sd; inverse transform in weights/harmonization.json"
|
| 33 |
},
|
| 34 |
"evaluation": {
|
| 35 |
-
"
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"model_type": "aria_linear_head",
|
| 3 |
+
"model_version": "v0.2.3.3",
|
| 4 |
+
"status": "formal cold-start release; G3 PASS under revision R1; T_diag uncalibrated (pending ruling); §2.5 locked benchmark run 2026-10-18, not significant vs. the v1 frozen CAPE-V severity head",
|
| 5 |
+
"contract": {
|
| 6 |
+
"input": "segment A: 16 kHz mono waveform, or a raw recording (stage 0 is bundled)",
|
| 7 |
+
"output": "abi_prime [0,10], y_hat_mm = 10 * abi_prime (PVQD CAPE-V overall severity, mm), s [0,100]",
|
| 8 |
+
"validated_input": "PVQD protocol compound recordings (sustained /a/, /i/ + six CAPE-V sentences), whole file through stage 0",
|
| 9 |
+
"out_of_scope": "any-recording vowel cutting; the front end that locates and cuts a sustained vowel does not exist yet",
|
| 10 |
+
"health_to_severity_mapping": "none; abi_prime is CAPE-V severity / 10 directly"
|
| 11 |
+
},
|
| 12 |
+
"pipeline": {
|
| 13 |
+
"chain": "stage 0 segment A -> native 9 features -> L3 linear head -> clip [0,10] (G6 identity) -> g table -> S",
|
| 14 |
+
"seg_ver": "stage0-v0.2.1.2",
|
| 15 |
+
"feature_engine": "native",
|
| 16 |
+
"head": "single affine linear regression (refit L3, lambda = 100)",
|
| 17 |
+
"feature_names": [
|
| 18 |
+
"cpps",
|
| 19 |
+
"jitter_local",
|
| 20 |
+
"gne_max",
|
| 21 |
+
"hfno6000",
|
| 22 |
+
"hnrd",
|
| 23 |
+
"h1h2",
|
| 24 |
+
"shimmer_local_db",
|
| 25 |
+
"shimmer_local",
|
| 26 |
+
"psd"
|
| 27 |
+
],
|
| 28 |
+
"abi_prime_clip": [
|
| 29 |
+
0.0,
|
| 30 |
+
10.0
|
| 31 |
+
],
|
| 32 |
+
"g_ver": 3,
|
| 33 |
+
"g_space": "ABI_prime_corr [0,10] -> S [0,100]",
|
| 34 |
+
"g6_correction_active": false
|
| 35 |
+
},
|
| 36 |
+
"package_layout": {
|
| 37 |
+
"weights/model.json": "L3 coefficients, clip, g version, cold-start thresholds, dev metrics, package identity",
|
| 38 |
+
"weights/g_table.json": "dimension chain g constant table (knots / p / sigma) with its acceptance record",
|
| 39 |
+
"inference/predict.py": "entry point: --audio | --segment-a | --requests",
|
| 40 |
+
"inference/stage0/": "stage 0, seg_ver stage0-v0.2.1.2 (13 modules, byte-identical to the frozen prediction run)",
|
| 41 |
+
"inference/native_features/": "native nine-feature engine (5 modules, byte-identical to the frozen prediction run)",
|
| 42 |
+
"history/v0.1-pre/": "archived v0.1-pre package (card, weights, entry point)"
|
| 43 |
+
},
|
| 44 |
+
"dependencies": {
|
| 45 |
+
"python": ">=3.10",
|
| 46 |
+
"core": [
|
| 47 |
+
"numpy",
|
| 48 |
+
"scipy",
|
| 49 |
+
"soundfile",
|
| 50 |
+
"onnxruntime"
|
| 51 |
+
],
|
| 52 |
+
"segment_a_only_path_needs": [
|
| 53 |
+
"numpy",
|
| 54 |
+
"scipy"
|
| 55 |
+
],
|
| 56 |
+
"vad_model": {
|
| 57 |
+
"name": "SileroVAD.onnx (Silero v5)",
|
| 58 |
+
"sha256": "1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3",
|
| 59 |
+
"bundled": false,
|
| 60 |
+
"resolution": "--vad-model PATH, $ARIA_VAD_MODEL, or inference/models/SileroVAD.onnx"
|
| 61 |
+
}
|
| 62 |
},
|
| 63 |
"training_data": {
|
| 64 |
"corpus": "PVQD",
|
| 65 |
+
"fit_pool_rows": 212,
|
| 66 |
+
"holdout_rows": 12,
|
| 67 |
+
"category_distribution_fit_pool": {
|
| 68 |
+
"normal": 84,
|
| 69 |
+
"mild": 79,
|
| 70 |
+
"moderate": 31,
|
| 71 |
+
"severe": 18
|
| 72 |
+
},
|
| 73 |
+
"label": "PVQD CAPE-V overall severity average / 10",
|
| 74 |
+
"caliber": "relaxed / training (R1 and W_窗数 not excluded)",
|
| 75 |
+
"locked_60_used_in_fit": false,
|
| 76 |
"test_labels_in_package": false,
|
| 77 |
+
"source_repo": "aria-training"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
},
|
| 79 |
"evaluation": {
|
| 80 |
+
"dev": {
|
| 81 |
+
"holdout_n": 12,
|
| 82 |
+
"holdout_rs": 0.8881118881118882,
|
| 83 |
+
"holdout_rs_L0": 0.8951048951048951,
|
| 84 |
+
"holdout_mae_abi": 1.254975337236816,
|
| 85 |
+
"cv_mean_rs": 0.64148287217087,
|
| 86 |
+
"cv_rs_range": 0.20672451853288648,
|
| 87 |
+
"g3_refit_gate": "PASS under revision R1 (tau = 0.369)",
|
| 88 |
+
"g3_refit_gate_original": "FAIL (CV range > 0.10; not-worse-than-L0 -0.007)"
|
| 89 |
+
},
|
| 90 |
+
"locked_benchmark": {
|
| 91 |
+
"run": "§2.5, 2026-10-18, one-shot",
|
| 92 |
+
"comparator": "v1 frozen CAPE-V severity head (native mm, not recomputed)",
|
| 93 |
+
"paired_n": 57,
|
| 94 |
+
"primary_endpoint_median_d_mm": 0.693,
|
| 95 |
+
"primary_endpoint_ci95_mm": [
|
| 96 |
+
-2.408,
|
| 97 |
+
3.012
|
| 98 |
+
],
|
| 99 |
+
"wilcoxon_one_sided_p": 0.790005,
|
| 100 |
+
"sign_test_one_sided_p": 0.395683,
|
| 101 |
+
"verdict": "not significant; do not claim v0.2.3.3 beats the v1 severity head on the locked set",
|
| 102 |
+
"aria_rs": 0.7164,
|
| 103 |
+
"aria_rs_ci95": [
|
| 104 |
+
0.5338,
|
| 105 |
+
0.8429
|
| 106 |
+
],
|
| 107 |
+
"aria_mae_mm": 11.971,
|
| 108 |
+
"aria_mae_ci95_mm": [
|
| 109 |
+
9.553,
|
| 110 |
+
14.482
|
| 111 |
+
],
|
| 112 |
+
"v1_rs": 0.813,
|
| 113 |
+
"v1_mae_mm": 10.951,
|
| 114 |
+
"strata_descriptive": "Normal/Mild comparable; Moderate/Severe descriptively weaker for v0.2.3.3",
|
| 115 |
+
"disclosures": {
|
| 116 |
+
"intent_to_screen": "60 allocated -> 58 scored -> 2 VAD rejections (PT084, PT134)",
|
| 117 |
+
"rejected_in_package_caliber": "PT097 rejected:R2|R5 (no prediction produced)",
|
| 118 |
+
"product_caliber": "22 scored / 36 rejected:R1, descriptive only, not part of any endpoint"
|
| 119 |
+
},
|
| 120 |
+
"frozen_predictions_sha256": "3ff22e6a55b13673b17d7d02ba5ac564ca777de96e927186172991508b144d69",
|
| 121 |
+
"generation_manifest_sha256": "08655d7bdf733f2d6ea138008f7ce10076e55017de60582058a2da2fe6002c38",
|
| 122 |
+
"report": "aria-training/docs/v0233-locked-benchmark-result.md"
|
| 123 |
+
}
|
| 124 |
+
},
|
| 125 |
+
"release_selfcheck": {
|
| 126 |
+
"what": "re-ran the 58 locked recordings through this package's own entry point and compared against the frozen prediction file",
|
| 127 |
+
"predictions_csv_sha256": "3ff22e6a55b13673b17d7d02ba5ac564ca777de96e927186172991508b144d69",
|
| 128 |
+
"byte_identical_to_frozen": true,
|
| 129 |
+
"max_abs_diff_abi_prime": 0.0,
|
| 130 |
+
"tolerance": 1e-06,
|
| 131 |
+
"status_rows_matching": "58/58 (57 scored / 1 rejected:R2|R5)",
|
| 132 |
+
"modules_verified_against_generation_manifest": "18/18 (13 stage 0 + 5 native) byte-identical",
|
| 133 |
+
"report": "aria-training/artifacts/release-selfcheck-v0.2.3.3/selfcheck_report.md"
|
| 134 |
+
},
|
| 135 |
+
"cold_start_thresholds": {
|
| 136 |
+
"T_diag_L0": 3.44,
|
| 137 |
+
"S_diag": 99.87,
|
| 138 |
+
"T_red_cold": 90.0,
|
| 139 |
+
"T_yellow_cold": 75.0,
|
| 140 |
+
"status": "UNCALIBRATED placeholder (T_diag from the L0 spec does not carry over to the severity label); the package therefore emits measurements only, no product band decision"
|
| 141 |
+
},
|
| 142 |
+
"supersedes": "v0.2.3.2 (draft, praat-script, archived in aria-training); v0.2.0.1 (breathiness label, archived in aria-training); v0.1-pre (this repo, archived under history/v0.1-pre/)"
|
| 143 |
}
|
history/v0.1-pre/KNOWN_ISSUES.md
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Aria v0.1-pre 已知部署问题
|
| 2 |
+
|
| 3 |
+
此页只记录已知问题,不在 v0.1-pre 中修复:
|
| 4 |
+
|
| 5 |
+
- 端侧哨兵不一致:离线管线将 `HNR=-200` 视为不可用,端侧当前使用 `0.0`。
|
| 6 |
+
- 部署维度不一致:离线 Aria 头是 220 维(F2 206 维 + 14 个可用性指示),现有端侧接口仍是 206 维;待产品决策。
|
| 7 |
+
- phone 域鲁棒性:需要全量 phone 域重训后再评估。§2.5 已于 2026-10-18 运行(D1):**未达显著**(单侧 Wilcoxon p = 0.581281;median d = −0.063 mm,95% CI [−1.221, +1.728];n = 58),因此 phone 域结论不能由该测试推出。
|
history/v0.1-pre/README.md
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
model_type: aria_linear_head
|
| 4 |
+
# v0.1-pre is archived under history/v0.1-pre/; its §2.5 test (D1) has since run
|
| 5 |
+
# and landed not significant. See model_card.md in this directory.
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# Aria v0.1-pre
|
| 9 |
+
|
| 10 |
+
> **归档说明(2026-10-18)。** 本 README 随本仓升级到 v0.2.3.3 移入 `history/v0.1-pre/`,其 `inference.py` 与 `weights/` 也一并移入本目录。要复跑下面的 Quickstart:把 `inference.py` 与 `weights/` 拷回包根(即把它们放到 `inference/inference.py` 与 `weights/`),或直接调用 `inference.load_artifacts("<本仓根>/history/v0.1-pre")` 并传入显式 root。
|
| 11 |
+
|
| 12 |
+
本目录是可直接运行的本地 Hugging Face 模型包草案。
|
| 13 |
+
|
| 14 |
+
> ⚠️ **Test evaluation 已运行(D1,2026-10-18):未达显著**(单侧 Wilcoxon p = 0.581281;median d = −0.063 mm,95% CI [−1.221, +1.728];n = 58)。dev 结果之外不得据此声称优于 v1 severity 头。详见 [model_card.md](model_card.md)。
|
| 15 |
+
|
| 16 |
+
Dev:Spearman **0.7422**,MAE **10.98 mm**(n=36,speaker-grouped,seed=7)。完整方法、局限和已知部署问题见 [model_card.md](model_card.md)。
|
| 17 |
+
## Quickstart
|
| 18 |
+
|
| 19 |
+
依赖只有 Python 3.10+ 和 NumPy:
|
| 20 |
+
|
| 21 |
+
```bash
|
| 22 |
+
python -m pip install numpy
|
| 23 |
+
python - <<'PY'
|
| 24 |
+
import numpy as np
|
| 25 |
+
from inference.inference import predict
|
| 26 |
+
|
| 27 |
+
# X 必须是 (n, 398):[14 acoustic | 192 embedding mean | 192 embedding std]
|
| 28 |
+
X = np.load("your_canonical_features.npy")
|
| 29 |
+
out = predict(X)
|
| 30 |
+
print(out["severity_mm"])
|
| 31 |
+
PY
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
传入原始声学可用性时使用 `predict(X, acoustic_available=mask)`,其中 `mask` 为 `(n,14)` 的 0/1 数组。`predict` 不读取任何标签。
|
| 35 |
+
|
| 36 |
+
权重、scaler、harmonization 和填补参数在 `weights/`;产品推理实现只依赖 NumPy。训练与评估代码在独立的 `aria-training` repo。
|
history/v0.1-pre/config.json
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_type": "aria_linear_head",
|
| 3 |
+
"model_version": "v0.1-pre",
|
| 4 |
+
"status": "pre-release; dev-only package (archived under history/v0.1-pre/ in the v0.2.3.3 release)",
|
| 5 |
+
"archived_note": "The §2.5 locked test for this package ran 2026-10-18 (D1) and landed not significant: one-sided Wilcoxon p = 0.581281; median d = -0.063 mm, 95% CI [-1.221, +1.728], n = 58. Report: VocalHealth-v2 reports/d1-locked-test-result.md. This file is otherwise the v0.1-pre artifact unchanged.",
|
| 6 |
+
"architecture": {
|
| 7 |
+
"head": "single affine linear regression",
|
| 8 |
+
"input_raw_dim": 398,
|
| 9 |
+
"input_layout": "[14 acoustic | 192 embedding mean | 192 embedding std]",
|
| 10 |
+
"feature_definition": "F2 = 14 acoustic + 192 embedding mean + 14 binary availability indicators",
|
| 11 |
+
"input_model_dim": 220,
|
| 12 |
+
"output": "normalized PVQD CAPE-V severity z-score; invert with harmonization parameters",
|
| 13 |
+
"folded": true,
|
| 14 |
+
"selected_model": "Ridge",
|
| 15 |
+
"ridge_alpha": 100.0
|
| 16 |
+
},
|
| 17 |
+
"training_data": {
|
| 18 |
+
"corpus": "PVQD",
|
| 19 |
+
"selection": "clean-only exact-provenance rows",
|
| 20 |
+
"clean_rows": 236,
|
| 21 |
+
"train_rows": 200,
|
| 22 |
+
"dev_rows": 36,
|
| 23 |
+
"split": "speaker-grouped",
|
| 24 |
+
"seed": 7,
|
| 25 |
+
"test_used": false,
|
| 26 |
+
"test_labels_in_package": false,
|
| 27 |
+
"source_bundle_sha256": "4689ea1ffa5130568100baf352e9b64a5eb4ddf1e55f19c46a49a959b09af2de"
|
| 28 |
+
},
|
| 29 |
+
"preprocessing": {
|
| 30 |
+
"sentinel_policy": "core_v1",
|
| 31 |
+
"imputation": "train-only median on trusted acoustic values",
|
| 32 |
+
"availability_indicators": "14 unscaled indicators appended after standardized F2; 1=available and task-trusted",
|
| 33 |
+
"harmonization": "PVQD CAPE-V train-only mean/sd; inverse transform in weights/harmonization.json"
|
| 34 |
+
},
|
| 35 |
+
"evaluation": {
|
| 36 |
+
"dev_spearman": 0.7422282432240754,
|
| 37 |
+
"dev_mae_mm": 10.97657309220328,
|
| 38 |
+
"dev_n": 36,
|
| 39 |
+
"test_status": "run 2026-10-18 (D1); not significant vs. the v1 frozen CAPE-V severity head",
|
| 40 |
+
"test_one_sided_wilcoxon_p": 0.581281214201513,
|
| 41 |
+
"test_median_d_mm": -0.063,
|
| 42 |
+
"test_n": 58
|
| 43 |
+
}
|
| 44 |
+
}
|
history/v0.1-pre/inference.py
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Minimal Aria v0.1-pre inference (NumPy only).
|
| 2 |
+
|
| 3 |
+
Input rows are raw canonical features in the order
|
| 4 |
+
[14 acoustic | 192 embedding mean | 192 embedding std]. The model uses
|
| 5 |
+
F2=[14 acoustic | 192 embedding mean] plus 14 availability indicators.
|
| 6 |
+
No labels are loaded or required.
|
| 7 |
+
"""
|
| 8 |
+
from __future__ import annotations
|
| 9 |
+
|
| 10 |
+
import argparse
|
| 11 |
+
import json
|
| 12 |
+
from pathlib import Path
|
| 13 |
+
from typing import Any
|
| 14 |
+
|
| 15 |
+
import numpy as np
|
| 16 |
+
|
| 17 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 18 |
+
CANONICAL_LAYOUT = "acoustic14_embedding_mean192_embedding_std192"
|
| 19 |
+
N_ACOUSTIC = 14
|
| 20 |
+
N_EMBED = 192
|
| 21 |
+
N_RAW = 398
|
| 22 |
+
N_MODEL = 220
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def _json(path: Path) -> dict[str, Any]:
|
| 26 |
+
with path.open() as f:
|
| 27 |
+
return json.load(f)
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def load_artifacts(root: str | Path = ROOT) -> dict[str, Any]:
|
| 31 |
+
root = Path(root)
|
| 32 |
+
weights = _json(root / "weights" / "linear_head.json")
|
| 33 |
+
scaler = _json(root / "weights" / "scaler.json")
|
| 34 |
+
harm = _json(root / "weights" / "harmonization.json")
|
| 35 |
+
imp = _json(root / "weights" / "imputation.json")
|
| 36 |
+
w = np.asarray(weights["weight"], dtype=np.float64)
|
| 37 |
+
if w.shape != (N_MODEL,) or not np.isfinite(w).all():
|
| 38 |
+
raise ValueError("linear head must contain 220 finite weights")
|
| 39 |
+
b = float(weights["bias"])
|
| 40 |
+
mean = np.asarray(scaler["mean"], dtype=np.float64)
|
| 41 |
+
scale = np.asarray(scaler["scale"], dtype=np.float64)
|
| 42 |
+
medians = np.asarray(imp["medians"], dtype=np.float64)
|
| 43 |
+
if mean.shape != (206,) or scale.shape != (206,) or (scale <= 0).any():
|
| 44 |
+
raise ValueError("invalid 206-column scaler")
|
| 45 |
+
if medians.shape != (14,) or not np.isfinite(medians).all():
|
| 46 |
+
raise ValueError("invalid 14-column imputation medians")
|
| 47 |
+
if not np.isfinite(b) or not np.isfinite(mean).all() or not np.isfinite(scale).all():
|
| 48 |
+
raise ValueError("non-finite model artifact")
|
| 49 |
+
return {"weight": w, "bias": b, "mean": mean, "scale": scale,
|
| 50 |
+
"medians": medians, "harmonization": harm, "imputation": imp}
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def _sentinel_mask(acoustic: np.ndarray, policy: str) -> np.ndarray:
|
| 54 |
+
if policy != "core_v1":
|
| 55 |
+
raise ValueError(f"unsupported sentinel policy: {policy}")
|
| 56 |
+
mask = np.zeros(acoustic.shape, dtype=bool)
|
| 57 |
+
mask[:, 0] = acoustic[:, 0] == -200.0
|
| 58 |
+
f0_failure = acoustic[:, 8] == 0.0
|
| 59 |
+
mask[f0_failure, 8:11] = True
|
| 60 |
+
return mask
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def transform_features(X: np.ndarray, *, acoustic_available: np.ndarray | None,
|
| 64 |
+
artifacts: dict[str, Any], feature_layout: str) -> np.ndarray:
|
| 65 |
+
"""Apply the frozen sentinel, imputation, scaler and indicator contract."""
|
| 66 |
+
if feature_layout != CANONICAL_LAYOUT:
|
| 67 |
+
raise ValueError("feature_layout must declare the canonical acoustic-first 398-column layout")
|
| 68 |
+
X = np.asarray(X, dtype=np.float64)
|
| 69 |
+
if X.ndim != 2 or X.shape[1] != N_RAW:
|
| 70 |
+
raise ValueError("X must have shape (n, 398): [14 acoustic | 192 mean | 192 std]")
|
| 71 |
+
if not np.isfinite(X[:, N_ACOUSTIC:]).all() or np.isinf(X[:, :N_ACOUSTIC]).any():
|
| 72 |
+
raise ValueError("embedding features and infinities must be finite; acoustic NaN is allowed")
|
| 73 |
+
raw_ac = X[:, :N_ACOUSTIC].copy()
|
| 74 |
+
if acoustic_available is None:
|
| 75 |
+
available = np.isfinite(raw_ac)
|
| 76 |
+
else:
|
| 77 |
+
available = np.asarray(acoustic_available)
|
| 78 |
+
if available.shape != raw_ac.shape or not np.isin(available, (0, 1, False, True)).all():
|
| 79 |
+
raise ValueError("acoustic_available must have shape (n, 14) and contain only 0/1")
|
| 80 |
+
available = available.astype(bool) & np.isfinite(raw_ac)
|
| 81 |
+
sentinels = _sentinel_mask(raw_ac, artifacts["imputation"]["policy"])
|
| 82 |
+
trusted = available & ~sentinels
|
| 83 |
+
filled = raw_ac
|
| 84 |
+
filled[~trusted] = np.take(artifacts["medians"], np.nonzero(~trusted)[1])
|
| 85 |
+
selected = np.column_stack([filled, X[:, N_ACOUSTIC:N_ACOUSTIC + N_EMBED]])
|
| 86 |
+
standardized = (selected - artifacts["mean"]) / artifacts["scale"]
|
| 87 |
+
return np.column_stack([standardized, trusted.astype(np.float64)])
|
| 88 |
+
|
| 89 |
+
|
| 90 |
+
def predict(X: np.ndarray, *, acoustic_available: np.ndarray | None = None,
|
| 91 |
+
feature_layout: str = CANONICAL_LAYOUT,
|
| 92 |
+
artifacts: dict[str, Any] | None = None) -> dict[str, np.ndarray]:
|
| 93 |
+
artifacts = load_artifacts() if artifacts is None else artifacts
|
| 94 |
+
processed = transform_features(X, acoustic_available=acoustic_available,
|
| 95 |
+
artifacts=artifacts, feature_layout=feature_layout)
|
| 96 |
+
z = processed @ artifacts["weight"] + artifacts["bias"]
|
| 97 |
+
harm = artifacts["harmonization"]
|
| 98 |
+
native = z * float(harm["sd_mm"]) + float(harm["mean_mm"])
|
| 99 |
+
return {"normalized_severity_z": z, "severity_mm": native,
|
| 100 |
+
"availability": processed[:, 206:].astype(bool)}
|
| 101 |
+
|
| 102 |
+
|
| 103 |
+
def _load_input(path: Path) -> tuple[np.ndarray, np.ndarray | None, str]:
|
| 104 |
+
if path.suffix == ".npy":
|
| 105 |
+
return np.load(path, allow_pickle=False), None, CANONICAL_LAYOUT
|
| 106 |
+
if path.suffix == ".npz":
|
| 107 |
+
with np.load(path, allow_pickle=False) as d:
|
| 108 |
+
if "X" not in d:
|
| 109 |
+
raise ValueError("NPZ input must contain X")
|
| 110 |
+
X = d["X"]
|
| 111 |
+
availability = d["acoustic_available"] if "acoustic_available" in d else None
|
| 112 |
+
if "feature_layout" in d:
|
| 113 |
+
value = d["feature_layout"][0]
|
| 114 |
+
layout = value.decode() if isinstance(value, bytes) else str(value)
|
| 115 |
+
else:
|
| 116 |
+
layout = ""
|
| 117 |
+
return X, availability, layout
|
| 118 |
+
raise ValueError("input must be .npy or .npz")
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
def main() -> None:
|
| 122 |
+
ap = argparse.ArgumentParser(description=__doc__)
|
| 123 |
+
ap.add_argument("--input", required=True, type=Path,
|
| 124 |
+
help="raw (n,398) .npy or canonical .npz; labels are ignored")
|
| 125 |
+
ap.add_argument("--output", required=True, type=Path,
|
| 126 |
+
help="JSON prediction output")
|
| 127 |
+
ap.add_argument("--feature-layout", default=CANONICAL_LAYOUT,
|
| 128 |
+
help="explicit layout declaration for .npy input")
|
| 129 |
+
args = ap.parse_args()
|
| 130 |
+
X, availability, embedded_layout = _load_input(args.input)
|
| 131 |
+
layout = args.feature_layout if args.input.suffix == ".npy" else embedded_layout
|
| 132 |
+
result = predict(X, acoustic_available=availability, feature_layout=layout)
|
| 133 |
+
payload = {"model": "aria-v0.1-pre", "n": int(len(X)),
|
| 134 |
+
"normalized_severity_z": result["normalized_severity_z"].tolist(),
|
| 135 |
+
"severity_mm": result["severity_mm"].tolist(),
|
| 136 |
+
"availability": result["availability"].astype(int).tolist()}
|
| 137 |
+
args.output.parent.mkdir(parents=True, exist_ok=True)
|
| 138 |
+
args.output.write_text(json.dumps(payload, indent=2) + "\n")
|
| 139 |
+
print(json.dumps({"model": payload["model"], "n": payload["n"],
|
| 140 |
+
"output": str(args.output)}, indent=2))
|
| 141 |
+
|
| 142 |
+
|
| 143 |
+
if __name__ == "__main__":
|
| 144 |
+
main()
|
history/v0.1-pre/model_card.md
ADDED
|
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Aria v0.1-pre
|
| 2 |
+
|
| 3 |
+
> **归档说明(2026-10-18)。** 本文件是 v0.1-pre 发布包的卡,随本仓升级到 v0.2.3.3 移入 `history/v0.1-pre/`(原路径 `model_card.md`;同批移入 `README.md`、`KNOWN_ISSUES.md`、`config.json`、`weights/*.json`、`inference.py`)。除下方 pending 一处按 D1 结果补写外,正文未改。原包仍可从本仓 tag `v0.1-pre` 取得。
|
| 4 |
+
|
| 5 |
+
> ⚠️ **Test evaluation 已运行(D1,2026-10-18 一次性锁定测试):未达显著。** 单侧 Wilcoxon signed-rank p = **0.581281**(α = 0.05,W = 829.0),主终点 median d = **−0.063 mm**(95% CI [−1.221, +1.728],跨 0),配对符号检验 29/58(单侧 p = 0.552158)。**v0.1-pre 在锁定集上未显著优于 v1 冻结 CAPE-V severity 头,不得写「显著优于」。** 描述性指标:v0.1-pre rs = 0.76864 / MAE = 11.465 mm;v1 rs = 0.82127 / MAE = 10.961 mm(n = 58)。本包仍只包含本地预发布权重,不代表测试集结果,也未上传测试标签;完整报告见 VocalHealth-v2 的 `reports/d1-locked-test-result.md`。
|
| 6 |
+
|
| 7 |
+
Aria 是 VocalHealth v2 的 Tier 1 Ridge 线性头。它把原始 398 列按 `[14 acoustic | 192 embedding mean | 192 embedding std]` 解释,使用 F2 的 14 个声学特征和 192 维 embedding mean,再追加 14 个声学可用性指示变量,得到 220 维输入。权重已折叠为一个线性层;推理输出先是 PVQD CAPE-V 的标准化严重度,再用保存的 train-only 参数逆变换到 mm。
|
| 8 |
+
|
| 9 |
+
## Dev 结果
|
| 10 |
+
|
| 11 |
+
- Spearman: **0.7422**
|
| 12 |
+
- MAE: **10.98 mm**
|
| 13 |
+
- n=36,speaker-grouped split,seed=7
|
| 14 |
+
- 训练样本:236 条 clean-only 行(200 train / 36 dev)
|
| 15 |
+
- 来源 bundle SHA-256:`4689ea1ffa5130568100baf352e9b64a5eb4ddf1e55f19c46a49a959b09af2de`
|
| 16 |
+
|
| 17 |
+
## 方法与预处理
|
| 18 |
+
|
| 19 |
+
声学缺失采用 `core_v1`:`HNR=-200` 标为不可用;`F0 median=0` 同时标记 F0 median/IQR/CV 不可用。不可用值使用 train-only、task-trusted 的逐列中位数填补;指示变量在填补前生成并保持未标准化。严重度 harmonization 使用 PVQD train-only CAPE-V mean/sd,推理时逆变换回 mm。
|
| 20 |
+
|
| 21 |
+
## 局限
|
| 22 |
+
|
| 23 |
+
- 增强特征定义曾混用;本 v0.1-pre 只使用 clean-only 重拟合结果,增强样本已排除。
|
| 24 |
+
- missingness 与域标签共线,线性头可能学习到采集域/缺失模式而非纯粹的声学严重度。
|
| 25 |
+
- narrow 8 kHz 身份削弱约 **22%**;phone 域鲁棒性需要全量重训后再判断。
|
| 26 |
+
- 端侧当前存在 sentinel(离线 `-200`、端侧 `0.0`)和维度(离线 220、端侧 206)不一致;详见 `KNOWN_ISSUES.md`。
|
{weights → history/v0.1-pre/weights}/harmonization.json
RENAMED
|
File without changes
|
{weights → history/v0.1-pre/weights}/imputation.json
RENAMED
|
File without changes
|
{weights → history/v0.1-pre/weights}/linear_head.json
RENAMED
|
File without changes
|
{weights → history/v0.1-pre/weights}/scaler.json
RENAMED
|
File without changes
|
inference/__init__.py
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Aria v0.2.3.3 cold-start inference package.
|
| 2 |
+
|
| 3 |
+
from inference.predict import load_package, predict_recording, score_segment
|
| 4 |
+
|
| 5 |
+
package = load_package() # weights/model.json + weights/g_table.json
|
| 6 |
+
out = predict_recording("rec.wav", package, vad_model="SileroVAD.onnx")
|
| 7 |
+
|
| 8 |
+
``segment_to_score`` (``score_segment``) is the measurement kernel proper:
|
| 9 |
+
segment A in, ABI' / y_hat_mm / S out. ``predict_recording`` prepends stage 0 so
|
| 10 |
+
a raw recording can be fed in; that path is validated only on PVQD protocol
|
| 11 |
+
compound recordings.
|
| 12 |
+
"""
|
| 13 |
+
|
| 14 |
+
from inference.predict import (abi_prime, load_package, nine_features,
|
| 15 |
+
predict_corpus, predict_recording, resolve_vad_model,
|
| 16 |
+
score_segment, severity_from_abi, statuses)
|
| 17 |
+
|
| 18 |
+
__all__ = [
|
| 19 |
+
"load_package", "abi_prime", "severity_from_abi", "nine_features",
|
| 20 |
+
"score_segment", "predict_recording", "predict_corpus", "statuses",
|
| 21 |
+
"resolve_vad_model",
|
| 22 |
+
]
|
inference/native_features/__init__.py
ADDED
|
File without changes
|
inference/native_features/abi_native.py
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Native (numpy) ABI nine-feature engine on stage-0 segment A.
|
| 2 |
+
|
| 3 |
+
Reimplements PART 2 of the original ABI Praat script (see
|
| 4 |
+
``training/abi_praat.py``) on top of ``praat_port`` without calling Praat.
|
| 5 |
+
The input contract matches ``abi_praat.segment_features``: segment A at any
|
| 6 |
+
rate, resampled to 44.1 kHz with Praat's sinc resampler (precision 50)
|
| 7 |
+
before the measures run.
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from __future__ import annotations
|
| 11 |
+
|
| 12 |
+
import math
|
| 13 |
+
|
| 14 |
+
import numpy as np
|
| 15 |
+
|
| 16 |
+
from . import praat_port as P
|
| 17 |
+
|
| 18 |
+
FEATURE_NAMES = ("cpps", "jitter_local", "gne_max", "hfno6000", "hnrd",
|
| 19 |
+
"h1h2", "shimmer_local_db", "shimmer_local", "psd")
|
| 20 |
+
SCRIPT_RATE = 44_100
|
| 21 |
+
_NAN = float("nan")
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
def _ltas_1to1_db(spec: P.Spectrum) -> np.ndarray:
|
| 25 |
+
"""Spectrum_to_Ltas_1to1: dB/Hz value of every bin."""
|
| 26 |
+
p = 2.0 * (spec.re ** 2 + spec.im ** 2) * spec.dx
|
| 27 |
+
with np.errstate(divide="ignore"):
|
| 28 |
+
return np.where(p == 0.0, -300.0, 10.0 * np.log10(p / 4.0e-10))
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def _ltas_bands_db(spec: P.Spectrum, bandwidth: float):
|
| 32 |
+
"""Spectrum_to_Ltas(bandwidth) -> (values, x1, dx, xmax)."""
|
| 33 |
+
nb = math.ceil(spec.xmax / bandwidth)
|
| 34 |
+
if bandwidth <= spec.dx:
|
| 35 |
+
raise P.PraatError(f"Bandwidth must be greater than {spec.dx}.")
|
| 36 |
+
energy = 2.0 * (spec.re ** 2 + spec.im ** 2)
|
| 37 |
+
fmin = np.arange(nb) * bandwidth
|
| 38 |
+
mean_e = P.mean_nointerp(energy, 0.0, spec.dx, 0.0, spec.xmax, fmin, fmin + bandwidth)
|
| 39 |
+
p = mean_e * spec.dx
|
| 40 |
+
with np.errstate(divide="ignore"):
|
| 41 |
+
vals = np.where(p == 0.0, -300.0, 10.0 * np.log10(p / 4.0e-10))
|
| 42 |
+
return vals, 0.5 * bandwidth, bandwidth, spec.xmax
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
def _ltas_mean_energy_db(v, x1, dx, xmax, fmin, fmax) -> float:
|
| 46 |
+
m = P.mean_nointerp(10.0 ** (0.1 * v), x1, dx, 0.0, xmax, fmin, fmax)[0]
|
| 47 |
+
return 10.0 * math.log10(m)
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def window_measures(win: P.Sound):
|
| 51 |
+
"""One 0.1 s window of PART 2 -> (hfno6000, hnrd, h1h2)."""
|
| 52 |
+
spec = P.to_spectrum(win, fast=True)
|
| 53 |
+
ltas = _ltas_1to1_db(spec)
|
| 54 |
+
lx1, ldx, lxmax = 0.0, spec.dx, spec.xmax
|
| 55 |
+
lmin = P.vector_window_extremum(ltas, lx1, ldx, 0.0, lxmax, 0.0, 10000.0, False)
|
| 56 |
+
ltas = ltas - lmin # Formula self+|min| / self-min
|
| 57 |
+
e_lo = _ltas_mean_energy_db(ltas, lx1, ldx, lxmax, 0.0, 6000.0)
|
| 58 |
+
e_hi = _ltas_mean_energy_db(ltas, lx1, ldx, lxmax, 6000.0, 10000.0)
|
| 59 |
+
hfno = e_lo / e_hi
|
| 60 |
+
|
| 61 |
+
mid = win.xmin + 0.05
|
| 62 |
+
cg = P.to_power_cepstrogram(win, 60.0, 0.002, 5000.0, 50.0)
|
| 63 |
+
nt = cg.z.shape[1]
|
| 64 |
+
iframe = min(max(P.x_to_nearest_index(cg.t1, cg.dt, mid), 1), nt)
|
| 65 |
+
nq = cg.z.shape[0]
|
| 66 |
+
dqc = ((nq - 1) * cg.dq) / (nq - 1)
|
| 67 |
+
col = P.subtract_trend_robust(cg.z[:, iframe - 1:iframe], dqc)[:, 0]
|
| 68 |
+
_, qpeak = P.cepstrum_peak(col, dqc, 60.0, 400.0, P.PEAK_PARABOLIC)
|
| 69 |
+
f_ceps = 1.0 / qpeak
|
| 70 |
+
|
| 71 |
+
l6, l6x1, l6dx, l6xmax = _ltas_bands_db(spec, 6.0)
|
| 72 |
+
_, f_peak = P.vector_extremum_and_x(l6, l6x1, l6dx, 0.0, l6xmax, f_ceps - 20, f_ceps + 20,
|
| 73 |
+
P.PEAK_NONE, True)
|
| 74 |
+
|
| 75 |
+
def ltas_max(lo, hi):
|
| 76 |
+
return P.vector_window_extremum(ltas, lx1, ldx, 0.0, lxmax, lo, hi, True)
|
| 77 |
+
|
| 78 |
+
def ltas_min(lo, hi):
|
| 79 |
+
return P.vector_window_extremum(ltas, lx1, ldx, 0.0, lxmax, lo, hi, False)
|
| 80 |
+
|
| 81 |
+
first = math.ceil(500 / f_peak)
|
| 82 |
+
last = math.floor(1500 / f_peak)
|
| 83 |
+
peaks = [ltas_max(p * f_peak - 20, p * f_peak + 20) for p in range(first, last + 1)]
|
| 84 |
+
last_v = math.floor(1500 / f_ceps) - 1
|
| 85 |
+
valleys = [ltas_min(v * f_peak + 20, (v + 1) * f_peak - 20) for v in range(first, last_v + 1)]
|
| 86 |
+
if not peaks or not valleys:
|
| 87 |
+
hnrd = _NAN # Table "Get mean" of an empty table
|
| 88 |
+
else:
|
| 89 |
+
hnrd = float(np.mean(peaks)) - float(np.mean(valleys))
|
| 90 |
+
|
| 91 |
+
h1 = P.vector_window_extremum(l6, l6x1, l6dx, 0.0, l6xmax, f_peak - 20, f_peak + 20, True)
|
| 92 |
+
h2 = P.vector_window_extremum(l6, l6x1, l6dx, 0.0, l6xmax, 2 * f_peak - 20, 2 * f_peak + 20, True)
|
| 93 |
+
return hfno, hnrd, h1 - h2
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
def spectral_features(abi: P.Sound) -> dict:
|
| 97 |
+
"""PART 2 window loop + global CPPS on the 44.1 kHz Sound ``abi``."""
|
| 98 |
+
start, duration = abi.xmin, abi.xmax - abi.xmin
|
| 99 |
+
n_windows = math.floor(duration / 0.1)
|
| 100 |
+
rows = []
|
| 101 |
+
for n in range(1, n_windows + 1):
|
| 102 |
+
end_w = start + n * 0.1
|
| 103 |
+
win = P.extract_part(abi, end_w - 0.1, end_w, preserve_times=True)
|
| 104 |
+
rows.append(window_measures(win))
|
| 105 |
+
rows = np.array(rows, dtype=np.float64).reshape(-1, 3)
|
| 106 |
+
out = {}
|
| 107 |
+
for k, name in enumerate(("hfno6000", "hnrd", "h1h2")):
|
| 108 |
+
col = rows[:, k]
|
| 109 |
+
# Table "Get mean" throws on an undefined cell (whole script fails) and
|
| 110 |
+
# returns undefined for zero rows; both end as NaN here.
|
| 111 |
+
out[name] = float(col.mean()) if col.size and np.all(np.isfinite(col)) else _NAN
|
| 112 |
+
cg = P.to_power_cepstrogram(abi, 60.0, 0.002, 5000.0, 50.0)
|
| 113 |
+
out["cpps"] = P.cpps(cg, abi.xmin, abi.xmax, 0.01, 0.001, 60.0, 330.0)
|
| 114 |
+
return out
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
def to_script_rate(x: np.ndarray, sample_rate: int) -> P.Sound:
|
| 118 |
+
s = P.Sound.from_samples(x, sample_rate)
|
| 119 |
+
if sample_rate == SCRIPT_RATE:
|
| 120 |
+
return s
|
| 121 |
+
return P.resample(s, SCRIPT_RATE, 50)
|
| 122 |
+
|
| 123 |
+
|
| 124 |
+
def segment_features(segment_a: np.ndarray, sample_rate: int = 16_000) -> dict:
|
| 125 |
+
"""Nine ABI variables from segment A; NaN where Praat would be undefined."""
|
| 126 |
+
from .gne_port import gne_maximum
|
| 127 |
+
from .voice_port import voice_report
|
| 128 |
+
|
| 129 |
+
abi = to_script_rate(segment_a, sample_rate)
|
| 130 |
+
values = spectral_features(abi)
|
| 131 |
+
values.update(voice_report(abi))
|
| 132 |
+
values["gne_max"] = gne_maximum(abi)
|
| 133 |
+
return {k: values[k] for k in FEATURE_NAMES}
|
inference/native_features/gne_port.py
ADDED
|
@@ -0,0 +1,134 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""GNE maximum: ``To Harmonicity (gne)... 500 4500 1000 80`` then ``Get maximum``.
|
| 2 |
+
|
| 3 |
+
Port of Sound_to_Harmonicity_GNE (Praat 6.1.38, fon/Sound_to_Harmonicity_GNE.cpp)
|
| 4 |
+
with Sound_to_LPC_autocorrelation and LPC_Sound_filterInverse (LPC/Sound_and_LPC.cpp).
|
| 5 |
+
Inner products and the inverse-filter accumulation use numpy's summation
|
| 6 |
+
order rather than Praat's, so results agree to rounding, not bit for bit.
|
| 7 |
+
"""
|
| 8 |
+
|
| 9 |
+
from __future__ import annotations
|
| 10 |
+
|
| 11 |
+
import math
|
| 12 |
+
|
| 13 |
+
import numpy as np
|
| 14 |
+
|
| 15 |
+
from . import praat_port as P
|
| 16 |
+
from .praat_port import index_to_x
|
| 17 |
+
|
| 18 |
+
|
| 19 |
+
def _lpc_autocorrelation(sound: P.Sound, order=13, analysis_width=30e-3, dt=10e-3):
|
| 20 |
+
"""Sound_to_LPC_autocorrelation, pre-emphasis off (1e9 Hz). Returns (t1, dt, A)."""
|
| 21 |
+
fs = 1.0 / sound.dx
|
| 22 |
+
window_duration = 2.0 * analysis_width
|
| 23 |
+
if math.floor(window_duration / sound.dx) <= order:
|
| 24 |
+
raise P.PraatError("Analysis window duration too short.")
|
| 25 |
+
if window_duration > sound.dx * sound.nx:
|
| 26 |
+
window_duration = sound.dx * sound.nx
|
| 27 |
+
n_frames, t1 = P.short_term_analysis(sound.x1, sound.dx, sound.nx, window_duration, dt)
|
| 28 |
+
nsf = int(P.c_round(window_duration * fs))
|
| 29 |
+
window = P.gaussian_window(nsf)
|
| 30 |
+
A = np.zeros((n_frames, order))
|
| 31 |
+
for k in range(n_frames):
|
| 32 |
+
t = index_to_x(t1, dt, k + 1)
|
| 33 |
+
idx = P.into_sound_indices(sound, t - 0.5 * window_duration, nsf)
|
| 34 |
+
x = P.gather(sound.z, idx)
|
| 35 |
+
x = (x - x.mean()) * window
|
| 36 |
+
r = np.array([np.dot(x[:nsf - i], x[i:]) for i in range(order + 1)])
|
| 37 |
+
if r[0] == 0.0:
|
| 38 |
+
continue # zero coefficients
|
| 39 |
+
a = np.zeros(order + 2) # a[1..order+1], 1-based
|
| 40 |
+
a[1] = 1.0
|
| 41 |
+
rc1 = -r[1] / r[0]
|
| 42 |
+
a[2] = rc1
|
| 43 |
+
gain = r[0] + r[1] * rc1
|
| 44 |
+
ncoef = order
|
| 45 |
+
for i in range(2, order + 1):
|
| 46 |
+
s = 0.0
|
| 47 |
+
for j in range(1, i + 1):
|
| 48 |
+
s += r[i - j + 1] * a[j]
|
| 49 |
+
rci = -s / gain
|
| 50 |
+
for j in range(2, i // 2 + 2):
|
| 51 |
+
at = a[j] + rci * a[i - j + 2]
|
| 52 |
+
a[i - j + 2] += rci * a[j]
|
| 53 |
+
a[j] = at
|
| 54 |
+
a[i + 1] = rci
|
| 55 |
+
gain += rci * s
|
| 56 |
+
if gain <= 0.0:
|
| 57 |
+
ncoef = i - 1
|
| 58 |
+
break
|
| 59 |
+
A[k, :ncoef] = a[2:2 + ncoef]
|
| 60 |
+
return t1, dt, A
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def _filter_inverse(sound: P.Sound, t1, dt, A):
|
| 64 |
+
n = sound.nx
|
| 65 |
+
order = A.shape[1]
|
| 66 |
+
times = np.array([index_to_x(sound.x1, sound.dx, i) for i in range(1, n + 1)])
|
| 67 |
+
frame = np.floor((times - t1) / dt + 1.0 + 0.5).astype(np.int64) # Sampled_xToNearestIndex
|
| 68 |
+
valid = (frame >= 1) & (frame <= A.shape[0])
|
| 69 |
+
coef = np.zeros((n, order))
|
| 70 |
+
coef[valid] = A[frame[valid] - 1]
|
| 71 |
+
src = sound.z.copy()
|
| 72 |
+
for j in range(1, order + 1):
|
| 73 |
+
prev = np.zeros(n)
|
| 74 |
+
prev[j:] = sound.z[:-j]
|
| 75 |
+
src += coef[:, j - 1] * prev # usableDepth: zeros before sample 1
|
| 76 |
+
src[~valid] = 0.0
|
| 77 |
+
return P.Sound(src, sound.xmin, sound.xmax, sound.x1, sound.dx)
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def _band_filter(re, im, dx, fmid, bandwidth):
|
| 81 |
+
x = np.arange(re.size) * dx
|
| 82 |
+
fmin, fmax = fmid - bandwidth / 2.0, fmid + bandwidth / 2.0
|
| 83 |
+
out = (x < fmin) | (x > fmax)
|
| 84 |
+
factor = 0.5 + 0.5 * np.cos((2.0 * math.pi / bandwidth) * (x - fmid))
|
| 85 |
+
factor[out] = 0.0
|
| 86 |
+
return re * factor, im * factor
|
| 87 |
+
|
| 88 |
+
|
| 89 |
+
def _lagged(E, n, d):
|
| 90 |
+
"""C[row, col] = sum_i row[i] * col[i + d]."""
|
| 91 |
+
if d >= 0:
|
| 92 |
+
return E[:, :n - d] @ E[:, d:].T
|
| 93 |
+
return E[:, -d:] @ E[:, :n + d].T
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
def gne_maximum(me: P.Sound, fmin=500.0, fmax=4500.0, bandwidth=1000.0, step=80.0) -> float:
|
| 97 |
+
original = P.resample(me, 10000, 500)
|
| 98 |
+
original.z -= original.z.mean()
|
| 99 |
+
duration = me.xmax - me.xmin
|
| 100 |
+
t1, dt, A = _lpc_autocorrelation(original)
|
| 101 |
+
flat = _filter_inverse(original, t1, dt, A)
|
| 102 |
+
spec = P.to_spectrum(flat, fast=True)
|
| 103 |
+
h_re, h_im = spec.im.copy(), -spec.re
|
| 104 |
+
envelopes = []
|
| 105 |
+
fmid = fmin
|
| 106 |
+
while fmid <= fmax:
|
| 107 |
+
b_re, b_im = _band_filter(spec.re, spec.im, spec.dx, fmid, bandwidth)
|
| 108 |
+
hb_re, hb_im = _band_filter(h_re, h_im, spec.dx, fmid, bandwidth)
|
| 109 |
+
band = P.spectrum_to_sound(P.Spectrum(b_re, b_im, spec.xmax, spec.dx))
|
| 110 |
+
hband = P.spectrum_to_sound(P.Spectrum(hb_re, hb_im, spec.xmax, spec.dx))
|
| 111 |
+
env = P.extract_part(band, 0.0, duration, preserve_times=True)
|
| 112 |
+
m = env.nx
|
| 113 |
+
e = np.sqrt(env.z ** 2 + hband.z[:m] ** 2)
|
| 114 |
+
envelopes.append(e - e.mean())
|
| 115 |
+
fmid += step
|
| 116 |
+
E = np.array(envelopes)
|
| 117 |
+
nenv, n = E.shape
|
| 118 |
+
power = np.einsum("ij,ij->i", E, E)
|
| 119 |
+
norm = np.sqrt(power)
|
| 120 |
+
cc = np.full((nenv, nenv), -np.inf)
|
| 121 |
+
# lags -3..3 samples (Sounds_crossCorrelate_short, -3.1e-4..3.1e-4 at 10 kHz)
|
| 122 |
+
lag_lo = math.ceil(-3.1e-4 / original.dx)
|
| 123 |
+
lag_hi = math.floor(3.1e-4 / original.dx)
|
| 124 |
+
for d in range(lag_lo, lag_hi + 1):
|
| 125 |
+
with np.errstate(all="ignore"): # Accelerate BLAS raises spurious FP flags
|
| 126 |
+
C = _lagged(E, n, d)
|
| 127 |
+
cc = np.maximum(cc, C)
|
| 128 |
+
with np.errstate(invalid="ignore", divide="ignore"):
|
| 129 |
+
cc = cc / np.outer(norm, norm)
|
| 130 |
+
rows, cols = np.tril_indices(nenv, -1)
|
| 131 |
+
keep = np.abs(rows - cols) >= bandwidth / 2.0 / step
|
| 132 |
+
vals = cc[rows[keep], cols[keep]]
|
| 133 |
+
vals = vals[np.isfinite(vals)]
|
| 134 |
+
return float(max(0.0, vals.max())) if vals.size else 0.0
|
inference/native_features/praat_port.py
ADDED
|
@@ -0,0 +1,834 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Numpy port of the Praat 6.1.38 primitives the ABI script calls.
|
| 2 |
+
|
| 3 |
+
Every function names the Praat routine it follows (source tag v6.1.38,
|
| 4 |
+
the version bundled with parselmouth 0.4.7). Index conventions are kept
|
| 5 |
+
1-based in comments and converted at the array boundary; floating-point
|
| 6 |
+
expressions keep Praat's operation order where it affects branching
|
| 7 |
+
(window edges, sample-index rounding).
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from __future__ import annotations
|
| 11 |
+
|
| 12 |
+
import math
|
| 13 |
+
from dataclasses import dataclass
|
| 14 |
+
from fractions import Fraction
|
| 15 |
+
|
| 16 |
+
import numpy as np
|
| 17 |
+
|
| 18 |
+
UNDEF = float("nan")
|
| 19 |
+
_EPS = 2.0 ** -53 # LAPACK dlamch("Epsilon"), round-to-nearest
|
| 20 |
+
_SQRT_EPS = math.sqrt(_EPS)
|
| 21 |
+
_GOLDEN = 1.0 - 0.6180339887498948482045868343656381177203
|
| 22 |
+
|
| 23 |
+
|
| 24 |
+
class PraatError(RuntimeError):
|
| 25 |
+
pass
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
def iround(x: float) -> int:
|
| 29 |
+
"""Melder_iround: tie up."""
|
| 30 |
+
return int(math.floor(x + 0.5))
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def fma(a: float, b: float, c: float) -> float:
|
| 34 |
+
"""Correctly rounded a*b + c.
|
| 35 |
+
|
| 36 |
+
The parselmouth 0.4.7 arm64 build contracts Praat's ``x1 + (i - 1) * dx``
|
| 37 |
+
(Sampled_indexToX and friends) into a fused multiply-add; the single
|
| 38 |
+
rounding decides ties in later Melder_iround calls, so it is reproduced
|
| 39 |
+
exactly here.
|
| 40 |
+
"""
|
| 41 |
+
a, b, c = float(a), float(b), float(c) # numpy ints would overflow inside Fraction
|
| 42 |
+
if math.isfinite(a) and math.isfinite(b) and math.isfinite(c):
|
| 43 |
+
return float(Fraction(a) * Fraction(b) + Fraction(c))
|
| 44 |
+
return a * b + c
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def index_to_x(x1: float, dx: float, index) -> float:
|
| 48 |
+
"""Sampled_indexToX with FMA contraction: x1 + (index - 1) * dx."""
|
| 49 |
+
return fma(float(index - 1), dx, x1)
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def index_to_x_array(x1: float, dx: float, count: int, first: int = 1) -> np.ndarray:
|
| 53 |
+
return np.array([index_to_x(x1, dx, first + k) for k in range(count)])
|
| 54 |
+
|
| 55 |
+
|
| 56 |
+
def c_round(x: float) -> float:
|
| 57 |
+
"""C ``round``: ties away from zero (Sound_createSimple)."""
|
| 58 |
+
return math.copysign(math.floor(abs(x) + 0.5), x)
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
# --------------------------------------------------------------------------
|
| 62 |
+
# Sampled objects
|
| 63 |
+
# --------------------------------------------------------------------------
|
| 64 |
+
|
| 65 |
+
@dataclass
|
| 66 |
+
class Sound:
|
| 67 |
+
z: np.ndarray # samples, 0-based array of Praat z[1][1..nx]
|
| 68 |
+
xmin: float
|
| 69 |
+
xmax: float
|
| 70 |
+
x1: float
|
| 71 |
+
dx: float
|
| 72 |
+
|
| 73 |
+
@property
|
| 74 |
+
def nx(self) -> int:
|
| 75 |
+
return int(self.z.size)
|
| 76 |
+
|
| 77 |
+
@classmethod
|
| 78 |
+
def from_samples(cls, z, sample_rate: float) -> "Sound":
|
| 79 |
+
z = np.asarray(z, dtype=np.float64).copy()
|
| 80 |
+
dx = 1.0 / sample_rate
|
| 81 |
+
return cls(z, 0.0, z.size / sample_rate, 0.5 * dx, dx)
|
| 82 |
+
|
| 83 |
+
def copy(self) -> "Sound":
|
| 84 |
+
return Sound(self.z.copy(), self.xmin, self.xmax, self.x1, self.dx)
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def x_to_low_index(x1, dx, x):
|
| 88 |
+
return math.floor((x - x1) / dx + 1.0)
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
def x_to_high_index(x1, dx, x):
|
| 92 |
+
return math.ceil((x - x1) / dx + 1.0)
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
def x_to_nearest_index(x1, dx, x):
|
| 96 |
+
return iround((x - x1) / dx + 1.0)
|
| 97 |
+
|
| 98 |
+
|
| 99 |
+
def window_samples(x1, dx, nx, xmin, xmax):
|
| 100 |
+
"""Sampled_getWindowSamples -> (imin, imax, count), 1-based."""
|
| 101 |
+
imin = 1 + math.ceil((xmin - x1) / dx)
|
| 102 |
+
imax = 1 + math.floor((xmax - x1) / dx)
|
| 103 |
+
imin = max(imin, 1)
|
| 104 |
+
imax = min(imax, nx)
|
| 105 |
+
return imin, imax, (imax - imin + 1 if imin <= imax else 0)
|
| 106 |
+
|
| 107 |
+
|
| 108 |
+
def short_term_analysis(x1, dx, nx, window_duration, time_step):
|
| 109 |
+
"""Sampled_shortTermAnalysis -> (numberOfFrames, firstTime)."""
|
| 110 |
+
my_duration = dx * nx
|
| 111 |
+
if window_duration > my_duration:
|
| 112 |
+
raise PraatError("shorter than window length.")
|
| 113 |
+
n = math.floor((my_duration - window_duration) / time_step) + 1
|
| 114 |
+
our_mid = x1 - 0.5 * dx + 0.5 * my_duration
|
| 115 |
+
return n, our_mid - 0.5 * (n * time_step) + 0.5 * time_step
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
def _autowindow(dom_lo, dom_hi, xmin, xmax):
|
| 119 |
+
if xmin >= xmax:
|
| 120 |
+
return dom_lo, dom_hi
|
| 121 |
+
return xmin, xmax
|
| 122 |
+
|
| 123 |
+
|
| 124 |
+
def _intersect(dom_lo, dom_hi, xmin, xmax):
|
| 125 |
+
"""Function_intersectRangeWithDomain for xmin < xmax."""
|
| 126 |
+
if xmin == xmax:
|
| 127 |
+
return None
|
| 128 |
+
xmin = max(xmin, dom_lo)
|
| 129 |
+
xmax = min(xmax, dom_hi)
|
| 130 |
+
if xmax <= xmin:
|
| 131 |
+
return None
|
| 132 |
+
return xmin, xmax
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
# --------------------------------------------------------------------------
|
| 136 |
+
# Interpolation (NUMinterpol.cpp)
|
| 137 |
+
# --------------------------------------------------------------------------
|
| 138 |
+
|
| 139 |
+
def _sinc_group(y, x, ml, depth):
|
| 140 |
+
"""Sinc branch of NUM_interpolate_sinc for one fixed depth >= 3."""
|
| 141 |
+
j = np.arange(depth, dtype=np.float64)
|
| 142 |
+
sign = np.where(np.arange(depth) % 2 == 0, 1.0, -1.0)
|
| 143 |
+
out = np.empty(x.size)
|
| 144 |
+
step = max(1, 2_000_000 // (2 * depth))
|
| 145 |
+
for s in range(0, x.size, step):
|
| 146 |
+
xs, mls = x[s:s + step], ml[s:s + step]
|
| 147 |
+
frac = xs - mls
|
| 148 |
+
a0 = math.pi * frac
|
| 149 |
+
aL = a0[:, None] + math.pi * j
|
| 150 |
+
wL = (0.5 * np.sin(a0))[:, None] * sign / aL * (1.0 + np.cos(aL / (frac + depth)[:, None]))
|
| 151 |
+
idxL = mls[:, None] - np.arange(depth) # 1-based
|
| 152 |
+
a0r = math.pi * (mls + 1 - xs)
|
| 153 |
+
aR = a0r[:, None] + math.pi * j
|
| 154 |
+
wR = (0.5 * np.sin(a0r))[:, None] * sign / aR * (1.0 + np.cos(aR / (depth + 1.0 - frac)[:, None]))
|
| 155 |
+
idxR = mls[:, None] + 1 + np.arange(depth)
|
| 156 |
+
out[s:s + step] = (y[idxL - 1] * wL).sum(axis=1) + (y[idxR - 1] * wR).sum(axis=1)
|
| 157 |
+
return out
|
| 158 |
+
|
| 159 |
+
|
| 160 |
+
def interpolate_sinc(y: np.ndarray, x, max_depth: int):
|
| 161 |
+
"""NUM_interpolate_sinc for an array of 1-based real indices ``x``."""
|
| 162 |
+
y = np.asarray(y, dtype=np.float64)
|
| 163 |
+
n = y.size
|
| 164 |
+
x = np.atleast_1d(np.asarray(x, dtype=np.float64))
|
| 165 |
+
out = np.empty(x.size)
|
| 166 |
+
if n < 1:
|
| 167 |
+
out[:] = UNDEF
|
| 168 |
+
return out
|
| 169 |
+
ml = np.floor(x).astype(np.int64)
|
| 170 |
+
lo, hi = x < 1, x > n
|
| 171 |
+
out[lo] = y[0]
|
| 172 |
+
out[hi] = y[-1]
|
| 173 |
+
exact = ~lo & ~hi & (x == ml)
|
| 174 |
+
out[exact] = y[ml[exact] - 1]
|
| 175 |
+
rest = ~(lo | hi | exact)
|
| 176 |
+
depth = np.minimum(np.minimum(max_depth, ml), n - ml) # midright-1 == ml
|
| 177 |
+
for d in np.unique(depth[rest]):
|
| 178 |
+
sel = rest & (depth == d)
|
| 179 |
+
xs, mls = x[sel], ml[sel]
|
| 180 |
+
if d <= 0:
|
| 181 |
+
out[sel] = y[np.floor(xs + 0.5).astype(np.int64) - 1]
|
| 182 |
+
elif d == 1:
|
| 183 |
+
out[sel] = y[mls - 1] + (xs - mls) * (y[mls] - y[mls - 1])
|
| 184 |
+
elif d == 2:
|
| 185 |
+
yl, yr = y[mls - 1], y[mls]
|
| 186 |
+
dyl = 0.5 * (yr - y[mls - 2])
|
| 187 |
+
dyr = 0.5 * (y[mls + 1] - yl)
|
| 188 |
+
fil, fir = xs - mls, mls + 1 - xs
|
| 189 |
+
out[sel] = yl * fir + yr * fil - fil * fir * (0.5 * (dyr - dyl) + (fil - 0.5) * (dyl + dyr - 2 * (yr - yl)))
|
| 190 |
+
else:
|
| 191 |
+
out[sel] = _sinc_group(y, xs, mls, int(d))
|
| 192 |
+
return out
|
| 193 |
+
|
| 194 |
+
|
| 195 |
+
def interpolate_sinc_scalar(y, x, max_depth):
|
| 196 |
+
return float(interpolate_sinc(y, np.array([x]), max_depth)[0])
|
| 197 |
+
|
| 198 |
+
|
| 199 |
+
def minimize_brent(f, a, b, tol):
|
| 200 |
+
"""NUMminimize_brent -> (x, fx)."""
|
| 201 |
+
v = a + _GOLDEN * (b - a)
|
| 202 |
+
fv = f(v)
|
| 203 |
+
x = w = v
|
| 204 |
+
fx = fw = fv
|
| 205 |
+
for _ in range(60):
|
| 206 |
+
middle = (a + b) / 2.0
|
| 207 |
+
tol_act = _SQRT_EPS * abs(x) + tol / 3.0
|
| 208 |
+
rng = b - a
|
| 209 |
+
if abs(x - middle) + rng / 2.0 <= 2.0 * tol_act:
|
| 210 |
+
return x, fx
|
| 211 |
+
new_step = _GOLDEN * ((b - x) if x < middle else (a - x))
|
| 212 |
+
if abs(x - w) >= tol_act:
|
| 213 |
+
t = (x - w) * (fx - fv)
|
| 214 |
+
q = (x - v) * (fx - fw)
|
| 215 |
+
p = (x - v) * q - (x - w) * t
|
| 216 |
+
q = 2.0 * (q - t)
|
| 217 |
+
if q > 0.0:
|
| 218 |
+
p = -p
|
| 219 |
+
else:
|
| 220 |
+
q = -q
|
| 221 |
+
if abs(p) < abs(new_step * q) and p > q * (a - x + 2.0 * tol_act) and p < q * (b - x - 2.0 * tol_act):
|
| 222 |
+
new_step = p / q
|
| 223 |
+
if abs(new_step) < tol_act:
|
| 224 |
+
new_step = tol_act if new_step > 0.0 else -tol_act
|
| 225 |
+
t = x + new_step
|
| 226 |
+
ft = f(t)
|
| 227 |
+
if ft <= fx:
|
| 228 |
+
if t < x:
|
| 229 |
+
b = x
|
| 230 |
+
else:
|
| 231 |
+
a = x
|
| 232 |
+
v, w, x = w, x, t
|
| 233 |
+
fv, fw, fx = fw, fx, ft
|
| 234 |
+
else:
|
| 235 |
+
if t < x:
|
| 236 |
+
a = t
|
| 237 |
+
else:
|
| 238 |
+
b = t
|
| 239 |
+
if ft <= fw or w == x:
|
| 240 |
+
v, w = w, t
|
| 241 |
+
fv, fw = fw, ft
|
| 242 |
+
elif ft <= fv or v == x or v == w:
|
| 243 |
+
v, fv = t, ft
|
| 244 |
+
return x, fx
|
| 245 |
+
|
| 246 |
+
|
| 247 |
+
PEAK_NONE, PEAK_PARABOLIC, PEAK_CUBIC, PEAK_SINC70, PEAK_SINC700 = 0, 1, 2, 3, 4
|
| 248 |
+
_PEAK_TO_VALUE_DEPTH = {PEAK_CUBIC: 2, PEAK_SINC70: 70, PEAK_SINC700: 700}
|
| 249 |
+
|
| 250 |
+
|
| 251 |
+
def improve_extremum(y, ixmid, depth, is_maximum):
|
| 252 |
+
"""NUMimproveExtremum -> (value, ixmid_real); ``ixmid`` 1-based."""
|
| 253 |
+
n = y.size
|
| 254 |
+
if ixmid <= 1:
|
| 255 |
+
return float(y[0]), 1.0
|
| 256 |
+
if ixmid >= n:
|
| 257 |
+
return float(y[-1]), float(n)
|
| 258 |
+
if depth <= PEAK_NONE:
|
| 259 |
+
return float(y[ixmid - 1]), float(ixmid)
|
| 260 |
+
if depth == PEAK_PARABOLIC:
|
| 261 |
+
dy = 0.5 * (y[ixmid] - y[ixmid - 2])
|
| 262 |
+
d2y = 2 * y[ixmid - 1] - y[ixmid - 2] - y[ixmid]
|
| 263 |
+
return float(y[ixmid - 1] + 0.5 * dy * dy / d2y), ixmid + dy / d2y
|
| 264 |
+
vdepth = _PEAK_TO_VALUE_DEPTH[depth]
|
| 265 |
+
sgn = -1.0 if is_maximum else 1.0
|
| 266 |
+
xr, fr = minimize_brent(lambda t: sgn * interpolate_sinc_scalar(y, t, vdepth),
|
| 267 |
+
ixmid - 1, ixmid + 1, 1e-10)
|
| 268 |
+
return (-fr if is_maximum else fr), xr
|
| 269 |
+
|
| 270 |
+
|
| 271 |
+
def vector_extremum_and_x(y, x1, dx, dom_lo, dom_hi, xmin, xmax, interp, is_maximum):
|
| 272 |
+
"""Vector_getMaximumAndX / Vector_getMinimumAndX on a single channel."""
|
| 273 |
+
nx = y.size
|
| 274 |
+
xmin, xmax = _autowindow(dom_lo, dom_hi, xmin, xmax)
|
| 275 |
+
imin, imax, count = window_samples(x1, dx, nx, xmin, xmax)
|
| 276 |
+
better = (lambda a, b: a > b) if is_maximum else (lambda a, b: a < b)
|
| 277 |
+
if count == 0:
|
| 278 |
+
depth = 1 if interp > PEAK_NONE else 0
|
| 279 |
+
yl = interpolate_sinc_scalar(y, (xmin - x1) / dx + 1.0, depth)
|
| 280 |
+
yr = interpolate_sinc_scalar(y, (xmax - x1) / dx + 1.0, depth)
|
| 281 |
+
val = max(yl, yr) if is_maximum else min(yl, yr)
|
| 282 |
+
xx = (xmin + xmax) / 2.0 if yl == yr else (xmin if better(yl, yr) else xmax)
|
| 283 |
+
return val, xx
|
| 284 |
+
ext, xi = float(y[imin - 1]), float(imin)
|
| 285 |
+
if better(y[imax - 1], ext):
|
| 286 |
+
ext, xi = float(y[imax - 1]), float(imax)
|
| 287 |
+
lo, hi = (imin + 1 if imin == 1 else imin), (imax - 1 if imax == nx else imax)
|
| 288 |
+
seg = y[lo - 2:hi + 1] # y[lo-1 .. hi+1] (1-based)
|
| 289 |
+
if seg.size >= 3:
|
| 290 |
+
c, l, r = seg[1:-1], seg[:-2], seg[2:]
|
| 291 |
+
cand = np.nonzero((c > l) & (c >= r) if is_maximum else (c < l) & (c <= r))[0] + lo
|
| 292 |
+
for i in cand:
|
| 293 |
+
val, ir = improve_extremum(y, int(i), interp, is_maximum)
|
| 294 |
+
if better(val, ext):
|
| 295 |
+
ext, xi = val, ir
|
| 296 |
+
xx = fma(xi - 1, dx, x1)
|
| 297 |
+
return ext, min(max(xx, xmin), xmax)
|
| 298 |
+
|
| 299 |
+
|
| 300 |
+
def vector_window_extremum(y, x1, dx, dom_lo, dom_hi, xmin, xmax, is_maximum):
|
| 301 |
+
"""Value-only Vector_getMaximum/Minimum with interpolation NONE."""
|
| 302 |
+
xmin, xmax = _autowindow(dom_lo, dom_hi, xmin, xmax)
|
| 303 |
+
imin, imax, count = window_samples(x1, dx, y.size, xmin, xmax)
|
| 304 |
+
if count == 0:
|
| 305 |
+
return vector_extremum_and_x(y, x1, dx, dom_lo, dom_hi, xmin, xmax, PEAK_NONE, is_maximum)[0]
|
| 306 |
+
seg = y[imin - 1:imax]
|
| 307 |
+
return float(seg.max() if is_maximum else seg.min())
|
| 308 |
+
|
| 309 |
+
|
| 310 |
+
# --------------------------------------------------------------------------
|
| 311 |
+
# Sampled_getMean (Sampled.cpp)
|
| 312 |
+
# --------------------------------------------------------------------------
|
| 313 |
+
|
| 314 |
+
def mean_nointerp(v, x1, dx, dom_lo, dom_hi, xmin, xmax):
|
| 315 |
+
"""Sampled_getMean(..., interpolate=false) vectorised over window arrays.
|
| 316 |
+
|
| 317 |
+
``v`` holds the (all defined) values in the requested unit.
|
| 318 |
+
"""
|
| 319 |
+
v = np.asarray(v, dtype=np.float64)
|
| 320 |
+
nx = v.size
|
| 321 |
+
xmin = np.atleast_1d(np.asarray(xmin, dtype=np.float64)).copy()
|
| 322 |
+
xmax = np.atleast_1d(np.asarray(xmax, dtype=np.float64)).copy()
|
| 323 |
+
auto = xmin >= xmax
|
| 324 |
+
xmin[auto], xmax[auto] = dom_lo, dom_hi
|
| 325 |
+
ok = xmin != xmax
|
| 326 |
+
xmin = np.maximum(xmin, dom_lo)
|
| 327 |
+
xmax = np.minimum(xmax, dom_hi)
|
| 328 |
+
ok &= xmax > xmin
|
| 329 |
+
rimin = (xmin - x1) / dx + 1.0
|
| 330 |
+
rimax = (xmax - x1) / dx + 1.0
|
| 331 |
+
ok &= (rimax >= 0.5) & (rimin < nx + 0.5)
|
| 332 |
+
imin = np.where(rimin < 0.5, 0, np.floor(rimin + 0.5)).astype(np.int64)
|
| 333 |
+
imax = np.where(rimax >= nx + 0.5, nx + 1, np.floor(rimax + 0.5)).astype(np.int64)
|
| 334 |
+
csum = np.concatenate(([0.0], np.cumsum(v))) # csum[k] = sum v[1..k]
|
| 335 |
+
lo = np.clip(imin + 1, 1, nx + 1)
|
| 336 |
+
hi = np.clip(imax - 1, 0, nx)
|
| 337 |
+
inner = np.where(hi >= lo, csum[np.maximum(hi, 0)] - csum[np.maximum(lo - 1, 0)], 0.0)
|
| 338 |
+
rng = np.where(hi >= lo, (hi - lo + 1).astype(np.float64), 0.0)
|
| 339 |
+
total, width = inner.copy(), rng.copy()
|
| 340 |
+
same = imin == imax
|
| 341 |
+
vi = v[np.clip(imin, 1, nx) - 1]
|
| 342 |
+
va = v[np.clip(imax, 1, nx) - 1]
|
| 343 |
+
ph = rimax - rimin
|
| 344 |
+
total = np.where(same, ph * vi, total)
|
| 345 |
+
width = np.where(same, ph, width)
|
| 346 |
+
left = ~same & (imin >= 1)
|
| 347 |
+
phl = imin - rimin + 0.5
|
| 348 |
+
total = np.where(left, total + phl * vi, total)
|
| 349 |
+
width = np.where(left, width + phl, width)
|
| 350 |
+
right = ~same & (imax <= nx)
|
| 351 |
+
phr = rimax - imax + 0.5
|
| 352 |
+
total = np.where(right, total + phr * va, total)
|
| 353 |
+
width = np.where(right, width + phr, width)
|
| 354 |
+
with np.errstate(invalid="ignore", divide="ignore"):
|
| 355 |
+
out = np.where(ok & (width > 0.0), total / width, UNDEF)
|
| 356 |
+
return out
|
| 357 |
+
|
| 358 |
+
|
| 359 |
+
def interp_mean_weights(x1, dx, nx, dom_lo, dom_hi, xmin, xmax):
|
| 360 |
+
"""Sampled_getMean(..., interpolate=true) as a linear functional.
|
| 361 |
+
|
| 362 |
+
Returns ``(indices, weights)`` (0-based) with weights already divided by
|
| 363 |
+
the definition range, for a sampled function whose values are all
|
| 364 |
+
defined. ``None`` when the mean is undefined.
|
| 365 |
+
"""
|
| 366 |
+
xmin, xmax = _autowindow(dom_lo, dom_hi, xmin, xmax)
|
| 367 |
+
r = _intersect(dom_lo, dom_hi, xmin, xmax)
|
| 368 |
+
if r is None:
|
| 369 |
+
return None
|
| 370 |
+
xmin, xmax = r
|
| 371 |
+
w: dict[int, float] = {}
|
| 372 |
+
rng = 0.0
|
| 373 |
+
|
| 374 |
+
def add(i, c):
|
| 375 |
+
w[i] = w.get(i, 0.0) + c
|
| 376 |
+
|
| 377 |
+
imin, imax, count = window_samples(x1, dx, nx, xmin, xmax)
|
| 378 |
+
if count > 0:
|
| 379 |
+
left_edge = x1 - 0.5 * dx
|
| 380 |
+
right_edge = left_edge + nx * dx
|
| 381 |
+
for i in range(imin, imax + 1):
|
| 382 |
+
add(i, 1.0)
|
| 383 |
+
rng += count
|
| 384 |
+
if xmin > left_edge:
|
| 385 |
+
phase = (fma(imin - 1, dx, x1) - xmin) / dx
|
| 386 |
+
rng -= 0.5
|
| 387 |
+
add(imin, -0.5)
|
| 388 |
+
if imin - 1 >= 1:
|
| 389 |
+
rng += phase
|
| 390 |
+
add(imin, phase * (1.0 - 0.5 * phase))
|
| 391 |
+
add(imin - 1, phase * 0.5 * phase)
|
| 392 |
+
else:
|
| 393 |
+
phase = min(phase, 0.5)
|
| 394 |
+
rng += phase
|
| 395 |
+
add(imin, phase)
|
| 396 |
+
if xmax < right_edge:
|
| 397 |
+
phase = (xmax - index_to_x(x1, dx, imax)) / dx
|
| 398 |
+
rng -= 0.5
|
| 399 |
+
add(imax, -0.5)
|
| 400 |
+
if imax + 1 <= nx:
|
| 401 |
+
rng += phase
|
| 402 |
+
add(imax, phase * (1.0 - 0.5 * phase))
|
| 403 |
+
add(imax + 1, phase * 0.5 * phase)
|
| 404 |
+
else:
|
| 405 |
+
phase = min(phase, 0.5)
|
| 406 |
+
rng += phase
|
| 407 |
+
add(imax, phase)
|
| 408 |
+
else:
|
| 409 |
+
# no sample centres between xmin and xmax: imin == imax + 1 here
|
| 410 |
+
lv_i, rv_i = imax, imin
|
| 411 |
+
p1 = (xmin - index_to_x(x1, dx, imax)) / dx
|
| 412 |
+
p2 = (xmax - index_to_x(x1, dx, imax)) / dx
|
| 413 |
+
if imin == imax + 1:
|
| 414 |
+
ldef, rdef = 1 <= lv_i <= nx, 1 <= rv_i <= nx
|
| 415 |
+
if ldef and rdef:
|
| 416 |
+
rng += p2 - p1
|
| 417 |
+
c = 0.5 * (p1 + p2)
|
| 418 |
+
add(lv_i, (p2 - p1) * (1.0 - c))
|
| 419 |
+
add(rv_i, (p2 - p1) * c)
|
| 420 |
+
elif ldef and p1 < 0.5:
|
| 421 |
+
p2 = min(p2, 0.5)
|
| 422 |
+
rng += p2 - p1
|
| 423 |
+
add(lv_i, p2 - p1)
|
| 424 |
+
elif rdef and p2 > 0.5:
|
| 425 |
+
p1 = max(p1, 0.5)
|
| 426 |
+
rng += p2 - p1
|
| 427 |
+
add(rv_i, p2 - p1)
|
| 428 |
+
if rng <= 0.0:
|
| 429 |
+
return None
|
| 430 |
+
idx = np.fromiter(w.keys(), dtype=np.int64) - 1
|
| 431 |
+
wt = np.fromiter(w.values(), dtype=np.float64) / rng
|
| 432 |
+
return idx, wt
|
| 433 |
+
|
| 434 |
+
|
| 435 |
+
# --------------------------------------------------------------------------
|
| 436 |
+
# Sound operations
|
| 437 |
+
# --------------------------------------------------------------------------
|
| 438 |
+
|
| 439 |
+
def _to_old_order(spec: np.ndarray, n: int) -> np.ndarray:
|
| 440 |
+
"""numpy rfft -> Praat NUMrealft order [DC, Nyq, re1, im1, ...] (n even)."""
|
| 441 |
+
p = np.empty(n)
|
| 442 |
+
p[0] = spec[0].real
|
| 443 |
+
p[1] = spec[n // 2].real
|
| 444 |
+
p[2::2] = spec[1:n // 2].real
|
| 445 |
+
p[3::2] = spec[1:n // 2].imag
|
| 446 |
+
return p
|
| 447 |
+
|
| 448 |
+
|
| 449 |
+
def _from_old_order(p: np.ndarray, n: int) -> np.ndarray:
|
| 450 |
+
spec = np.empty(n // 2 + 1, dtype=np.complex128)
|
| 451 |
+
spec[0] = p[0]
|
| 452 |
+
spec[n // 2] = p[1]
|
| 453 |
+
spec[1:n // 2] = p[2::2] + 1j * p[3::2]
|
| 454 |
+
return spec
|
| 455 |
+
|
| 456 |
+
|
| 457 |
+
def resample(me: Sound, fs: float, precision: int) -> Sound:
|
| 458 |
+
"""Sound_resample."""
|
| 459 |
+
upfactor = fs * me.dx
|
| 460 |
+
if abs(upfactor - 2.0) < 1e-6:
|
| 461 |
+
raise NotImplementedError("Sound_upsample path (exact 2x) not ported")
|
| 462 |
+
if abs(upfactor - 1.0) < 1e-6:
|
| 463 |
+
return me.copy()
|
| 464 |
+
n = iround((me.xmax - me.xmin) * fs)
|
| 465 |
+
if n < 1:
|
| 466 |
+
raise PraatError("The resampled Sound would have no samples.")
|
| 467 |
+
src = me
|
| 468 |
+
if upfactor < 1.0:
|
| 469 |
+
anti, nfft = 1000, 1
|
| 470 |
+
while nfft < me.nx + anti * 2:
|
| 471 |
+
nfft *= 2
|
| 472 |
+
data = np.zeros(nfft)
|
| 473 |
+
data[anti:anti + me.nx] = me.z
|
| 474 |
+
p = _to_old_order(np.fft.rfft(data), nfft)
|
| 475 |
+
p[math.floor(upfactor * nfft) - 1:] = 0.0
|
| 476 |
+
p[1] = 0.0
|
| 477 |
+
back = np.fft.irfft(_from_old_order(p, nfft), nfft)
|
| 478 |
+
src = Sound(back[anti:anti + me.nx].copy(), me.xmin, me.xmax, me.x1, me.dx)
|
| 479 |
+
dxn = 1.0 / fs
|
| 480 |
+
x1n = 0.5 * (me.xmin + me.xmax - (n - 1) / fs)
|
| 481 |
+
x = x1n + np.arange(n) * dxn
|
| 482 |
+
index = (x - src.x1) / src.dx + 1.0
|
| 483 |
+
if precision <= 1:
|
| 484 |
+
left = np.floor(index).astype(np.int64)
|
| 485 |
+
frac = index - left
|
| 486 |
+
ok = (left >= 1) & (left < src.nx)
|
| 487 |
+
z = np.zeros(n)
|
| 488 |
+
li = left[ok]
|
| 489 |
+
z[ok] = (1 - frac[ok]) * src.z[li - 1] + frac[ok] * src.z[li]
|
| 490 |
+
else:
|
| 491 |
+
z = interpolate_sinc(src.z, index, precision)
|
| 492 |
+
return Sound(z, me.xmin, me.xmax, x1n, dxn)
|
| 493 |
+
|
| 494 |
+
|
| 495 |
+
def extract_part(me: Sound, tmin, tmax, preserve_times: bool) -> Sound:
|
| 496 |
+
"""Sound_extractPart, rectangular window, relative width 1."""
|
| 497 |
+
tmin, tmax = _autowindow(me.xmin, me.xmax, tmin, tmax)
|
| 498 |
+
itmin = 1 + math.ceil((tmin - me.x1) / me.dx)
|
| 499 |
+
itmax = 1 + math.floor((tmax - me.x1) / me.dx)
|
| 500 |
+
if itmax < itmin:
|
| 501 |
+
raise PraatError("Extracted Sound would contain no samples.")
|
| 502 |
+
n = itmax - itmin + 1
|
| 503 |
+
z = np.zeros(n)
|
| 504 |
+
lo, hi = max(1, itmin), min(itmax, me.nx)
|
| 505 |
+
if hi >= lo:
|
| 506 |
+
z[lo - itmin:hi - itmin + 1] = me.z[lo - 1:hi]
|
| 507 |
+
x1 = index_to_x(me.x1, me.dx, itmin)
|
| 508 |
+
xmin, xmax = tmin, tmax
|
| 509 |
+
if not preserve_times:
|
| 510 |
+
xmin, xmax, x1 = 0.0, tmax - tmin, x1 - tmin
|
| 511 |
+
return Sound(z, xmin, xmax, x1, me.dx)
|
| 512 |
+
|
| 513 |
+
|
| 514 |
+
def pre_emphasis(me: Sound, frequency: float) -> None:
|
| 515 |
+
"""Sound_preEmphasis (dwtools version), in place."""
|
| 516 |
+
if frequency >= 0.5 / me.dx:
|
| 517 |
+
return
|
| 518 |
+
c = math.exp(-2.0 * math.pi * frequency * me.dx)
|
| 519 |
+
me.z[1:] = me.z[1:] - c * me.z[:-1]
|
| 520 |
+
|
| 521 |
+
|
| 522 |
+
def gaussian_window(n: int) -> np.ndarray:
|
| 523 |
+
"""Sound_createGaussian samples for ``n`` points."""
|
| 524 |
+
i = np.arange(1, n + 1, dtype=np.float64)
|
| 525 |
+
imid = 0.5 * (n + 1)
|
| 526 |
+
edge = math.exp(-12.0)
|
| 527 |
+
return (np.exp(-48.0 * (i - imid) * (i - imid) / (n + 1) / (n + 1)) - edge) / (1 - edge)
|
| 528 |
+
|
| 529 |
+
|
| 530 |
+
def into_sound_indices(me: Sound, start_time: float, n: int):
|
| 531 |
+
"""Sound_into_Sound: 1-based source indices for ``n`` target samples."""
|
| 532 |
+
index = x_to_nearest_index(me.x1, me.dx, start_time)
|
| 533 |
+
return index + np.arange(n)
|
| 534 |
+
|
| 535 |
+
|
| 536 |
+
def gather(z: np.ndarray, idx: np.ndarray) -> np.ndarray:
|
| 537 |
+
"""z[idx] with 1-based ``idx``; zero outside 1..nx."""
|
| 538 |
+
ok = (idx >= 1) & (idx <= z.size)
|
| 539 |
+
out = np.zeros(idx.shape)
|
| 540 |
+
out[ok] = z[idx[ok] - 1]
|
| 541 |
+
return out
|
| 542 |
+
|
| 543 |
+
|
| 544 |
+
@dataclass
|
| 545 |
+
class Spectrum:
|
| 546 |
+
re: np.ndarray
|
| 547 |
+
im: np.ndarray
|
| 548 |
+
xmax: float
|
| 549 |
+
dx: float # x1 = xmin = 0
|
| 550 |
+
|
| 551 |
+
@property
|
| 552 |
+
def nx(self):
|
| 553 |
+
return self.re.size
|
| 554 |
+
|
| 555 |
+
|
| 556 |
+
def to_spectrum(me: Sound, fast: bool = True) -> Spectrum:
|
| 557 |
+
"""Sound_to_Spectrum (mono)."""
|
| 558 |
+
n = me.nx
|
| 559 |
+
if fast:
|
| 560 |
+
n = 2
|
| 561 |
+
while n < me.nx:
|
| 562 |
+
n *= 2
|
| 563 |
+
data = np.zeros(n)
|
| 564 |
+
data[:me.nx] = me.z
|
| 565 |
+
spec = np.fft.rfft(data)
|
| 566 |
+
return Spectrum(spec.real * me.dx, spec.imag * me.dx, 0.5 / me.dx, 1.0 / (me.dx * n))
|
| 567 |
+
|
| 568 |
+
|
| 569 |
+
def spectrum_to_sound(me: Spectrum) -> Sound:
|
| 570 |
+
"""Spectrum_to_Sound."""
|
| 571 |
+
last = (me.nx - 1) * me.dx
|
| 572 |
+
odd = me.im[-1] != 0.0 or me.xmax - last > 0.25 * me.dx
|
| 573 |
+
n = 2 * me.nx - (1 if odd else 2)
|
| 574 |
+
spec = me.re + 1j * me.im
|
| 575 |
+
amp = me.dx * n * np.fft.irfft(spec, n)
|
| 576 |
+
fs = n * me.dx
|
| 577 |
+
duration = 1.0 / me.dx
|
| 578 |
+
return Sound(amp, 0.0, duration, 0.5 / fs, 1.0 / fs)
|
| 579 |
+
|
| 580 |
+
|
| 581 |
+
# --------------------------------------------------------------------------
|
| 582 |
+
# PowerCepstrogram / PowerCepstrum (LPC/PowerCepstrogram.cpp, PowerCepstrum.cpp)
|
| 583 |
+
# --------------------------------------------------------------------------
|
| 584 |
+
|
| 585 |
+
@dataclass
|
| 586 |
+
class Cepstrogram:
|
| 587 |
+
z: np.ndarray # (nq, nFrames), power values
|
| 588 |
+
t1: float
|
| 589 |
+
dt: float
|
| 590 |
+
dq: float # q1 = 0, qmax = (nq - 1) * dq
|
| 591 |
+
|
| 592 |
+
|
| 593 |
+
def to_power_cepstrogram(me: Sound, pitch_floor=60.0, dt=0.002, fmax=5000.0, preemph=50.0) -> Cepstrogram:
|
| 594 |
+
"""Sound_to_PowerCepstrogram."""
|
| 595 |
+
window_duration = 2.0 * (3.0 / pitch_floor)
|
| 596 |
+
if window_duration > me.dx * me.nx:
|
| 597 |
+
window_duration = me.dx * me.nx
|
| 598 |
+
fs = 2 * fmax
|
| 599 |
+
sound = resample(me, fs, 50)
|
| 600 |
+
pre_emphasis(sound, preemph)
|
| 601 |
+
n_frames, t1 = short_term_analysis(me.x1, me.dx, me.nx, window_duration, dt)
|
| 602 |
+
nsf = int(c_round(window_duration * fs))
|
| 603 |
+
window = gaussian_window(nsf)
|
| 604 |
+
nfft = 2
|
| 605 |
+
while nfft < nsf:
|
| 606 |
+
nfft *= 2
|
| 607 |
+
nq = nfft // 2 + 1
|
| 608 |
+
qmax = 0.5 * nfft / fs
|
| 609 |
+
times = index_to_x_array(t1, dt, n_frames)
|
| 610 |
+
starts = np.array([x_to_nearest_index(sound.x1, sound.dx, t - window_duration / 2) for t in times])
|
| 611 |
+
idx = starts[:, None] + np.arange(nsf)[None, :]
|
| 612 |
+
frames = gather(sound.z, idx)
|
| 613 |
+
frames -= frames.mean(axis=1, keepdims=True)
|
| 614 |
+
frames *= window
|
| 615 |
+
spec = np.fft.rfft(frames, nfft, axis=1) * (1.0 / fs) # Sound_to_Spectrum scaling dx
|
| 616 |
+
logspec = np.log(spec.real ** 2 + spec.imag ** 2 + 1e-300)
|
| 617 |
+
dx_spec = fs / nfft
|
| 618 |
+
ceps = dx_spec * nfft * np.fft.irfft(logspec, nfft, axis=1) # Spectrum_to_Sound
|
| 619 |
+
z = (ceps[:, :nq] ** 2).T.copy()
|
| 620 |
+
return Cepstrogram(z, t1, dt, qmax / (nq - 1))
|
| 621 |
+
|
| 622 |
+
|
| 623 |
+
def _db(z):
|
| 624 |
+
return 10.0 * np.log10(z + 1e-30)
|
| 625 |
+
|
| 626 |
+
|
| 627 |
+
def theil_fit(x: np.ndarray, y: np.ndarray):
|
| 628 |
+
"""NUMlineFit_theil, incomplete method; ``y`` may be (n,) or (n, k)."""
|
| 629 |
+
n = x.size
|
| 630 |
+
if n == 1:
|
| 631 |
+
return np.zeros(y.shape[1:]), y[0]
|
| 632 |
+
if n == 2:
|
| 633 |
+
m = (y[1] - y[0]) / (x[1] - x[0])
|
| 634 |
+
return m, y[0] - m * x[0]
|
| 635 |
+
half = n // 2
|
| 636 |
+
n2 = half + 1 if n % 2 == 1 else half
|
| 637 |
+
yy = y if y.ndim > 1 else y[:, None]
|
| 638 |
+
mbs = (yy[n2:n2 + half] - yy[:half]) / (x[n2:n2 + half] - x[:half])[:, None]
|
| 639 |
+
m = _quantile_half(np.sort(mbs, axis=0))
|
| 640 |
+
b = _quantile_half(np.sort(yy - m[None, :] * x[:, None], axis=0))
|
| 641 |
+
if y.ndim == 1:
|
| 642 |
+
return float(m[0]), float(b[0])
|
| 643 |
+
return m, b
|
| 644 |
+
|
| 645 |
+
|
| 646 |
+
def _quantile_half(a: np.ndarray) -> np.ndarray:
|
| 647 |
+
"""NUMquantile(sorted a, 0.5) along axis 0."""
|
| 648 |
+
n = a.shape[0]
|
| 649 |
+
if n == 1:
|
| 650 |
+
return a[0]
|
| 651 |
+
place = 0.5 * n + 0.5
|
| 652 |
+
left = min(max(int(math.floor(place)), 1), n - 1)
|
| 653 |
+
lo, hi = a[left - 1], a[left]
|
| 654 |
+
return np.where(hi == lo, lo, lo + (place - left) * (hi - lo))
|
| 655 |
+
|
| 656 |
+
|
| 657 |
+
def _q_axis(nq, dq):
|
| 658 |
+
return np.arange(nq) * dq
|
| 659 |
+
|
| 660 |
+
|
| 661 |
+
def subtract_trend_robust(z: np.ndarray, dq: float) -> np.ndarray:
|
| 662 |
+
"""PowerCepstrum_subtractTrend(0.001, 0, LINEAR, ROBUST_FAST) per column.
|
| 663 |
+
|
| 664 |
+
``qendFit <= qstartFit`` makes Praat fit over the whole quefrency domain.
|
| 665 |
+
"""
|
| 666 |
+
nq = z.shape[0]
|
| 667 |
+
q = _q_axis(nq, dq)
|
| 668 |
+
y = _db(z)
|
| 669 |
+
m, b = theil_fit(q, y)
|
| 670 |
+
qq = q.copy()
|
| 671 |
+
qq[0] = 0.5 * dq
|
| 672 |
+
diff = y - (np.multiply.outer(qq, m) + b)
|
| 673 |
+
diff = np.maximum(diff, 0.0)
|
| 674 |
+
return np.exp(diff * math.log(10.0) / 10.0) - 1e-30
|
| 675 |
+
|
| 676 |
+
|
| 677 |
+
def cepstrum_peak(zcol: np.ndarray, dq: float, pitch_floor: float, pitch_ceiling: float, interp=PEAK_PARABOLIC):
|
| 678 |
+
"""PowerCepstrum_getMaximumAndQuefrency -> (peak dB, quefrency)."""
|
| 679 |
+
y = _db(zcol)
|
| 680 |
+
qmax = (zcol.size - 1) * dq
|
| 681 |
+
return vector_extremum_and_x(y, 0.0, dq, 0.0, qmax, 1.0 / pitch_ceiling, 1.0 / pitch_floor, interp, True)
|
| 682 |
+
|
| 683 |
+
|
| 684 |
+
def _smooth_matrix(n, x1, dx, dom_lo, dom_hi, half):
|
| 685 |
+
"""Dense matrix W with (W @ v)[i] = Sampled_getMean(v, x_i - half, x_i + half, interpolate)."""
|
| 686 |
+
from scipy import sparse
|
| 687 |
+
|
| 688 |
+
rows, cols, vals = [], [], []
|
| 689 |
+
for i in range(n):
|
| 690 |
+
xm = index_to_x(x1, dx, i + 1)
|
| 691 |
+
r = interp_mean_weights(x1, dx, n, dom_lo, dom_hi, xm - half, xm + half)
|
| 692 |
+
if r is None:
|
| 693 |
+
raise PraatError("undefined smoothing mean")
|
| 694 |
+
rows.append(np.full(r[0].size, i))
|
| 695 |
+
cols.append(r[0])
|
| 696 |
+
vals.append(r[1])
|
| 697 |
+
return sparse.csr_matrix((np.concatenate(vals), (np.concatenate(rows), np.concatenate(cols))), shape=(n, n))
|
| 698 |
+
|
| 699 |
+
|
| 700 |
+
def cpps(cg: Cepstrogram, xmin: float, xmax: float, time_window=0.01, quef_window=0.001,
|
| 701 |
+
pitch_floor=60.0, pitch_ceiling=330.0) -> float:
|
| 702 |
+
"""PowerCepstrogram_getCPPS(no tilt before smoothing, parabolic, 0.001-0 Straight Robust)."""
|
| 703 |
+
z = cg.z
|
| 704 |
+
nq, nt = z.shape
|
| 705 |
+
ymax = (nq - 1) * cg.dq
|
| 706 |
+
if math.floor(time_window / cg.dt) > 1:
|
| 707 |
+
Wt = _smooth_matrix(nt, cg.t1, cg.dt, xmin, xmax, 0.5 * time_window)
|
| 708 |
+
z = (Wt @ z.T).T
|
| 709 |
+
if math.floor(quef_window / cg.dq) > 1:
|
| 710 |
+
dqc = ymax / (nq - 1) # PowerCepstrum_create(ymax, ny)
|
| 711 |
+
if quef_window / dqc > 1.0:
|
| 712 |
+
Wq = _smooth_matrix(nq, 0.0, dqc, 0.0, ymax, 0.5 * quef_window)
|
| 713 |
+
z = np.asarray(Wq @ z)
|
| 714 |
+
dqc = ymax / (nq - 1)
|
| 715 |
+
q = _q_axis(nq, dqc)
|
| 716 |
+
m, b = theil_fit(q, _db(z))
|
| 717 |
+
vals = np.empty(nt)
|
| 718 |
+
for j in range(nt):
|
| 719 |
+
peak, qpeak = cepstrum_peak(z[:, j], dqc, pitch_floor, pitch_ceiling)
|
| 720 |
+
vals[j] = peak - (m[j] * qpeak + b[j])
|
| 721 |
+
return float(np.mean(vals))
|
| 722 |
+
|
| 723 |
+
|
| 724 |
+
# --------------------------------------------------------------------------
|
| 725 |
+
# Row-wise sinc interpolation and lock-step Brent (vectorised over problems)
|
| 726 |
+
# --------------------------------------------------------------------------
|
| 727 |
+
|
| 728 |
+
def interpolate_sinc_rows(Y: np.ndarray, rows: np.ndarray, x: np.ndarray, max_depth: int) -> np.ndarray:
|
| 729 |
+
"""NUM_interpolate_sinc of row ``rows[k]`` of ``Y`` at 1-based ``x[k]``."""
|
| 730 |
+
n = Y.shape[1]
|
| 731 |
+
x = np.asarray(x, dtype=np.float64)
|
| 732 |
+
rows = np.asarray(rows, dtype=np.int64)
|
| 733 |
+
out = np.empty(x.size)
|
| 734 |
+
ml = np.floor(x).astype(np.int64)
|
| 735 |
+
lo, hi = x < 1, x > n
|
| 736 |
+
out[lo] = Y[rows[lo], 0]
|
| 737 |
+
out[hi] = Y[rows[hi], -1]
|
| 738 |
+
exact = ~lo & ~hi & (x == ml)
|
| 739 |
+
out[exact] = Y[rows[exact], ml[exact] - 1]
|
| 740 |
+
rest = ~(lo | hi | exact)
|
| 741 |
+
depth = np.minimum(np.minimum(max_depth, ml), n - ml)
|
| 742 |
+
for d in np.unique(depth[rest]):
|
| 743 |
+
sel = np.nonzero(rest & (depth == d))[0]
|
| 744 |
+
xs, mls, rs = x[sel], ml[sel], rows[sel]
|
| 745 |
+
if d <= 0:
|
| 746 |
+
out[sel] = Y[rs, np.floor(xs + 0.5).astype(np.int64) - 1]
|
| 747 |
+
continue
|
| 748 |
+
if d == 1:
|
| 749 |
+
out[sel] = Y[rs, mls - 1] + (xs - mls) * (Y[rs, mls] - Y[rs, mls - 1])
|
| 750 |
+
continue
|
| 751 |
+
if d == 2:
|
| 752 |
+
yl, yr = Y[rs, mls - 1], Y[rs, mls]
|
| 753 |
+
dyl = 0.5 * (yr - Y[rs, mls - 2])
|
| 754 |
+
dyr = 0.5 * (Y[rs, mls + 1] - yl)
|
| 755 |
+
fil, fir = xs - mls, mls + 1 - xs
|
| 756 |
+
out[sel] = yl * fir + yr * fil - fil * fir * (0.5 * (dyr - dyl) + (fil - 0.5) * (dyl + dyr - 2 * (yr - yl)))
|
| 757 |
+
continue
|
| 758 |
+
d = int(d)
|
| 759 |
+
j = np.arange(d, dtype=np.float64)
|
| 760 |
+
sign = np.where(np.arange(d) % 2 == 0, 1.0, -1.0)
|
| 761 |
+
frac = xs - mls
|
| 762 |
+
a0 = math.pi * frac
|
| 763 |
+
aL = a0[:, None] + math.pi * j
|
| 764 |
+
wL = (0.5 * np.sin(a0))[:, None] * sign / aL * (1.0 + np.cos(aL / (frac + d)[:, None]))
|
| 765 |
+
a0r = math.pi * (mls + 1 - xs)
|
| 766 |
+
aR = a0r[:, None] + math.pi * j
|
| 767 |
+
wR = (0.5 * np.sin(a0r))[:, None] * sign / aR * (1.0 + np.cos(aR / (d + 1.0 - frac)[:, None]))
|
| 768 |
+
idxL = mls[:, None] - np.arange(d)
|
| 769 |
+
idxR = mls[:, None] + 1 + np.arange(d)
|
| 770 |
+
out[sel] = (Y[rs[:, None], idxL - 1] * wL).sum(axis=1) + (Y[rs[:, None], idxR - 1] * wR).sum(axis=1)
|
| 771 |
+
return out
|
| 772 |
+
|
| 773 |
+
|
| 774 |
+
def minimize_brent_many(f, a: np.ndarray, b: np.ndarray, tol: float):
|
| 775 |
+
"""NUMminimize_brent run in lock-step for many independent problems.
|
| 776 |
+
|
| 777 |
+
``f(x, idx)`` evaluates problems ``idx`` at points ``x``.
|
| 778 |
+
Returns ``(x, fx)`` arrays.
|
| 779 |
+
"""
|
| 780 |
+
a = np.asarray(a, dtype=np.float64).copy()
|
| 781 |
+
b = np.asarray(b, dtype=np.float64).copy()
|
| 782 |
+
m = a.size
|
| 783 |
+
allidx = np.arange(m)
|
| 784 |
+
v = a + _GOLDEN * (b - a)
|
| 785 |
+
fv = f(v, allidx)
|
| 786 |
+
x, w = v.copy(), v.copy()
|
| 787 |
+
fx, fw = fv.copy(), fv.copy()
|
| 788 |
+
active = np.ones(m, dtype=bool)
|
| 789 |
+
for _ in range(60):
|
| 790 |
+
idx = np.nonzero(active)[0]
|
| 791 |
+
if idx.size == 0:
|
| 792 |
+
break
|
| 793 |
+
ai, bi, xi, wi, vi = a[idx], b[idx], x[idx], w[idx], v[idx]
|
| 794 |
+
fxi, fwi, fvi = fx[idx], fw[idx], fv[idx]
|
| 795 |
+
middle = (ai + bi) / 2.0
|
| 796 |
+
tol_act = _SQRT_EPS * np.abs(xi) + tol / 3.0
|
| 797 |
+
done = np.abs(xi - middle) + (bi - ai) / 2.0 <= 2.0 * tol_act
|
| 798 |
+
active[idx[done]] = False
|
| 799 |
+
keep = ~done
|
| 800 |
+
idx = idx[keep]
|
| 801 |
+
if idx.size == 0:
|
| 802 |
+
break
|
| 803 |
+
ai, bi, xi, wi, vi = ai[keep], bi[keep], xi[keep], wi[keep], vi[keep]
|
| 804 |
+
fxi, fwi, fvi = fxi[keep], fwi[keep], fvi[keep]
|
| 805 |
+
middle, tol_act = middle[keep], tol_act[keep]
|
| 806 |
+
new_step = _GOLDEN * np.where(xi < middle, bi - xi, ai - xi)
|
| 807 |
+
tryp = np.abs(xi - wi) >= tol_act
|
| 808 |
+
with np.errstate(invalid="ignore", divide="ignore"):
|
| 809 |
+
t = (xi - wi) * (fxi - fvi)
|
| 810 |
+
q = (xi - vi) * (fxi - fwi)
|
| 811 |
+
p = (xi - vi) * q - (xi - wi) * t
|
| 812 |
+
q = 2.0 * (q - t)
|
| 813 |
+
p = np.where(q > 0.0, -p, p)
|
| 814 |
+
q = np.where(q > 0.0, q, -q)
|
| 815 |
+
acc = tryp & (np.abs(p) < np.abs(new_step * q)) & (p > q * (ai - xi + 2.0 * tol_act)) & (p < q * (bi - xi - 2.0 * tol_act))
|
| 816 |
+
new_step = np.where(acc, p / np.where(acc, q, 1.0), new_step)
|
| 817 |
+
small = np.abs(new_step) < tol_act
|
| 818 |
+
new_step = np.where(small, np.where(new_step > 0.0, tol_act, -tol_act), new_step)
|
| 819 |
+
tt = xi + new_step
|
| 820 |
+
ft = f(tt, idx)
|
| 821 |
+
better = ft <= fxi
|
| 822 |
+
# better: shrink around t
|
| 823 |
+
na = np.where(better, np.where(tt < xi, ai, xi), np.where(tt < xi, tt, ai))
|
| 824 |
+
nb = np.where(better, np.where(tt < xi, xi, bi), np.where(tt < xi, bi, tt))
|
| 825 |
+
c1 = ~better & ((ft <= fwi) | (wi == xi))
|
| 826 |
+
c2 = ~better & ~c1 & ((ft <= fvi) | (vi == xi) | (vi == wi))
|
| 827 |
+
nv = np.where(better, wi, np.where(c1, wi, np.where(c2, tt, vi)))
|
| 828 |
+
nfv = np.where(better, fwi, np.where(c1, fwi, np.where(c2, ft, fvi)))
|
| 829 |
+
nw = np.where(better, xi, np.where(c1, tt, wi))
|
| 830 |
+
nfw = np.where(better, fxi, np.where(c1, ft, fwi))
|
| 831 |
+
nx_ = np.where(better, tt, xi)
|
| 832 |
+
nfx = np.where(better, ft, fxi)
|
| 833 |
+
a[idx], b[idx], v[idx], fv[idx], w[idx], fw[idx], x[idx], fx[idx] = na, nb, nv, nfv, nw, nfw, nx_, nfx
|
| 834 |
+
return x, fx
|
inference/native_features/voice_port.py
ADDED
|
@@ -0,0 +1,554 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Pitch (ac), PointProcess (cc) and Voice report, ported from Praat 6.1.38.
|
| 2 |
+
|
| 3 |
+
Script calls reproduced (PART 2 of the ABI script):
|
| 4 |
+
|
| 5 |
+
To Pitch... 0 70 600
|
| 6 |
+
To PointProcess (cc)
|
| 7 |
+
Voice report... 0 0 70 600 1.3 1.6 0.03 0.45
|
| 8 |
+
|
| 9 |
+
then ``extractNumber`` on the report text, including its rounding to the
|
| 10 |
+
printed digits (Melder_percent / Melder_fixed / Melder_fixedExponent).
|
| 11 |
+
"""
|
| 12 |
+
|
| 13 |
+
from __future__ import annotations
|
| 14 |
+
|
| 15 |
+
import math
|
| 16 |
+
|
| 17 |
+
import numpy as np
|
| 18 |
+
|
| 19 |
+
from . import praat_port as P
|
| 20 |
+
from .praat_port import fma, index_to_x
|
| 21 |
+
|
| 22 |
+
_NAN = float("nan")
|
| 23 |
+
_LOG2E = 1.4426950408889634073599246810018921374266
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def _log2(x):
|
| 27 |
+
return math.log(x) * _LOG2E
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
# --------------------------------------------------------------------------
|
| 31 |
+
# Sound_to_Pitch_ac (Sound_to_Pitch.cpp) + Pitch_pathFinder (Pitch.cpp)
|
| 32 |
+
# --------------------------------------------------------------------------
|
| 33 |
+
|
| 34 |
+
class Pitch:
|
| 35 |
+
def __init__(self, xmin, xmax, nx, dx, x1, ceiling, freq, strength):
|
| 36 |
+
self.xmin, self.xmax, self.nx, self.dx, self.x1 = xmin, xmax, nx, dx, x1
|
| 37 |
+
self.ceiling = ceiling
|
| 38 |
+
self.freq = freq # frequency of candidate 1 per frame
|
| 39 |
+
self.strength = strength
|
| 40 |
+
|
| 41 |
+
def voiced(self, i): # 1-based
|
| 42 |
+
f = self.freq[i - 1]
|
| 43 |
+
return f > 0.0 and f < self.ceiling
|
| 44 |
+
|
| 45 |
+
def value_at_time(self, x):
|
| 46 |
+
"""Pitch_getValueAtTime(HERTZ, linear)."""
|
| 47 |
+
if x < self.xmin or x > self.xmax:
|
| 48 |
+
return _NAN
|
| 49 |
+
index_real = (x - self.x1) / self.dx + 1.0
|
| 50 |
+
left = math.floor(index_real)
|
| 51 |
+
phase = index_real - left
|
| 52 |
+
if phase < 0.5:
|
| 53 |
+
near, far = left, left + 1
|
| 54 |
+
else:
|
| 55 |
+
far, near = left, left + 1
|
| 56 |
+
phase = 1.0 - phase
|
| 57 |
+
if near < 1 or near > self.nx or not self.voiced(near):
|
| 58 |
+
return _NAN
|
| 59 |
+
nv = self.freq[near - 1]
|
| 60 |
+
if far < 1 or far > self.nx or not self.voiced(far):
|
| 61 |
+
return nv
|
| 62 |
+
fv = self.freq[far - 1]
|
| 63 |
+
return fma(phase, fv - nv, nv)
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
def short_term_analysis_fused(x1, dx, nx, window_duration, time_step):
|
| 67 |
+
"""Sampled_shortTermAnalysis with Clang FMA contraction of both sums."""
|
| 68 |
+
my_duration = dx * nx
|
| 69 |
+
if window_duration > my_duration:
|
| 70 |
+
raise P.PraatError("shorter than window length.")
|
| 71 |
+
n = math.floor((my_duration - window_duration) / time_step) + 1
|
| 72 |
+
our_mid = fma(0.5, my_duration, fma(-0.5, dx, x1))
|
| 73 |
+
thy_duration = n * time_step
|
| 74 |
+
return n, fma(0.5, time_step, fma(-0.5, thy_duration, our_mid))
|
| 75 |
+
|
| 76 |
+
|
| 77 |
+
def sound_to_pitch_ac(sound: P.Sound, time_step=0.0, minimum_pitch=70.0, ceiling=600.0,
|
| 78 |
+
periods_per_window=3.0, max_candidates=15, silence=0.03, voicing=0.45,
|
| 79 |
+
octave_cost=0.01, octave_jump=0.35, vuv_cost=0.14) -> Pitch:
|
| 80 |
+
z, dx, x1 = sound.z, sound.dx, sound.x1
|
| 81 |
+
nx = z.size
|
| 82 |
+
if max_candidates < ceiling / minimum_pitch:
|
| 83 |
+
max_candidates = math.floor(ceiling / minimum_pitch)
|
| 84 |
+
dt = time_step if time_step > 0.0 else periods_per_window / minimum_pitch / 4.0
|
| 85 |
+
duration = dx * nx
|
| 86 |
+
if minimum_pitch < periods_per_window / duration:
|
| 87 |
+
raise P.PraatError("minimum pitch too low for this Sound")
|
| 88 |
+
nsamp_period = math.floor(1.0 / dx / minimum_pitch)
|
| 89 |
+
halfnsamp_period = nsamp_period // 2 + 1
|
| 90 |
+
ceiling = min(ceiling, 0.5 / dx)
|
| 91 |
+
dt_window = periods_per_window / minimum_pitch
|
| 92 |
+
nsamp_window = math.floor(dt_window / dx)
|
| 93 |
+
halfnsamp_window = nsamp_window // 2 - 1
|
| 94 |
+
if halfnsamp_window < 2:
|
| 95 |
+
raise P.PraatError("Analysis window too short.")
|
| 96 |
+
nsamp_window = halfnsamp_window * 2
|
| 97 |
+
maximum_lag = min(math.floor(nsamp_window / periods_per_window) + 2, nsamp_window)
|
| 98 |
+
n_frames, t1 = short_term_analysis_fused(x1, dx, nx, dt_window, dt)
|
| 99 |
+
|
| 100 |
+
freq = np.zeros(n_frames)
|
| 101 |
+
strength = np.zeros(n_frames)
|
| 102 |
+
pitch = Pitch(sound.xmin, sound.xmax, n_frames, dt, t1, ceiling, freq, strength)
|
| 103 |
+
global_peak = float(np.max(np.abs(z - np.mean(z)))) if nx else 0.0
|
| 104 |
+
if global_peak == 0.0:
|
| 105 |
+
return pitch
|
| 106 |
+
|
| 107 |
+
nfft = 1
|
| 108 |
+
while nfft < nsamp_window * 1.5:
|
| 109 |
+
nfft *= 2
|
| 110 |
+
brent_ixmax = math.floor(nsamp_window * 0.5)
|
| 111 |
+
i = np.arange(1, nsamp_window + 1, dtype=np.float64)
|
| 112 |
+
window = 0.5 - 0.5 * np.cos(i * 2 * math.pi / (nsamp_window + 1))
|
| 113 |
+
wr = np.fft.irfft(np.abs(np.fft.rfft(window, nfft)) ** 2, nfft)
|
| 114 |
+
window_r = wr / wr[0]
|
| 115 |
+
|
| 116 |
+
times = np.array([index_to_x(t1, dt, k) for k in range(1, n_frames + 1)])
|
| 117 |
+
left = np.floor((times - x1) / dx + 1.0).astype(np.int64)
|
| 118 |
+
right = left + 1
|
| 119 |
+
csum = np.concatenate(([0.0], np.cumsum(z)))
|
| 120 |
+
local_mean = (csum[left + nsamp_period] - csum[right - nsamp_period - 1]) / (2 * nsamp_period)
|
| 121 |
+
start = right - halfnsamp_window
|
| 122 |
+
idx = start[:, None] + np.arange(nsamp_window)[None, :]
|
| 123 |
+
frames = (z[idx - 1] - local_mean[:, None]) * window[None, :]
|
| 124 |
+
lp_lo = max(halfnsamp_window + 1 - halfnsamp_period, 1)
|
| 125 |
+
lp_hi = min(halfnsamp_window + halfnsamp_period, nsamp_window)
|
| 126 |
+
local_peak = np.max(np.abs(frames[:, lp_lo - 1:lp_hi]), axis=1)
|
| 127 |
+
intensity = np.where(local_peak > global_peak, 1.0, local_peak / global_peak)
|
| 128 |
+
ac = np.fft.irfft(np.abs(np.fft.rfft(frames, nfft, axis=1)) ** 2, nfft, axis=1)
|
| 129 |
+
with np.errstate(invalid="ignore", divide="ignore"):
|
| 130 |
+
rpos = ac[:, 1:brent_ixmax + 1] / (ac[:, :1] * window_r[None, 1:brent_ixmax + 1])
|
| 131 |
+
B = brent_ixmax
|
| 132 |
+
R = np.concatenate((rpos[:, ::-1], np.ones((n_frames, 1)), rpos), axis=1) # column = lag + B
|
| 133 |
+
|
| 134 |
+
# First pass: local maxima of r, parabolic frequency, sinc-30 strength.
|
| 135 |
+
lag_hi = min(maximum_lag, brent_ixmax) - 1 # i < maximumLag && i < brent_ixmax
|
| 136 |
+
lags = np.arange(2, lag_hi + 1)
|
| 137 |
+
rc, rl, rr = R[:, lags + B], R[:, lags - 1 + B], R[:, lags + 1 + B]
|
| 138 |
+
is_peak = (rc > 0.5 * voicing) & (rc > rl) & (rc >= rr)
|
| 139 |
+
is_peak &= (local_peak > 0.0)[:, None]
|
| 140 |
+
fi, li = np.nonzero(is_peak)
|
| 141 |
+
lag = lags[li]
|
| 142 |
+
r0, rm, rp = R[fi, lag + B], R[fi, lag - 1 + B], R[fi, lag + 1 + B]
|
| 143 |
+
dr = 0.5 * (rp - rm)
|
| 144 |
+
d2r = 2.0 * r0 - rm - rp
|
| 145 |
+
f_max = 1.0 / dx / (lag + dr / d2r)
|
| 146 |
+
s_max = P.interpolate_sinc_rows(R, fi, 1.0 / dx / f_max + B + 1, 30)
|
| 147 |
+
s_max = np.where(s_max > 1.0, 1.0 / s_max, s_max)
|
| 148 |
+
|
| 149 |
+
cand_f = [[0.0] for _ in range(n_frames)]
|
| 150 |
+
cand_s = [[0.0] for _ in range(n_frames)]
|
| 151 |
+
cand_i = [[0] for _ in range(n_frames)]
|
| 152 |
+
for k in range(fi.size):
|
| 153 |
+
f = int(fi[k])
|
| 154 |
+
cf, cs, ci = cand_f[f], cand_s[f], cand_i[f]
|
| 155 |
+
fr, st = float(f_max[k]), float(s_max[k])
|
| 156 |
+
if len(cf) < max_candidates:
|
| 157 |
+
cf.append(fr), cs.append(st), ci.append(int(lag[k]))
|
| 158 |
+
continue
|
| 159 |
+
weakest, place = 2.0, 0
|
| 160 |
+
for iw in range(1, max_candidates):
|
| 161 |
+
ls = cs[iw] - octave_cost * _log2(minimum_pitch / cf[iw])
|
| 162 |
+
if ls < weakest:
|
| 163 |
+
weakest, place = ls, iw
|
| 164 |
+
if st - octave_cost * _log2(minimum_pitch / fr) <= weakest:
|
| 165 |
+
place = 0
|
| 166 |
+
if place:
|
| 167 |
+
cf[place], cs[place], ci[place] = fr, st, int(lag[k])
|
| 168 |
+
|
| 169 |
+
# Second pass: Brent on sinc-70 interpolation around each candidate lag.
|
| 170 |
+
pf = np.array([f for f in range(n_frames) for _ in cand_i[f][1:]], dtype=np.int64)
|
| 171 |
+
pl = np.array([v for f in range(n_frames) for v in cand_i[f][1:]], dtype=np.float64)
|
| 172 |
+
if pf.size:
|
| 173 |
+
ixmid = pl + B + 1
|
| 174 |
+
|
| 175 |
+
def neg_sinc(xx, sel):
|
| 176 |
+
return -P.interpolate_sinc_rows(R, pf[sel], xx, 70)
|
| 177 |
+
|
| 178 |
+
xr, fr = P.minimize_brent_many(neg_sinc, ixmid - 1, ixmid + 1, 1e-10)
|
| 179 |
+
ymid = -fr
|
| 180 |
+
xmid = xr - B - 1
|
| 181 |
+
new_f = 1.0 / dx / xmid
|
| 182 |
+
new_s = np.where(ymid > 1.0, 1.0 / ymid, ymid)
|
| 183 |
+
k = 0
|
| 184 |
+
for f in range(n_frames):
|
| 185 |
+
for c in range(1, len(cand_f[f])):
|
| 186 |
+
cand_f[f][c], cand_s[f][c] = float(new_f[k]), float(new_s[k])
|
| 187 |
+
k += 1
|
| 188 |
+
|
| 189 |
+
_path_finder(pitch, cand_f, cand_s, intensity, silence, voicing, octave_cost,
|
| 190 |
+
octave_jump, vuv_cost, ceiling)
|
| 191 |
+
return pitch
|
| 192 |
+
|
| 193 |
+
|
| 194 |
+
def _path_finder(pitch, cand_f, cand_s, intensity, silence, voicing, octave_cost,
|
| 195 |
+
octave_jump, vuv_cost, ceiling):
|
| 196 |
+
n = pitch.nx
|
| 197 |
+
maxn = max(len(c) for c in cand_f)
|
| 198 |
+
corr = 0.01 / pitch.dx
|
| 199 |
+
octave_jump *= corr
|
| 200 |
+
vuv_cost *= corr
|
| 201 |
+
F = np.zeros((n, maxn))
|
| 202 |
+
D = np.full((n, maxn), -np.inf)
|
| 203 |
+
nc = np.array([len(c) for c in cand_f])
|
| 204 |
+
for f in range(n):
|
| 205 |
+
F[f, :nc[f]] = cand_f[f]
|
| 206 |
+
unv = (2.0 - intensity[f] / (silence / (1.0 + voicing))) if silence > 0 else 0.0
|
| 207 |
+
unv = voicing + max(0.0, unv)
|
| 208 |
+
for c in range(nc[f]):
|
| 209 |
+
fr = cand_f[f][c]
|
| 210 |
+
voiceless = not (fr > 0.0 and fr < ceiling)
|
| 211 |
+
D[f, c] = unv if voiceless else cand_s[f][c] - octave_cost * _log2(ceiling / fr)
|
| 212 |
+
V = (F > 0.0) & (F < ceiling)
|
| 213 |
+
psi = np.zeros((n, maxn), dtype=np.int64)
|
| 214 |
+
with np.errstate(divide="ignore", invalid="ignore"):
|
| 215 |
+
logF = np.where(V, np.log(np.where(V, F, 1.0)) * _LOG2E, 0.0)
|
| 216 |
+
for f in range(1, n):
|
| 217 |
+
prev = D[f - 1]
|
| 218 |
+
m1, m2 = nc[f - 1], nc[f]
|
| 219 |
+
v1, v2 = V[f - 1, :m1], V[f, :m2]
|
| 220 |
+
# transition cost [icand2, icand1]
|
| 221 |
+
both = v2[:, None] & v1[None, :]
|
| 222 |
+
with np.errstate(divide="ignore", invalid="ignore"):
|
| 223 |
+
jump = np.abs((np.log(F[f - 1, :m1][None, :] / np.where(v2, F[f, :m2], 1.0)[:, None])) * _LOG2E)
|
| 224 |
+
cost = np.where(both, octave_jump * jump, np.where(v2[:, None] ^ v1[None, :], vuv_cost, 0.0))
|
| 225 |
+
val = (prev[None, :m1] - cost) + D[f, :m2][:, None]
|
| 226 |
+
place = np.argmax(val, axis=1)
|
| 227 |
+
D[f, :m2] = val[np.arange(m2), place]
|
| 228 |
+
psi[f, :m2] = place
|
| 229 |
+
place = 0
|
| 230 |
+
best = D[n - 1, 0]
|
| 231 |
+
for c in range(1, nc[n - 1]):
|
| 232 |
+
if D[n - 1, c] > best:
|
| 233 |
+
place, best = c, D[n - 1, c]
|
| 234 |
+
for f in range(n - 1, -1, -1):
|
| 235 |
+
pitch.freq[f] = cand_f[f][place]
|
| 236 |
+
pitch.strength[f] = cand_s[f][place]
|
| 237 |
+
cand_f[f][0], cand_f[f][place] = cand_f[f][place], cand_f[f][0]
|
| 238 |
+
cand_s[f][0], cand_s[f][place] = cand_s[f][place], cand_s[f][0]
|
| 239 |
+
place = psi[f, place]
|
| 240 |
+
|
| 241 |
+
|
| 242 |
+
# --------------------------------------------------------------------------
|
| 243 |
+
# Sound_Pitch_to_PointProcess_cc (Pitch_to_PointProcess.cpp)
|
| 244 |
+
# --------------------------------------------------------------------------
|
| 245 |
+
|
| 246 |
+
def _voiced_interval_after(pitch: Pitch, after):
|
| 247 |
+
ileft = math.ceil((after - pitch.x1) / pitch.dx + 1.0)
|
| 248 |
+
if ileft > pitch.nx:
|
| 249 |
+
return None
|
| 250 |
+
ileft = max(ileft, 1)
|
| 251 |
+
while ileft <= pitch.nx and not pitch.voiced(ileft):
|
| 252 |
+
ileft += 1
|
| 253 |
+
if ileft > pitch.nx:
|
| 254 |
+
return None
|
| 255 |
+
iright = ileft
|
| 256 |
+
while iright <= pitch.nx and pitch.voiced(iright):
|
| 257 |
+
iright += 1
|
| 258 |
+
iright -= 1
|
| 259 |
+
tleft = index_to_x(pitch.x1, pitch.dx, ileft) - 0.5 * pitch.dx
|
| 260 |
+
tright = index_to_x(pitch.x1, pitch.dx, iright) + 0.5 * pitch.dx
|
| 261 |
+
if tleft >= pitch.xmax - 0.5 * pitch.dx:
|
| 262 |
+
return None
|
| 263 |
+
tleft = max(tleft, pitch.xmin)
|
| 264 |
+
tright = min(tright, pitch.xmax)
|
| 265 |
+
if tright <= after:
|
| 266 |
+
return None
|
| 267 |
+
return tleft, tright
|
| 268 |
+
|
| 269 |
+
|
| 270 |
+
def _find_extremum_abs(s: P.Sound, tmin, tmax):
|
| 271 |
+
"""Sound_findExtremum(includeMaxima = includeMinima = true)."""
|
| 272 |
+
imin = math.floor((tmin - s.x1) / s.dx + 1.0)
|
| 273 |
+
imax = math.ceil((tmax - s.x1) / s.dx + 1.0)
|
| 274 |
+
imin, imax = max(imin, 1), min(imax, s.nx)
|
| 275 |
+
n = imax - imin + 1
|
| 276 |
+
y = s.z[imin - 1:imax] if n > 0 else np.empty(0)
|
| 277 |
+
if n <= 0:
|
| 278 |
+
iext = 0.0
|
| 279 |
+
elif n == 1:
|
| 280 |
+
iext = 1.0
|
| 281 |
+
elif n == 2:
|
| 282 |
+
a, b = abs(y[0]), abs(y[1])
|
| 283 |
+
iext = 1.0 if a > b else 2.0 if a < b else 1.5
|
| 284 |
+
else:
|
| 285 |
+
mn, mx = float(y.min()), float(y.max())
|
| 286 |
+
if mn == mx:
|
| 287 |
+
iext = 0.5 * (n + 1.0)
|
| 288 |
+
else:
|
| 289 |
+
ie = (int(np.argmin(y)) if abs(mn) > abs(mx) else int(np.argmax(y))) + 1
|
| 290 |
+
if ie == 1:
|
| 291 |
+
iext = 1.0
|
| 292 |
+
elif ie == n:
|
| 293 |
+
iext = float(n)
|
| 294 |
+
else:
|
| 295 |
+
vm, vl, vr = y[ie - 1], y[ie - 2], y[ie]
|
| 296 |
+
iext = ie + 0.5 * (vr - vl) / (2 * vm - vl - vr)
|
| 297 |
+
if iext != 0.0:
|
| 298 |
+
return fma(imin - 1 + iext - 1, s.dx, s.x1)
|
| 299 |
+
return (tmin + tmax) / 2
|
| 300 |
+
|
| 301 |
+
|
| 302 |
+
def _find_max_correlation(s: P.Sound, t1, window_length, tmin2, tmax2):
|
| 303 |
+
"""Sound_findMaximumCorrelation -> (correlation, tout or None, peak)."""
|
| 304 |
+
z, x1, dx, nx = s.z, s.x1, s.dx, s.nx
|
| 305 |
+
half = 0.5 * window_length
|
| 306 |
+
ileft1 = P.iround((t1 - half - x1) / dx + 1.0)
|
| 307 |
+
iright1 = P.iround((t1 + half - x1) / dx + 1.0)
|
| 308 |
+
l2min = math.floor((tmin2 - half - x1) / dx + 1.0)
|
| 309 |
+
l2max = math.ceil((tmax2 - half - x1) / dx + 1.0)
|
| 310 |
+
i1 = np.arange(ileft1, iright1 + 1)
|
| 311 |
+
shifts = np.arange(l2min, l2max + 1) - ileft1
|
| 312 |
+
i2 = i1[None, :] + shifts[:, None]
|
| 313 |
+
ok = (i1[None, :] >= 1) & (i1[None, :] <= nx) & (i2 >= 1) & (i2 <= nx)
|
| 314 |
+
a1 = np.where(ok, z[np.clip(i1, 1, nx)[None, :] - 1], 0.0)
|
| 315 |
+
a2 = np.where(ok, z[np.clip(i2, 1, nx) - 1], 0.0)
|
| 316 |
+
norm1 = (a1 * a1).sum(axis=1)
|
| 317 |
+
norm2 = (a2 * a2).sum(axis=1)
|
| 318 |
+
prod = (a1 * a2).sum(axis=1)
|
| 319 |
+
peaks = np.abs(a2).max(axis=1) if a2.shape[1] else np.zeros(shifts.size)
|
| 320 |
+
best, r1b, r3b, ir, peak = -1.0, _NAN, _NAN, _NAN, 0.0
|
| 321 |
+
r1 = r2 = r3 = 0.0
|
| 322 |
+
for k in range(shifts.size):
|
| 323 |
+
r1, r2 = r2, r3
|
| 324 |
+
r3 = prod[k] / math.sqrt(norm1[k] * norm2[k]) if prod[k] != 0.0 else 0.0
|
| 325 |
+
if r2 > best and r2 >= r1 and r2 >= r3:
|
| 326 |
+
r1b, best, r3b = r1, r2, r3
|
| 327 |
+
ir = (l2min + k) - 1
|
| 328 |
+
peak = float(peaks[k])
|
| 329 |
+
tout = None
|
| 330 |
+
if best > -1.0:
|
| 331 |
+
d2r = 2 * best - r1b - r3b
|
| 332 |
+
if d2r != 0.0:
|
| 333 |
+
drr = 0.5 * (r3b - r1b)
|
| 334 |
+
best += 0.5 * drr * drr / d2r
|
| 335 |
+
ir += drr / d2r
|
| 336 |
+
tout = fma(ir - ileft1, dx, t1)
|
| 337 |
+
return best, tout, peak
|
| 338 |
+
|
| 339 |
+
|
| 340 |
+
def _add_point(points: list, t):
|
| 341 |
+
"""PointProcess_addPoint (sorted, no duplicates)."""
|
| 342 |
+
import bisect
|
| 343 |
+
|
| 344 |
+
if not points or t >= points[-1]:
|
| 345 |
+
if not points or t != points[-1] or True:
|
| 346 |
+
# Praat appends even when equal to the last point
|
| 347 |
+
points.append(t)
|
| 348 |
+
return
|
| 349 |
+
k = bisect.bisect_right(points, t) # number of points <= t
|
| 350 |
+
if k == 0 or points[k - 1] != t:
|
| 351 |
+
points.insert(k, t)
|
| 352 |
+
|
| 353 |
+
|
| 354 |
+
def sound_pitch_to_point_process_cc(s: P.Sound, pitch: Pitch) -> np.ndarray:
|
| 355 |
+
points: list[float] = []
|
| 356 |
+
t = pitch.xmin
|
| 357 |
+
added_right = -1e308
|
| 358 |
+
gp = max(abs(float(s.z.min())), abs(float(s.z.max())))
|
| 359 |
+
while True:
|
| 360 |
+
iv = _voiced_interval_after(pitch, t)
|
| 361 |
+
if iv is None:
|
| 362 |
+
break
|
| 363 |
+
tleft, tright = iv
|
| 364 |
+
tmiddle = (tleft + tright) / 2.0
|
| 365 |
+
f0m = pitch.value_at_time(tmiddle)
|
| 366 |
+
tmax = _find_extremum_abs(s, tmiddle - 0.5 / f0m, tmiddle + 0.5 / f0m)
|
| 367 |
+
_add_point(points, tmax)
|
| 368 |
+
tsave = tmax
|
| 369 |
+
while True:
|
| 370 |
+
f0 = pitch.value_at_time(tmax)
|
| 371 |
+
if math.isnan(f0):
|
| 372 |
+
break
|
| 373 |
+
corr, tout, peak = _find_max_correlation(s, tmax, 1.0 / f0, tmax - 1.25 / f0, tmax - 0.8 / f0)
|
| 374 |
+
if tout is not None:
|
| 375 |
+
tmax = tout
|
| 376 |
+
if corr == -1.0:
|
| 377 |
+
tmax -= 1.0 / f0
|
| 378 |
+
if tmax < tleft:
|
| 379 |
+
if corr > 0.7 and peak > 0.023333 * gp and tmax - added_right > 0.8 / f0:
|
| 380 |
+
_add_point(points, tmax)
|
| 381 |
+
break
|
| 382 |
+
if corr > 0.3 and (peak == 0.0 or peak > 0.01 * gp):
|
| 383 |
+
if tmax - added_right > 0.8 / f0:
|
| 384 |
+
_add_point(points, tmax)
|
| 385 |
+
tmax = tsave
|
| 386 |
+
while True:
|
| 387 |
+
f0 = pitch.value_at_time(tmax)
|
| 388 |
+
if math.isnan(f0):
|
| 389 |
+
break
|
| 390 |
+
corr, tout, peak = _find_max_correlation(s, tmax, 1.0 / f0, tmax + 0.8 / f0, tmax + 1.25 / f0)
|
| 391 |
+
if tout is not None:
|
| 392 |
+
tmax = tout
|
| 393 |
+
if corr == -1.0:
|
| 394 |
+
tmax += 1.0 / f0
|
| 395 |
+
if tmax > tright:
|
| 396 |
+
if corr > 0.7 and peak > 0.023333 * gp:
|
| 397 |
+
_add_point(points, tmax)
|
| 398 |
+
added_right = tmax
|
| 399 |
+
break
|
| 400 |
+
if corr > 0.3 and (peak == 0.0 or peak > 0.01 * gp):
|
| 401 |
+
_add_point(points, tmax)
|
| 402 |
+
added_right = tmax
|
| 403 |
+
t = tright
|
| 404 |
+
return np.array(points)
|
| 405 |
+
|
| 406 |
+
|
| 407 |
+
# --------------------------------------------------------------------------
|
| 408 |
+
# Voice report quantities (VoiceAnalysis.cpp, PointProcess.cpp, AmplitudeTier.cpp)
|
| 409 |
+
# --------------------------------------------------------------------------
|
| 410 |
+
|
| 411 |
+
def _is_period(t, i, pmin, pmax, mpf):
|
| 412 |
+
"""PointProcess_isPeriod, ``i`` 0-based left point."""
|
| 413 |
+
nt = t.size
|
| 414 |
+
if i < 0 or i + 1 >= nt:
|
| 415 |
+
return False
|
| 416 |
+
if pmin == pmax:
|
| 417 |
+
return True
|
| 418 |
+
iv = t[i + 1] - t[i]
|
| 419 |
+
if iv <= 0.0 or iv < pmin or iv > pmax:
|
| 420 |
+
return False
|
| 421 |
+
prev = t[i] - t[i - 1] if i >= 1 else None
|
| 422 |
+
nxt = t[i + 2] - t[i + 1] if i + 2 < nt else None
|
| 423 |
+
pf = iv / prev if prev is not None and prev > 0.0 else None
|
| 424 |
+
nf = iv / nxt if nxt is not None and nxt > 0.0 else None
|
| 425 |
+
if pf is None and nf is None:
|
| 426 |
+
return True
|
| 427 |
+
if pf is not None and 0.0 < pf < 1.0:
|
| 428 |
+
pf = 1.0 / pf
|
| 429 |
+
if nf is not None and 0.0 < nf < 1.0:
|
| 430 |
+
nf = 1.0 / nf
|
| 431 |
+
if pf is not None and pf > mpf and nf is not None and nf > mpf:
|
| 432 |
+
return False
|
| 433 |
+
return True
|
| 434 |
+
|
| 435 |
+
|
| 436 |
+
def _period_stats(t, pmin, pmax, mpf):
|
| 437 |
+
periods = [t[i + 1] - t[i] for i in range(t.size - 1) if _is_period(t, i, pmin, pmax, mpf)]
|
| 438 |
+
n = len(periods)
|
| 439 |
+
mean = sum(periods) / n if n > 0 else _NAN
|
| 440 |
+
if n < 2:
|
| 441 |
+
return mean, _NAN
|
| 442 |
+
m = sum(periods) / n
|
| 443 |
+
return mean, math.sqrt(sum((p - m) * (p - m) for p in periods) / (n - 1))
|
| 444 |
+
|
| 445 |
+
|
| 446 |
+
def _jitter_local(t, pmin, pmax, mpf, mean_period):
|
| 447 |
+
n_periods = t.size - 1
|
| 448 |
+
if n_periods < 2:
|
| 449 |
+
return _NAN
|
| 450 |
+
total = 0.0
|
| 451 |
+
for i in range(1, t.size - 1):
|
| 452 |
+
p1, p2 = t[i] - t[i - 1], t[i + 1] - t[i]
|
| 453 |
+
f = p1 / p2 if p1 > p2 else p2 / p1
|
| 454 |
+
if pmin == pmax or (pmin <= p1 <= pmax and pmin <= p2 <= pmax and f <= mpf):
|
| 455 |
+
total += abs(p1 - p2)
|
| 456 |
+
else:
|
| 457 |
+
n_periods -= 1
|
| 458 |
+
if n_periods < 2:
|
| 459 |
+
return _NAN
|
| 460 |
+
return (total / (n_periods - 1)) / mean_period
|
| 461 |
+
|
| 462 |
+
|
| 463 |
+
def _hann_rms(s: P.Sound, tmid, wl, wr):
|
| 464 |
+
imin, imax, count = P.window_samples(s.x1, s.dx, s.nx, tmid - wl, tmid + wr)
|
| 465 |
+
if count < 3:
|
| 466 |
+
return _NAN
|
| 467 |
+
i = np.arange(imin, imax + 1)
|
| 468 |
+
tt = np.array([fma(k - 1, s.dx, s.x1) for k in i])
|
| 469 |
+
width = np.where(tt < tmid, wl, wr)
|
| 470 |
+
w = 0.5 + 0.5 * np.cos(math.pi * ((tt - tmid) / width))
|
| 471 |
+
v = s.z[i - 1] * w
|
| 472 |
+
return math.sqrt(float(np.sum(v * v)) / float(np.sum(w * w)))
|
| 473 |
+
|
| 474 |
+
|
| 475 |
+
def _shimmer(t, s, pmin, pmax, mpf, maf):
|
| 476 |
+
if t.size < 3:
|
| 477 |
+
return _NAN, _NAN
|
| 478 |
+
pts = []
|
| 479 |
+
for i in range(1, t.size - 1):
|
| 480 |
+
p1, p2 = t[i] - t[i - 1], t[i + 1] - t[i]
|
| 481 |
+
f = p1 / p2 if p1 > p2 else p2 / p1
|
| 482 |
+
if pmin == pmax or (pmin <= p1 <= pmax and pmin <= p2 <= pmax and f <= mpf):
|
| 483 |
+
pk = _hann_rms(s, t[i], 0.2 * p1, 0.2 * p2)
|
| 484 |
+
if not math.isnan(pk) and pk > 0.0:
|
| 485 |
+
pts.append((t[i], pk))
|
| 486 |
+
num, cnt, db = 0.0, 0, 0.0
|
| 487 |
+
for k in range(1, len(pts)):
|
| 488 |
+
p = pts[k][0] - pts[k - 1][0]
|
| 489 |
+
if pmin == pmax or (pmin <= p <= pmax):
|
| 490 |
+
a1, a2 = pts[k - 1][1], pts[k][1]
|
| 491 |
+
af = a1 / a2 if a1 > a2 else a2 / a1
|
| 492 |
+
if af <= maf:
|
| 493 |
+
num += abs(a1 - a2)
|
| 494 |
+
db += abs(math.log10(a1 / a2))
|
| 495 |
+
cnt += 1
|
| 496 |
+
if cnt < 1:
|
| 497 |
+
return _NAN, _NAN
|
| 498 |
+
num /= cnt
|
| 499 |
+
den = sum(v for _, v in pts[:-1]) / (len(pts) - 1)
|
| 500 |
+
local = num / den if den != 0.0 else _NAN
|
| 501 |
+
return local, 20.0 * (db / cnt)
|
| 502 |
+
|
| 503 |
+
|
| 504 |
+
def _c_printf_round(value: float, digits: int) -> float:
|
| 505 |
+
"""``%.*f`` then strtod: Python's formatting is correctly rounded like macOS libc."""
|
| 506 |
+
return float(f"{value:.{digits}f}")
|
| 507 |
+
|
| 508 |
+
|
| 509 |
+
def _min_prec(v):
|
| 510 |
+
return -math.floor(math.log10(abs(v)))
|
| 511 |
+
|
| 512 |
+
|
| 513 |
+
def report_percent(value):
|
| 514 |
+
"""Melder_percent(value, 3) -> extractNumber -> value in percent units."""
|
| 515 |
+
if math.isnan(value):
|
| 516 |
+
return _NAN
|
| 517 |
+
if value == 0.0:
|
| 518 |
+
return 0.0
|
| 519 |
+
v = value * 100.0
|
| 520 |
+
s = f"{v:.{max(_min_prec(v), 3)}f}"
|
| 521 |
+
return (0.01 * float(s)) * 100.0 # script: extractNumber (gives fraction) * 100
|
| 522 |
+
|
| 523 |
+
|
| 524 |
+
def report_fixed(value, precision=3):
|
| 525 |
+
if math.isnan(value):
|
| 526 |
+
return _NAN
|
| 527 |
+
if value == 0.0:
|
| 528 |
+
return 0.0
|
| 529 |
+
return float(f"{value:.{max(_min_prec(value), precision)}f}")
|
| 530 |
+
|
| 531 |
+
|
| 532 |
+
def report_fixed_exponent(value, exponent=-3, precision=6):
|
| 533 |
+
if math.isnan(value):
|
| 534 |
+
return _NAN
|
| 535 |
+
if value == 0.0:
|
| 536 |
+
return 0.0
|
| 537 |
+
v = value / (10.0 ** exponent)
|
| 538 |
+
return float(f"{v:.{max(_min_prec(v), precision)}f}E{exponent}")
|
| 539 |
+
|
| 540 |
+
|
| 541 |
+
def voice_report(abi: P.Sound, raw: bool = False) -> dict:
|
| 542 |
+
"""Jitter/shimmer/PSD as the script extracts them from ``Voice report``."""
|
| 543 |
+
pitch = sound_to_pitch_ac(abi, 0.0, 70.0, 600.0)
|
| 544 |
+
t = sound_pitch_to_point_process_cc(abi, pitch)
|
| 545 |
+
floor_, ceiling = 70.0, 600.0
|
| 546 |
+
pmin, pmax = 0.8 / ceiling, 1.25 / floor_
|
| 547 |
+
mean_period, sd_period = _period_stats(t, pmin, pmax, 1.3)
|
| 548 |
+
jit = _jitter_local(t, pmin, pmax, 1.3, mean_period)
|
| 549 |
+
shim, shim_db = _shimmer(t, abi, pmin, pmax, 1.3, 1.6)
|
| 550 |
+
if raw:
|
| 551 |
+
return {"jitter_local": jit * 100, "shimmer_local": shim * 100,
|
| 552 |
+
"shimmer_local_db": shim_db, "psd": sd_period, "_points": t, "_pitch": pitch}
|
| 553 |
+
return {"jitter_local": report_percent(jit), "shimmer_local": report_percent(shim),
|
| 554 |
+
"shimmer_local_db": report_fixed(shim_db, 3), "psd": report_fixed_exponent(sd_period, -3, 6)}
|
inference/predict.py
ADDED
|
@@ -0,0 +1,439 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Aria v0.2.3.3 cold-start inference entry point.
|
| 3 |
+
|
| 4 |
+
Two input contracts, both supported here:
|
| 5 |
+
|
| 6 |
+
raw recording -> stage 0 -> segment A -> 9 native features -> ABI' -> S
|
| 7 |
+
segment A (16 kHz) -> 9 native features -> ABI' -> S
|
| 8 |
+
|
| 9 |
+
Chain (SSOT §3.3, see ``../model_card.md``):
|
| 10 |
+
|
| 11 |
+
stage 0 (seg_ver stage0-v0.2.1.2) -> segment A, 16 kHz mono float
|
| 12 |
+
native 9-feature engine -> x_1..x_9
|
| 13 |
+
L3 coefficients (weights/model.json)
|
| 14 |
+
ABI'_raw = b0 + sum_j b_j x_j
|
| 15 |
+
ABI' = clip(ABI'_raw, 0, 10) G6 is inactive (identity)
|
| 16 |
+
dimension chain g (weights/g_table.json)
|
| 17 |
+
S = g(ABI') in [0, 100]
|
| 18 |
+
y_hat_mm = 10 * ABI' (PVQD CAPE-V overall severity, mm)
|
| 19 |
+
|
| 20 |
+
Only the 57 scorable recordings of the §2.5 locked benchmark were produced
|
| 21 |
+
through the raw-recording path; anything that is not a PVQD protocol compound
|
| 22 |
+
recording is outside the validated scope of this package.
|
| 23 |
+
|
| 24 |
+
Dependencies: numpy, scipy (g tails), soundfile, onnxruntime (stage 0 VAD only).
|
| 25 |
+
"""
|
| 26 |
+
|
| 27 |
+
from __future__ import annotations
|
| 28 |
+
|
| 29 |
+
import argparse
|
| 30 |
+
import csv
|
| 31 |
+
import hashlib
|
| 32 |
+
import json
|
| 33 |
+
import multiprocessing as mp
|
| 34 |
+
import platform
|
| 35 |
+
import sys
|
| 36 |
+
from collections import Counter
|
| 37 |
+
from pathlib import Path
|
| 38 |
+
|
| 39 |
+
import numpy as np
|
| 40 |
+
from scipy.stats import norm
|
| 41 |
+
|
| 42 |
+
_PACKAGE_ROOT = Path(__file__).resolve().parents[1]
|
| 43 |
+
if str(_PACKAGE_ROOT) not in sys.path:
|
| 44 |
+
sys.path.insert(0, str(_PACKAGE_ROOT))
|
| 45 |
+
|
| 46 |
+
from inference.native_features import abi_native as N # noqa: E402
|
| 47 |
+
from inference.stage0 import constants as C # noqa: E402
|
| 48 |
+
from inference.stage0.audio import load_wav # noqa: E402
|
| 49 |
+
from inference.stage0.pipeline import run_stage0 # noqa: E402
|
| 50 |
+
from inference.stage0.vad import SileroVad # noqa: E402
|
| 51 |
+
|
| 52 |
+
ABI_DECIMALS = 10
|
| 53 |
+
GATE_ORDER = ("R1", "R2", "R3", "R4", "R5", "M_T5", "W_窗数")
|
| 54 |
+
# The package's own caliber, i.e. the one whose numbers the §2.5 locked
|
| 55 |
+
# benchmark reports: R1 and W_窗数 waived, R3 waived when voiced_ratio >= 0.30.
|
| 56 |
+
# R2 / R4 / R5 / M_T5 and the severe bypass reject as usual.
|
| 57 |
+
RELAXABLE = {"R1", "W_窗数"}
|
| 58 |
+
R3_WAIVE_VOICED_RATIO = 0.30
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
# --------------------------------------------------------------------------- #
|
| 62 |
+
# package loading
|
| 63 |
+
# --------------------------------------------------------------------------- #
|
| 64 |
+
def sha256_file(path) -> str:
|
| 65 |
+
digest = hashlib.sha256()
|
| 66 |
+
with Path(path).open("rb") as handle:
|
| 67 |
+
for chunk in iter(lambda: handle.read(1 << 20), b""):
|
| 68 |
+
digest.update(chunk)
|
| 69 |
+
return digest.hexdigest()
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def resolve_vad_model(vad_model=None) -> Path:
|
| 73 |
+
"""Silero v5 ONNX path: explicit > $ARIA_VAD_MODEL > package-bundled copy."""
|
| 74 |
+
import os
|
| 75 |
+
|
| 76 |
+
candidates = [vad_model, os.environ.get("ARIA_VAD_MODEL"),
|
| 77 |
+
Path(__file__).resolve().parent / "models" / "SileroVAD.onnx"]
|
| 78 |
+
for candidate in candidates:
|
| 79 |
+
if candidate and Path(candidate).is_file():
|
| 80 |
+
path = Path(candidate)
|
| 81 |
+
actual = sha256_file(path)
|
| 82 |
+
if actual != C.VAD_MODEL_SHA256:
|
| 83 |
+
raise SystemExit(
|
| 84 |
+
f"VAD model SHA-256 mismatch: {path} is {actual}, the pinned "
|
| 85 |
+
f"stage-0 constant table requires {C.VAD_MODEL_SHA256}")
|
| 86 |
+
return path
|
| 87 |
+
raise SystemExit(
|
| 88 |
+
"no Silero VAD model found. Pass --vad-model PATH, set $ARIA_VAD_MODEL, or "
|
| 89 |
+
f"place the pinned SileroVAD.onnx (SHA-256 {C.VAD_MODEL_SHA256}) at "
|
| 90 |
+
f"{Path(__file__).resolve().parent / 'models' / 'SileroVAD.onnx'}")
|
| 91 |
+
|
| 92 |
+
|
| 93 |
+
def load_package(root=None) -> dict:
|
| 94 |
+
"""Load weights/model.json + weights/g_table.json and verify they bind."""
|
| 95 |
+
root = Path(root) if root else _PACKAGE_ROOT
|
| 96 |
+
model_path, g_path = root / "weights" / "model.json", root / "weights" / "g_table.json"
|
| 97 |
+
model = json.loads(model_path.read_text())
|
| 98 |
+
g_table = json.loads(g_path.read_text())
|
| 99 |
+
want = model["g_table"]["sha256"]
|
| 100 |
+
actual = sha256_file(g_path)
|
| 101 |
+
if want != actual:
|
| 102 |
+
raise SystemExit(f"g_table.json SHA-256 mismatch: model.json says {want}, file is {actual}")
|
| 103 |
+
return {
|
| 104 |
+
"root": root,
|
| 105 |
+
"model_path": model_path,
|
| 106 |
+
"g_table_path": g_path,
|
| 107 |
+
"model_sha256": sha256_file(model_path),
|
| 108 |
+
"g_table_sha256": actual,
|
| 109 |
+
"model": model,
|
| 110 |
+
"g_table": g_table,
|
| 111 |
+
"feature_names": list(model["feature_names"]),
|
| 112 |
+
"b0": float(model["abi_prime_raw"]["coefficients"]["b0"]),
|
| 113 |
+
"b": np.asarray(model["abi_prime_raw"]["coefficients"]["b"], dtype=float),
|
| 114 |
+
"clip": tuple(float(v) for v in model["abi_prime_raw"]["clip"]),
|
| 115 |
+
}
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
# --------------------------------------------------------------------------- #
|
| 119 |
+
# the chain
|
| 120 |
+
# --------------------------------------------------------------------------- #
|
| 121 |
+
def abi_prime(values: dict, package: dict) -> float:
|
| 122 |
+
"""L3 linear head on the nine native features, clipped to [0, 10]."""
|
| 123 |
+
x = np.array([values[k] for k in package["feature_names"]], dtype=float)
|
| 124 |
+
if not np.all(np.isfinite(x)):
|
| 125 |
+
raise ValueError("ABI' needs all nine features finite")
|
| 126 |
+
raw = package["b0"] + float(x @ package["b"])
|
| 127 |
+
lo, hi = package["clip"]
|
| 128 |
+
return float(np.clip(raw, lo, hi))
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def severity_from_abi(abi: float, package: dict) -> float:
|
| 132 |
+
"""Dimension chain g: ABI' in [0, 10] -> S in [0, 100]."""
|
| 133 |
+
table = package["g_table"]
|
| 134 |
+
r = np.asarray(table["knots"], dtype=float)
|
| 135 |
+
p = np.asarray(table["p"], dtype=float)
|
| 136 |
+
sigma = float(table["sigma"])
|
| 137 |
+
if abi < r[0]:
|
| 138 |
+
return float(100.0 * norm.cdf(norm.ppf(p[0]) - (r[0] - abi) / sigma))
|
| 139 |
+
if abi > r[-1]:
|
| 140 |
+
return float(100.0 * norm.cdf(norm.ppf(p[-1]) + (abi - r[-1]) / sigma))
|
| 141 |
+
return float(100.0 * np.interp(abi, r, p))
|
| 142 |
+
|
| 143 |
+
|
| 144 |
+
def nine_features(segment_a: np.ndarray, sample_rate: int = C.TARGET_SR) -> dict:
|
| 145 |
+
"""Nine native ABI variables; NaN where Praat would be undefined."""
|
| 146 |
+
segment_a = np.asarray(segment_a, dtype=np.float32).astype(np.float64)
|
| 147 |
+
return N.segment_features(segment_a, sample_rate)
|
| 148 |
+
|
| 149 |
+
|
| 150 |
+
def score_segment(segment_a: np.ndarray, package: dict,
|
| 151 |
+
sample_rate: int = C.TARGET_SR) -> dict:
|
| 152 |
+
"""segment A -> nine features -> ABI' -> S."""
|
| 153 |
+
values = nine_features(segment_a, sample_rate)
|
| 154 |
+
finite = {k: (float(v) if v is not None and np.isfinite(v) else None)
|
| 155 |
+
for k, v in values.items()}
|
| 156 |
+
complete = all(finite[k] is not None for k in package["feature_names"])
|
| 157 |
+
out = {"features": finite, "abi_prime": None, "y_hat_mm": None, "s": None}
|
| 158 |
+
if complete:
|
| 159 |
+
abi = abi_prime(finite, package)
|
| 160 |
+
out.update(abi_prime=abi, y_hat_mm=10.0 * abi,
|
| 161 |
+
s=severity_from_abi(abi, package))
|
| 162 |
+
return out
|
| 163 |
+
|
| 164 |
+
|
| 165 |
+
def segment_from_recording(audio_path, vad_model=None, vad: SileroVad | None = None) -> dict:
|
| 166 |
+
"""Raw recording -> stage 0 -> segment A + 21-field report + diagnostics."""
|
| 167 |
+
audio_path = Path(audio_path)
|
| 168 |
+
if vad is None:
|
| 169 |
+
vad = SileroVad(model_path=resolve_vad_model(vad_model))
|
| 170 |
+
result = run_stage0(load_wav(audio_path), vad=vad)
|
| 171 |
+
return {
|
| 172 |
+
"sha256": sha256_file(audio_path),
|
| 173 |
+
"report": result.report,
|
| 174 |
+
"diagnostics": _jsonable(result.diagnostics),
|
| 175 |
+
"segment_a": result.segment_a.astype(np.float32),
|
| 176 |
+
}
|
| 177 |
+
|
| 178 |
+
|
| 179 |
+
def _jsonable(value):
|
| 180 |
+
if isinstance(value, dict):
|
| 181 |
+
return {k: _jsonable(v) for k, v in value.items()}
|
| 182 |
+
if isinstance(value, (list, tuple)):
|
| 183 |
+
return [_jsonable(v) for v in value]
|
| 184 |
+
if isinstance(value, (float, np.floating)):
|
| 185 |
+
return float(value) if np.isfinite(value) else None
|
| 186 |
+
if isinstance(value, np.integer):
|
| 187 |
+
return int(value)
|
| 188 |
+
if isinstance(value, np.bool_):
|
| 189 |
+
return bool(value)
|
| 190 |
+
return value
|
| 191 |
+
|
| 192 |
+
|
| 193 |
+
# --------------------------------------------------------------------------- #
|
| 194 |
+
# gate calibers
|
| 195 |
+
# --------------------------------------------------------------------------- #
|
| 196 |
+
def statuses(report: dict, diagnostics: dict, values) -> dict:
|
| 197 |
+
"""The two published calibers for one recording.
|
| 198 |
+
|
| 199 |
+
``training`` is the caliber of this package's own numbers (the §2.5 locked
|
| 200 |
+
benchmark and the dev metrics): R1 and W_窗数 waived, R3 waived when
|
| 201 |
+
``voiced_ratio >= 0.30``. ``product`` fires all seven gates.
|
| 202 |
+
"""
|
| 203 |
+
gates = tuple(g for g in GATE_ORDER if g in set(diagnostics["triggered_gates"]))
|
| 204 |
+
|
| 205 |
+
def reject_or(reason):
|
| 206 |
+
return f"rejected:{reason}"
|
| 207 |
+
|
| 208 |
+
if report["bypass_severe"]:
|
| 209 |
+
product = training = "bypass_severe"
|
| 210 |
+
return {"product": product, "training": training, "triggered_gates": list(gates),
|
| 211 |
+
"reject_code": None}
|
| 212 |
+
|
| 213 |
+
product = (reject_or(report["reject_code"]) if report["reject"] else "scored")
|
| 214 |
+
|
| 215 |
+
remaining = set(gates) - RELAXABLE
|
| 216 |
+
if "R3" in remaining and report["voiced_ratio"] >= R3_WAIVE_VOICED_RATIO:
|
| 217 |
+
remaining.discard("R3")
|
| 218 |
+
if remaining:
|
| 219 |
+
training = reject_or("|".join(g for g in GATE_ORDER if g in remaining))
|
| 220 |
+
elif values is None or any(values.get(k) is None for k in N.FEATURE_NAMES):
|
| 221 |
+
training = reject_or("missing_features")
|
| 222 |
+
else:
|
| 223 |
+
training = "scored"
|
| 224 |
+
return {"product": product, "training": training,
|
| 225 |
+
"triggered_gates": list(gates), "reject_code": report["reject_code"]}
|
| 226 |
+
|
| 227 |
+
|
| 228 |
+
def _distance_bin(distance_cm) -> str | None:
|
| 229 |
+
value = f"{int(distance_cm)}cm"
|
| 230 |
+
return value if value in ("15cm", "30cm", "50cm") else None
|
| 231 |
+
|
| 232 |
+
|
| 233 |
+
# --------------------------------------------------------------------------- #
|
| 234 |
+
# one recording, end to end
|
| 235 |
+
# --------------------------------------------------------------------------- #
|
| 236 |
+
def predict_recording(audio_path, package: dict, vad_model=None, vad=None,
|
| 237 |
+
distance_cm: int = 6, return_segment: bool = False) -> dict:
|
| 238 |
+
# the pipeline's segment A is a 16 kHz float32 mono waveform; the cast keeps
|
| 239 |
+
# an in-memory run bit-identical to one that round-trips the written WAV.
|
| 240 |
+
seg = segment_from_recording(audio_path, vad_model=vad_model, vad=vad)
|
| 241 |
+
report, diagnostics = seg["report"], seg["diagnostics"]
|
| 242 |
+
scored = score_segment(seg["segment_a"], package)
|
| 243 |
+
st = statuses(report, diagnostics, scored["features"])
|
| 244 |
+
out = {
|
| 245 |
+
"audio_path": str(audio_path),
|
| 246 |
+
"audio_sha256": seg["sha256"],
|
| 247 |
+
"stage0": {"seg_ver": report["seg_ver"], "report": report, "diagnostics": diagnostics},
|
| 248 |
+
"segment_a": {"seconds": float(len(seg["segment_a"]) / C.TARGET_SR),
|
| 249 |
+
"sample_rate": C.TARGET_SR},
|
| 250 |
+
**scored,
|
| 251 |
+
"status": st["training"],
|
| 252 |
+
"status_product_caliber": st["product"],
|
| 253 |
+
"triggered_gates": st["triggered_gates"],
|
| 254 |
+
}
|
| 255 |
+
if return_segment:
|
| 256 |
+
out["_segment_a"] = seg["segment_a"]
|
| 257 |
+
return out
|
| 258 |
+
|
| 259 |
+
|
| 260 |
+
# --------------------------------------------------------------------------- #
|
| 261 |
+
# batch mode (one row per request, corpus order preserved)
|
| 262 |
+
# --------------------------------------------------------------------------- #
|
| 263 |
+
_VAD = None
|
| 264 |
+
_VAD_MODEL = None
|
| 265 |
+
|
| 266 |
+
|
| 267 |
+
def _init_worker(vad_model):
|
| 268 |
+
global _VAD, _VAD_MODEL
|
| 269 |
+
_VAD_MODEL = vad_model
|
| 270 |
+
_VAD = SileroVad(model_path=resolve_vad_model(vad_model))
|
| 271 |
+
|
| 272 |
+
|
| 273 |
+
def _batch_job(job):
|
| 274 |
+
index, request, dataset, outdir, write_audio = job
|
| 275 |
+
try:
|
| 276 |
+
path = Path(dataset) / request["audio_path"]
|
| 277 |
+
if request.get("audio_sha256") and sha256_file(path) != request["audio_sha256"]:
|
| 278 |
+
raise ValueError(f"sha256 mismatch for {request['audio_path']}")
|
| 279 |
+
row = predict_recording(path, load_package(), distance_cm=request.get("distance_cm", 6),
|
| 280 |
+
vad=_VAD, return_segment=write_audio)
|
| 281 |
+
if write_audio:
|
| 282 |
+
import soundfile as sf
|
| 283 |
+
|
| 284 |
+
target = Path(outdir) / "segments" / request["audio_path"]
|
| 285 |
+
if target.suffix.lower() != ".wav":
|
| 286 |
+
target = target.with_suffix(target.suffix + ".wav")
|
| 287 |
+
target.parent.mkdir(parents=True, exist_ok=True)
|
| 288 |
+
sf.write(str(target), row.pop("_segment_a"), C.TARGET_SR, subtype="FLOAT")
|
| 289 |
+
row["speaker_id"] = request.get("speaker_id")
|
| 290 |
+
return index, row, None
|
| 291 |
+
except Exception as exc: # noqa: BLE001 - one bad file must not lose the run
|
| 292 |
+
return index, None, f"{type(exc).__name__}: {str(exc).splitlines()[0]}"
|
| 293 |
+
|
| 294 |
+
|
| 295 |
+
def predict_corpus(requests, dataset, outdir, package: dict, vad_model=None,
|
| 296 |
+
workers: int = 8, write_audio: bool = False) -> dict:
|
| 297 |
+
outdir = Path(outdir)
|
| 298 |
+
if outdir.exists():
|
| 299 |
+
raise SystemExit(f"refusing to overwrite existing output {outdir}")
|
| 300 |
+
outdir.mkdir(parents=True)
|
| 301 |
+
jobs = [(i, r, str(dataset), str(outdir), write_audio)
|
| 302 |
+
for i, r in enumerate(requests)]
|
| 303 |
+
results: list = [None] * len(jobs)
|
| 304 |
+
errors: dict = {}
|
| 305 |
+
if workers > 1 and len(jobs) > 1:
|
| 306 |
+
with mp.get_context("spawn").Pool(workers, initializer=_init_worker,
|
| 307 |
+
initargs=(vad_model,)) as pool:
|
| 308 |
+
for index, row, error in pool.imap_unordered(_batch_job, jobs, chunksize=2):
|
| 309 |
+
results[index] = row
|
| 310 |
+
if error:
|
| 311 |
+
errors[index] = error
|
| 312 |
+
else:
|
| 313 |
+
_init_worker(vad_model)
|
| 314 |
+
for job in jobs:
|
| 315 |
+
index, row, error = _batch_job(job)
|
| 316 |
+
results[index] = row
|
| 317 |
+
if error:
|
| 318 |
+
errors[index] = error
|
| 319 |
+
|
| 320 |
+
primary, product = [], []
|
| 321 |
+
prim_dist, prod_dist = Counter(), Counter()
|
| 322 |
+
for request, row, index in zip(requests, results, range(len(results))):
|
| 323 |
+
sid = request.get("speaker_id")
|
| 324 |
+
if row is None:
|
| 325 |
+
reason = errors.get(index, "unknown failure")
|
| 326 |
+
primary.append([sid, "", "", f"error:{reason}"])
|
| 327 |
+
product.append([sid, f"error:{reason}", "", ""])
|
| 328 |
+
prim_dist[f"error:{reason}"] += 1
|
| 329 |
+
prod_dist[f"error:{reason}"] += 1
|
| 330 |
+
continue
|
| 331 |
+
prim_dist[row["status"]] += 1
|
| 332 |
+
prod_dist[row["status_product_caliber"]] += 1
|
| 333 |
+
# The published interface emits a prediction only for scored rows; a
|
| 334 |
+
# rejected row carries no ŷ, even when the nine features were computable.
|
| 335 |
+
if row["status"] != "scored" or row["abi_prime"] is None:
|
| 336 |
+
primary.append([sid, "", "", row["status"]])
|
| 337 |
+
else:
|
| 338 |
+
primary.append([sid, f"{row['abi_prime']:.{ABI_DECIMALS}f}",
|
| 339 |
+
f"{row['y_hat_mm']:.{ABI_DECIMALS}f}", row["status"]])
|
| 340 |
+
product.append([sid, row["status_product_caliber"],
|
| 341 |
+
row["status_product_caliber"].split(":")[-1]
|
| 342 |
+
if ":" in row["status_product_caliber"] else "",
|
| 343 |
+
"|".join(row["triggered_gates"])])
|
| 344 |
+
|
| 345 |
+
_write_csv(outdir / "predictions.csv",
|
| 346 |
+
["speaker_id", "abi_prime", "y_hat_mm", "status"], primary)
|
| 347 |
+
_write_csv(outdir / "predictions_product_caliber.csv",
|
| 348 |
+
["speaker_id", "status", "reject_code", "triggered_gates"], product)
|
| 349 |
+
|
| 350 |
+
summary = {
|
| 351 |
+
"model_ver": package["model"]["model_ver"],
|
| 352 |
+
"seg_ver": C.SEG_VER,
|
| 353 |
+
"feature_engine": "native",
|
| 354 |
+
"status_caliber": ("training / package caliber: R1 & W_窗数 waived, R3 waived when "
|
| 355 |
+
"voiced_ratio>=0.30; R2/R4/R5/M_T5 & bypass reject"),
|
| 356 |
+
"environment": {"python": sys.version.split()[0], "platform": platform.platform(),
|
| 357 |
+
"numpy": np.__version__},
|
| 358 |
+
"package": {"model_json_sha256": package["model_sha256"],
|
| 359 |
+
"g_table_json_sha256": package["g_table_sha256"]},
|
| 360 |
+
"n_requests": len(requests),
|
| 361 |
+
"status_distribution": dict(sorted(prim_dist.items())),
|
| 362 |
+
"product_status_distribution": dict(sorted(prod_dist.items())),
|
| 363 |
+
"errors": {str(k): v for k, v in sorted(errors.items())},
|
| 364 |
+
"outputs_sha256": {
|
| 365 |
+
"predictions.csv": sha256_file(outdir / "predictions.csv"),
|
| 366 |
+
"predictions_product_caliber.csv":
|
| 367 |
+
sha256_file(outdir / "predictions_product_caliber.csv"),
|
| 368 |
+
},
|
| 369 |
+
}
|
| 370 |
+
(outdir / "run.json").write_text(
|
| 371 |
+
json.dumps(summary, indent=2, ensure_ascii=False) + "\n")
|
| 372 |
+
return summary
|
| 373 |
+
|
| 374 |
+
|
| 375 |
+
def _write_csv(path: Path, header, rows) -> None:
|
| 376 |
+
with path.open("w", newline="", encoding="utf-8") as handle:
|
| 377 |
+
writer = csv.writer(handle)
|
| 378 |
+
writer.writerow(header)
|
| 379 |
+
writer.writerows(rows)
|
| 380 |
+
|
| 381 |
+
|
| 382 |
+
# --------------------------------------------------------------------------- #
|
| 383 |
+
# CLI
|
| 384 |
+
# --------------------------------------------------------------------------- #
|
| 385 |
+
def main(argv=None) -> int:
|
| 386 |
+
parser = argparse.ArgumentParser(
|
| 387 |
+
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
| 388 |
+
parser.add_argument("--package", type=Path, default=_PACKAGE_ROOT,
|
| 389 |
+
help="package root holding weights/ (default: this package)")
|
| 390 |
+
parser.add_argument("--vad-model", type=Path,
|
| 391 |
+
help="SileroVad.onnx; default $ARIA_VAD_MODEL or inference/models/")
|
| 392 |
+
parser.add_argument("--audio", type=Path, help="one raw recording")
|
| 393 |
+
parser.add_argument("--segment-a", type=Path, help="one segment A WAV (16 kHz mono)")
|
| 394 |
+
parser.add_argument("--requests", type=Path, help="requests.jsonl: audio_path, "
|
| 395 |
+
"audio_sha256, distance_cm, speaker_id")
|
| 396 |
+
parser.add_argument("--dataset", type=Path, help="root the request audio_paths resolve in")
|
| 397 |
+
parser.add_argument("--out", type=Path, help="output JSON for --audio / --segment-a")
|
| 398 |
+
parser.add_argument("--outdir", type=Path, help="output directory for --requests")
|
| 399 |
+
parser.add_argument("--write-audio", action="store_true",
|
| 400 |
+
help="also write each segment A under --outdir/segments/")
|
| 401 |
+
parser.add_argument("--workers", type=int, default=8)
|
| 402 |
+
args = parser.parse_args(argv)
|
| 403 |
+
|
| 404 |
+
package = load_package(args.package)
|
| 405 |
+
if args.requests:
|
| 406 |
+
if not args.dataset or not args.outdir:
|
| 407 |
+
parser.error("--requests needs --dataset and --outdir")
|
| 408 |
+
requests = [json.loads(line) for line in
|
| 409 |
+
args.requests.read_text().splitlines() if line.strip()]
|
| 410 |
+
summary = predict_corpus(requests, args.dataset, args.outdir, package,
|
| 411 |
+
vad_model=args.vad_model, workers=args.workers,
|
| 412 |
+
write_audio=args.write_audio)
|
| 413 |
+
print(json.dumps(summary, indent=2, ensure_ascii=False))
|
| 414 |
+
return 0
|
| 415 |
+
|
| 416 |
+
if args.segment_a:
|
| 417 |
+
import soundfile as sf
|
| 418 |
+
|
| 419 |
+
pcm, sr = sf.read(str(args.segment_a))
|
| 420 |
+
if pcm.ndim > 1:
|
| 421 |
+
pcm = pcm.mean(axis=1)
|
| 422 |
+
scored = score_segment(pcm, package, sr)
|
| 423 |
+
out = {"source": str(args.segment_a), "sha256": sha256_file(args.segment_a),
|
| 424 |
+
"segment_a": {"seconds": float(len(pcm) / sr), "sample_rate": sr}, **scored}
|
| 425 |
+
elif args.audio:
|
| 426 |
+
out = predict_recording(args.audio, package, vad_model=args.vad_model)
|
| 427 |
+
else:
|
| 428 |
+
parser.error("give one of --audio, --segment-a, --requests")
|
| 429 |
+
|
| 430 |
+
text = json.dumps(out, indent=2, ensure_ascii=False)
|
| 431 |
+
if args.out:
|
| 432 |
+
args.out.write_text(text + "\n")
|
| 433 |
+
else:
|
| 434 |
+
print(text)
|
| 435 |
+
return 0
|
| 436 |
+
|
| 437 |
+
|
| 438 |
+
if __name__ == "__main__":
|
| 439 |
+
raise SystemExit(main())
|
inference/stage0/__init__.py
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Aria v0.2.1 stage 0 (training side).
|
| 2 |
+
|
| 3 |
+
Public entry points:
|
| 4 |
+
|
| 5 |
+
``run_stage0`` one recording -> segment A + 21-field quality report
|
| 6 |
+
``SileroVad`` Silero v5 ONNX detector at the SSOT operating point
|
| 7 |
+
``validate_report`` Schema v0.2.1 contract check
|
| 8 |
+
"""
|
| 9 |
+
|
| 10 |
+
from .audio import LoadedAudio, load_wav
|
| 11 |
+
from .pipeline import Stage0Result, run_stage0
|
| 12 |
+
from .report import SCHEMA_FIELDS, SCHEMA_VERSION, validate_report
|
| 13 |
+
from .vad import SileroVad
|
| 14 |
+
|
| 15 |
+
__all__ = [
|
| 16 |
+
"LoadedAudio", "load_wav", "run_stage0", "Stage0Result",
|
| 17 |
+
"SileroVad", "SCHEMA_FIELDS", "SCHEMA_VERSION", "validate_report",
|
| 18 |
+
]
|
inference/stage0/audio.py
ADDED
|
@@ -0,0 +1,97 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Audio loading and resampling for stage 0.
|
| 2 |
+
|
| 3 |
+
Stage 0 is the only place that touches the raw recording (SSOT §2). Every
|
| 4 |
+
downstream stage must consume ``Stage0Audio.data`` (the concatenated periodic
|
| 5 |
+
voiced segment A) and the quality report, never the original file.
|
| 6 |
+
"""
|
| 7 |
+
|
| 8 |
+
from __future__ import annotations
|
| 9 |
+
|
| 10 |
+
import hashlib
|
| 11 |
+
from dataclasses import dataclass
|
| 12 |
+
from pathlib import Path
|
| 13 |
+
from typing import Optional
|
| 14 |
+
|
| 15 |
+
import numpy as np
|
| 16 |
+
|
| 17 |
+
try:
|
| 18 |
+
from . import constants as C
|
| 19 |
+
except ImportError: # script-style import
|
| 20 |
+
import constants as C
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
@dataclass(frozen=True)
|
| 24 |
+
class LoadedAudio:
|
| 25 |
+
"""Raw PCM plus the provenance the report's ``input_spec`` needs."""
|
| 26 |
+
|
| 27 |
+
pcm: np.ndarray # float64, mono, shape (n,), range [-1, 1]
|
| 28 |
+
sample_rate: int
|
| 29 |
+
bit_depth: int # 16 for PCM_16; 0 when unknown
|
| 30 |
+
channels: int # channels in the source file before downmix
|
| 31 |
+
resampled: bool
|
| 32 |
+
source_sha256: str
|
| 33 |
+
|
| 34 |
+
@property
|
| 35 |
+
def duration_s(self) -> float:
|
| 36 |
+
return len(self.pcm) / float(self.sample_rate)
|
| 37 |
+
|
| 38 |
+
def input_spec(self) -> dict:
|
| 39 |
+
return {
|
| 40 |
+
"sample_rate_hz": int(self.sample_rate),
|
| 41 |
+
"bit_depth": int(self.bit_depth),
|
| 42 |
+
"channels": int(self.channels),
|
| 43 |
+
"resampled": bool(self.resampled),
|
| 44 |
+
"source_sha256": self.source_sha256,
|
| 45 |
+
}
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
_SUBTYPE_BITS = {
|
| 49 |
+
"PCM_S8": 8, "PCM_16": 16, "PCM_24": 24, "PCM_32": 32,
|
| 50 |
+
"FLOAT": 32, "DOUBLE": 64,
|
| 51 |
+
}
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def _mono(samples: np.ndarray) -> np.ndarray:
|
| 55 |
+
if samples.ndim == 1:
|
| 56 |
+
return samples.astype(np.float64, copy=False)
|
| 57 |
+
return samples.astype(np.float64, copy=False).mean(axis=1)
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def resample_to(samples: np.ndarray, src_sr: int, dst_sr: int) -> np.ndarray:
|
| 61 |
+
"""Band-limited rational resampling.
|
| 62 |
+
|
| 63 |
+
``scipy.signal.resample_poly`` uses a 20-tap-per-phase Kaiser filter by
|
| 64 |
+
default; that is the ENGINEERING choice this pipeline pins. Pure up/down
|
| 65 |
+
ratios fall back to interpolation only when the rate already matches.
|
| 66 |
+
"""
|
| 67 |
+
if src_sr == dst_sr:
|
| 68 |
+
return samples.astype(np.float64, copy=False)
|
| 69 |
+
from math import gcd
|
| 70 |
+
|
| 71 |
+
from scipy.signal import resample_poly
|
| 72 |
+
|
| 73 |
+
factor = gcd(int(src_sr), int(dst_sr))
|
| 74 |
+
up, down = int(dst_sr) // factor, int(src_sr) // factor
|
| 75 |
+
return resample_poly(samples, up, down).astype(np.float64, copy=False)
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
def load_wav(path: str | Path) -> LoadedAudio:
|
| 79 |
+
"""Load any soundfile-readable recording as mono float64 at 16 kHz."""
|
| 80 |
+
import soundfile as sf
|
| 81 |
+
|
| 82 |
+
path = Path(path)
|
| 83 |
+
digest = hashlib.sha256(path.read_bytes()).hexdigest()
|
| 84 |
+
info = sf.info(str(path))
|
| 85 |
+
raw, sr = sf.read(str(path), dtype="float64", always_2d=True)
|
| 86 |
+
channels = int(raw.shape[1])
|
| 87 |
+
mono = raw.mean(axis=1)
|
| 88 |
+
resampled = int(sr) != C.TARGET_SR
|
| 89 |
+
pcm = resample_to(mono, int(sr), C.TARGET_SR) if resampled else mono
|
| 90 |
+
return LoadedAudio(
|
| 91 |
+
pcm=np.asarray(pcm, dtype=np.float64),
|
| 92 |
+
sample_rate=C.TARGET_SR,
|
| 93 |
+
bit_depth=_SUBTYPE_BITS.get(info.subtype, 0),
|
| 94 |
+
channels=channels,
|
| 95 |
+
resampled=resampled,
|
| 96 |
+
source_sha256=digest,
|
| 97 |
+
)
|
inference/stage0/bypass.py
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Self-contamination-resistant noise floor and the severe bypass (SSOT §3.1.2).
|
| 2 |
+
|
| 3 |
+
The bypass exists because heavily disordered voices can fail every periodicity
|
| 4 |
+
test while still carrying sustained energy in the voice band. The v0.2 form
|
| 5 |
+
compared full-band energy to the median of VAD-silent frames, which silently
|
| 6 |
+
broke when breathiness filled every frame the VAD called silent. v0.2.1 pins
|
| 7 |
+
the noise floor to the P10 of all 50 ms frame RMS values and demands three
|
| 8 |
+
consecutive voice-band frames before any frame counts.
|
| 9 |
+
"""
|
| 10 |
+
|
| 11 |
+
from __future__ import annotations
|
| 12 |
+
|
| 13 |
+
from dataclasses import dataclass
|
| 14 |
+
|
| 15 |
+
import numpy as np
|
| 16 |
+
|
| 17 |
+
try:
|
| 18 |
+
from . import constants as C
|
| 19 |
+
except ImportError:
|
| 20 |
+
import constants as C
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
@dataclass(frozen=True)
|
| 24 |
+
class BypassEvidence:
|
| 25 |
+
frame_rms: np.ndarray
|
| 26 |
+
e_noise: float
|
| 27 |
+
active_mask: np.ndarray # frames satisfying (a), (b) and run length (c)
|
| 28 |
+
t_active_s: float
|
| 29 |
+
e_active: float
|
| 30 |
+
snr_active_db: float
|
| 31 |
+
trigger: bool
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
def frame_rms(samples: np.ndarray, frame_samples: int) -> np.ndarray:
|
| 35 |
+
"""Non-overlapping frame RMS over complete frames only."""
|
| 36 |
+
n_frames = len(samples) // frame_samples
|
| 37 |
+
if n_frames == 0:
|
| 38 |
+
return np.zeros(0)
|
| 39 |
+
usable = samples[:n_frames * frame_samples].reshape(n_frames, frame_samples)
|
| 40 |
+
return np.sqrt(np.mean(usable.astype(np.float64) ** 2, axis=1))
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
def band_energy_ratio(samples: np.ndarray, frame_samples: int,
|
| 44 |
+
band: tuple[float, float],
|
| 45 |
+
sample_rate: int = C.TARGET_SR) -> np.ndarray:
|
| 46 |
+
"""Fraction of each frame's spectral energy inside ``band``."""
|
| 47 |
+
n_frames = len(samples) // frame_samples
|
| 48 |
+
if n_frames == 0:
|
| 49 |
+
return np.zeros(0)
|
| 50 |
+
usable = samples[:n_frames * frame_samples].reshape(n_frames, frame_samples)
|
| 51 |
+
window = np.hanning(frame_samples)
|
| 52 |
+
spectrum = np.abs(np.fft.rfft(usable * window, axis=1)) ** 2
|
| 53 |
+
freqs = np.fft.rfftfreq(frame_samples, d=1.0 / sample_rate)
|
| 54 |
+
in_band = (freqs >= band[0]) & (freqs <= band[1])
|
| 55 |
+
total = spectrum.sum(axis=1)
|
| 56 |
+
banded = spectrum[:, in_band].sum(axis=1)
|
| 57 |
+
with np.errstate(divide="ignore", invalid="ignore"):
|
| 58 |
+
ratio = np.where(total > C.RMS_EPSILON, banded / total, 0.0)
|
| 59 |
+
return ratio
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
def qualify_run(mask: np.ndarray, min_run: int) -> np.ndarray:
|
| 63 |
+
"""Keep only True frames that belong to a run of length >= ``min_run``."""
|
| 64 |
+
if not mask.any():
|
| 65 |
+
return np.zeros_like(mask, dtype=bool)
|
| 66 |
+
out = np.zeros_like(mask, dtype=bool)
|
| 67 |
+
index = 0
|
| 68 |
+
n = len(mask)
|
| 69 |
+
while index < n:
|
| 70 |
+
if not mask[index]:
|
| 71 |
+
index += 1
|
| 72 |
+
continue
|
| 73 |
+
end = index
|
| 74 |
+
while end < n and mask[end]:
|
| 75 |
+
end += 1
|
| 76 |
+
if end - index >= min_run:
|
| 77 |
+
out[index:end] = True
|
| 78 |
+
index = end
|
| 79 |
+
return out
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def bypass_evidence(x_trim: np.ndarray, t_voiced_s: float,
|
| 83 |
+
sample_rate: int = C.TARGET_SR) -> BypassEvidence:
|
| 84 |
+
"""Compute T_active, the active-frame SNR and the trigger flag."""
|
| 85 |
+
frame = int(round(C.BYPASS_FRAME_S * sample_rate))
|
| 86 |
+
rms = frame_rms(x_trim, frame)
|
| 87 |
+
if rms.size == 0:
|
| 88 |
+
return BypassEvidence(rms, 0.0, np.zeros(0, dtype=bool), 0.0,
|
| 89 |
+
float("nan"), float("nan"), False)
|
| 90 |
+
e_noise_raw = float(np.percentile(rms, C.BYPASS_NOISE_PERCENTILE))
|
| 91 |
+
# Digital silence has a zero noise floor; without a floor every frame would
|
| 92 |
+
# count as "+6 dB above noise" on pure rounding error. The floor keeps the
|
| 93 |
+
# ratio test meaningful and leaves the reported E_noise unchanged.
|
| 94 |
+
e_noise = max(e_noise_raw, C.DIGITAL_SILENCE_RMS)
|
| 95 |
+
loud = rms > C.BYPASS_ACTIVE_OVER_NOISE * e_noise
|
| 96 |
+
banded = band_energy_ratio(x_trim, frame, C.BYPASS_BAND_HZ, sample_rate)
|
| 97 |
+
in_band = banded > C.BYPASS_BAND_RATIO_MIN
|
| 98 |
+
active = qualify_run(loud & in_band, C.BYPASS_MIN_CONSECUTIVE)
|
| 99 |
+
|
| 100 |
+
t_active = float(active.sum()) * C.BYPASS_FRAME_S
|
| 101 |
+
if active.any():
|
| 102 |
+
e_active = float(np.median(rms[active]))
|
| 103 |
+
if e_noise > C.RMS_EPSILON:
|
| 104 |
+
snr_active = 20.0 * np.log10(e_active / e_noise)
|
| 105 |
+
else:
|
| 106 |
+
snr_active = float("inf")
|
| 107 |
+
else:
|
| 108 |
+
e_active = float("nan")
|
| 109 |
+
snr_active = float("-inf")
|
| 110 |
+
|
| 111 |
+
trigger = (
|
| 112 |
+
t_voiced_s < C.BYPASS_VOICED_MAX_S
|
| 113 |
+
and t_active >= C.BYPASS_ACTIVE_MIN_S
|
| 114 |
+
and snr_active > C.BYPASS_SNR_MIN_DB
|
| 115 |
+
)
|
| 116 |
+
return BypassEvidence(rms, e_noise_raw, active, t_active, e_active,
|
| 117 |
+
float(snr_active), bool(trigger))
|
inference/stage0/constants.py
ADDED
|
@@ -0,0 +1,156 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Aria v0.2.1 stage 0 constant table.
|
| 2 |
+
|
| 3 |
+
Every number stage 0 uses lives here so the report's ``seg_ver`` can pin one
|
| 4 |
+
constant table. Each entry is tagged with where it came from:
|
| 5 |
+
|
| 6 |
+
``SPEC``
|
| 7 |
+
Written verbatim in the frozen SSOT (docs/m1/aria-v021-complete-scheme.md).
|
| 8 |
+
``PLACEHOLDER``
|
| 9 |
+
An engineering placeholder the SSOT explicitly defers or leaves unstated.
|
| 10 |
+
These are NOT calibrated conclusions and must not be quoted as such.
|
| 11 |
+
``ENGINEERING``
|
| 12 |
+
Needed to make a spec formula computable (frame length, hop, lag range).
|
| 13 |
+
Not a threshold the product decides on; recorded so results reproduce.
|
| 14 |
+
|
| 15 |
+
Do not edit a number here without recording it as a deviation in the
|
| 16 |
+
milestone report.
|
| 17 |
+
"""
|
| 18 |
+
|
| 19 |
+
from __future__ import annotations
|
| 20 |
+
|
| 21 |
+
from dataclasses import dataclass
|
| 22 |
+
|
| 23 |
+
SEG_VER = "stage0-v0.2.1.2" # .1 Silero context fix; .2 WADA G statistic
|
| 24 |
+
|
| 25 |
+
# --- Audio ---------------------------------------------------------------
|
| 26 |
+
TARGET_SR = 16_000 # SPEC §2: A is 16 kHz mono.
|
| 27 |
+
|
| 28 |
+
# --- Two-level boundary trim (SPEC §3.1.1) ------------------------------
|
| 29 |
+
EDGE_TRIM_S = 0.250 # SPEC: whole-recording head/tail trim.
|
| 30 |
+
SUBBOUNDARY_TRIM_S = 0.025 # SPEC: per VAD sub-segment head/tail trim.
|
| 31 |
+
# PLACEHOLDER per handoff §2.3 list.
|
| 32 |
+
|
| 33 |
+
# --- Silero VAD (SPEC §3.1.1, Q11-D1) -----------------------------------
|
| 34 |
+
VAD_THRESHOLD = 0.5 # SPEC: speech_prob >= 0.5.
|
| 35 |
+
VAD_MIN_SILENCE_S = 0.100 # SPEC: pauses shorter than 100 ms are merged.
|
| 36 |
+
VAD_CHUNK_SAMPLES = 512 # ENGINEERING: Silero v5 16 kHz frame.
|
| 37 |
+
VAD_CONTEXT_SAMPLES = 64 # ENGINEERING: Silero v5 ONNX left context
|
| 38 |
+
# prepended to every chunk (upstream OnnxWrapper).
|
| 39 |
+
VAD_MODEL_SHA256 = (
|
| 40 |
+
"1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3"
|
| 41 |
+
) # ENGINEERING: sha256 of SileroVAD.onnx used for every number in the M1 report.
|
| 42 |
+
|
| 43 |
+
# --- Periodic voiced confirmation (SPEC §3.1.1 Step 4) ------------------
|
| 44 |
+
PERIODICITY_PEAK_MIN = 0.45 # SPEC: normalized autocorrelation peak > 0.45.
|
| 45 |
+
PERIOD_FRAME_S = 0.040 # ENGINEERING: one period of a 25 Hz ... (40 ms).
|
| 46 |
+
PERIOD_HOP_S = 0.010 # ENGINEERING: 10 ms hop mirrors frame analysis.
|
| 47 |
+
F0_MIN_HZ = 60.0 # ENGINEERING: lag search lower bound.
|
| 48 |
+
F0_MAX_HZ = 400.0 # ENGINEERING: lag search upper bound.
|
| 49 |
+
PERIOD_RMS_FLOOR_RATIO = 0.01 # ENGINEERING: frames under 1% of the loudest
|
| 50 |
+
# frame RMS are treated as silence (ac on
|
| 51 |
+
# pure noise is meaningless).
|
| 52 |
+
|
| 53 |
+
# --- voiced_ok / gates (SPEC §3.1.1, §3.1.4) ----------------------------
|
| 54 |
+
VOICED_MIN_S = 10.0 # SPEC: T_voiced >= 10 s (Q06-D6 option B).
|
| 55 |
+
VOICED_RATIO_MIN = 0.30 # SPEC: voiced_ratio >= 0.30.
|
| 56 |
+
SNR_MIN_DB = 10.0 # SPEC: R2 threshold.
|
| 57 |
+
VOWEL_COVERAGE_MIN = 0.30 # SPEC: R4 threshold.
|
| 58 |
+
MISS_RATE_MAX = 0.50 # SPEC: R5 threshold.
|
| 59 |
+
MIN_BLOCKS = 20 # SPEC: W_窗数, 500 ms analysis blocks.
|
| 60 |
+
CVA_BLOCK_S = 0.500 # SPEC §3.4.1: 500 ms blocks.
|
| 61 |
+
|
| 62 |
+
# --- Severe bypass (SPEC §3.1.2) ----------------------------------------
|
| 63 |
+
BYPASS_FRAME_S = 0.050 # SPEC: 50 ms non-overlapping frames.
|
| 64 |
+
BYPASS_NOISE_PERCENTILE = 10.0 # SPEC: E_noise = P10 of frame RMS.
|
| 65 |
+
BYPASS_ACTIVE_OVER_NOISE = 2.0 # SPEC: +6 dB above noise floor.
|
| 66 |
+
BYPASS_BAND_HZ = (300.0, 4000.0) # SPEC: voice band.
|
| 67 |
+
BYPASS_BAND_RATIO_MIN = 0.60 # SPEC: energy fraction in voice band > 0.60.
|
| 68 |
+
BYPASS_MIN_CONSECUTIVE = 3 # SPEC: >= 3 consecutive frames.
|
| 69 |
+
BYPASS_ACTIVE_MIN_S = 10.0 # SPEC: T_active >= 10 s.
|
| 70 |
+
BYPASS_VOICED_MAX_S = 2.0 # SPEC: T_voiced < 2 s.
|
| 71 |
+
BYPASS_SNR_MIN_DB = 6.0 # SPEC: SNR_active > 6 dB.
|
| 72 |
+
RMS_EPSILON = 1e-12 # ENGINEERING: guards log / divide by zero.
|
| 73 |
+
|
| 74 |
+
# --- q_raw / q_dmg (SPEC §3.1.3) ----------------------------------------
|
| 75 |
+
CLIP_HARD_LIMIT = 0.99 # SPEC: |x| >= 0.99 counts as clipped.
|
| 76 |
+
CLIP_FLAT_LIMIT = 0.50 # SPEC: |x| >= 0.50 and flat counts as clipped.
|
| 77 |
+
CLIP_FLAT_DELTA = 1e-5 # SPEC: consecutive-sample flatness tolerance.
|
| 78 |
+
CLIP_FLAT_LOOKBACK = 2 # SPEC: max_{k=1,2} flatness against t-k.
|
| 79 |
+
SPECTRAL_WELCH_SEGMENT = 1024 # ENGINEERING: 64 ms at 16 kHz.
|
| 80 |
+
LF_RUMBLE_HZ = 120.0 # SPEC: f < 120 Hz.
|
| 81 |
+
TILT_LOW_HZ = (200.0, 1000.0) # SPEC: tilt denominator band.
|
| 82 |
+
TILT_HIGH_HZ = (2000.0, 4000.0) # SPEC: tilt numerator band.
|
| 83 |
+
Q_CLIP_SCALE = 0.05 # SPEC: q_clip = clip(clipping_ratio / 0.05).
|
| 84 |
+
Q_REVERB_T60_ONSET_S = 0.40 # SPEC: q_reverb knee.
|
| 85 |
+
Q_REVERB_T60_SCALE_S = 0.40 # SPEC: q_reverb scale.
|
| 86 |
+
Q_SNR_CLEAN_DB = 20.0 # SPEC: q_snr = clamp((20 - snr) / 10).
|
| 87 |
+
Q_SNR_SCALE_DB = 10.0 # SPEC: q_snr scale.
|
| 88 |
+
|
| 89 |
+
# --- Unstated stage-0 estimators (deviations, see report) ---------------
|
| 90 |
+
# T60_est: SSOT writes q_reverb but never defines how T60_est is measured.
|
| 91 |
+
T60_WINDOW_S = 0.300 # PLACEHOLDER: Schroeder decay fit window.
|
| 92 |
+
T60_MIN_R2 = 0.90 # PLACEHOLDER: fall back to 0 when fit is poor.
|
| 93 |
+
# codec_artifact_score: SSOT writes q_codec but never defines the score.
|
| 94 |
+
CODEC_HIGH_BAND_HZ = (6_500.0, 7_800.0) # PLACEHOLDER
|
| 95 |
+
CODEC_MID_BAND_HZ = (300.0, 3_400.0) # PLACEHOLDER
|
| 96 |
+
CODEC_RATIO_REF = 0.02 # PLACEHOLDER: high/mid ratio treated as clean.
|
| 97 |
+
# Vowel-like region detection: SSOT gives no detector, only the ratio.
|
| 98 |
+
VOWEL_PEAK_PROMINENCE_DB = 2.0 # PLACEHOLDER: intensity-peak prominence.
|
| 99 |
+
VOWEL_PEAK_MIN_GAP_S = 0.150 # PLACEHOLDER: syllable-nucleus minimum spacing.
|
| 100 |
+
VOWEL_REGION_S = 0.250 # PLACEHOLDER: peak expanded to fixed window.
|
| 101 |
+
VOWEL_F0_RANGE_HZ = (60.0, 400.0) # PLACEHOLDER: vowel-like requires periodicity.
|
| 102 |
+
VOWEL_F1_RANGE_HZ = (250.0, 950.0) # PLACEHOLDER: one clear low formant peak.
|
| 103 |
+
# WADA-SNR (Kim & Stern 2008): the paper's G statistic, table rebuilt by
|
| 104 |
+
# simulation from the paper's Gamma(0.4) speech model.
|
| 105 |
+
WADA_GAMMA_SHAPE = 0.4 # SPEC-cited paper constant.
|
| 106 |
+
WADA_SIM_SAMPLES = 2_000_000 # ENGINEERING: Monte-Carlo samples per table.
|
| 107 |
+
WADA_GRID_DB = (-20.0, 100.0) # ENGINEERING: paper table range.
|
| 108 |
+
WADA_AMPLITUDE_FLOOR = 1e-10 # ENGINEERING: log guard for exact zeros.
|
| 109 |
+
SNR_REPORT_MIN_DB = -20.0 # ENGINEERING: WADA table lower end.
|
| 110 |
+
SNR_REPORT_MAX_DB = 100.0 # ENGINEERING: WADA table upper end.
|
| 111 |
+
DIGITAL_SILENCE_RMS = 1e-6 # ENGINEERING: floor for E_noise so -120 dBFS
|
| 112 |
+
# digital silence cannot make every frame active.
|
| 113 |
+
WADA_SIM_SEED = 20261017 # ENGINEERING: reproducibility.
|
| 114 |
+
|
| 115 |
+
# --- Missingness (14 acoustic features, M1 hook only) -------------------
|
| 116 |
+
N_ACOUSTIC_FEATURES = 14
|
| 117 |
+
KEY3_FEATURES = ("cpps_db", "hnr_db", "f0_family") # SPEC: CPPs / HNR / F0-or-GNE.
|
| 118 |
+
MISS_GROUP_F0 = ("jitter_local", "jitter_rap", "jitter_ppq5",
|
| 119 |
+
"f0_median", "f0_iqr", "f0_cv", "voiced_fraction")
|
| 120 |
+
MISS_GROUP_ENERGY = ("hnr_db", "intensity_median", "intensity_iqr")
|
| 121 |
+
MISS_GROUP_SPECTRAL = ("cpps_db", "shimmer_local", "shimmer_apq3", "shimmer_apq5")
|
| 122 |
+
|
| 123 |
+
# --- Reject copy (not in SSOT; drafted here, flagged in the report) -----
|
| 124 |
+
REJECT_TEXT = {
|
| 125 |
+
"R1": "这段录音里能确认的持续发声不够 10 秒,状态参考分算不准。"
|
| 126 |
+
"请在安静的地方自然地说满一分钟以上再录一次。",
|
| 127 |
+
"R2": "录音环境偏吵,声音被背景噪声盖住了,测出来的状态参考会失真。"
|
| 128 |
+
"请换个安静一些的地方,靠近一点再录一次。",
|
| 129 |
+
"R3": "这段录音里能稳定发声的部分偏少,可能说得太短或中途停顿较多。"
|
| 130 |
+
"请连续自然地说一分钟左右再录一次。",
|
| 131 |
+
"R4": "录音里元音样的连续发声偏少,这次的内容分布不利于稳定比较。"
|
| 132 |
+
"请用平时说话的方式连续说一段,不用刻意念字。",
|
| 133 |
+
"R5": "这段录音里多项声学特征无法可靠计算,状态参考分不够可信。"
|
| 134 |
+
"请重录一次,尽量远离噪声源。",
|
| 135 |
+
"M_T5": "这段录音里核心的声学指标没能稳定提取,无法给出状态参考。"
|
| 136 |
+
"请重录一次;如果多次出现,可以换一台设备试试。",
|
| 137 |
+
"W_窗数": "录音里可分析的片段太少,不足以给出稳定的状态参考。"
|
| 138 |
+
"请自然地说满一分钟以上再录一次。",
|
| 139 |
+
}
|
| 140 |
+
|
| 141 |
+
|
| 142 |
+
def constant_table() -> dict:
|
| 143 |
+
"""Name -> value for everything upper-case in this module (JSON-safe)."""
|
| 144 |
+
import json
|
| 145 |
+
mod = globals()
|
| 146 |
+
out = {}
|
| 147 |
+
for name in sorted(mod):
|
| 148 |
+
if not name.isupper():
|
| 149 |
+
continue
|
| 150 |
+
value = mod[name]
|
| 151 |
+
try:
|
| 152 |
+
json.dumps(value)
|
| 153 |
+
except TypeError:
|
| 154 |
+
continue
|
| 155 |
+
out[name] = value
|
| 156 |
+
return out
|
inference/stage0/gates.py
ADDED
|
@@ -0,0 +1,78 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Seven stage-0 reject gates (SSOT §3.1.4).
|
| 2 |
+
|
| 3 |
+
Priority follows the SSOT listing order R1, R2, R3, R4, R5, M_T5, W_窗数; the
|
| 4 |
+
first triggered gate supplies ``reject_code`` / ``reject_text`` and every
|
| 5 |
+
triggered gate is retained in ``RejectDecision.triggered`` so nothing is hidden.
|
| 6 |
+
|
| 7 |
+
``bypass_severe`` wins over every gate: a recording that looks like a severe
|
| 8 |
+
voice gets the red-flag path, not a "try again" message.
|
| 9 |
+
"""
|
| 10 |
+
|
| 11 |
+
from __future__ import annotations
|
| 12 |
+
|
| 13 |
+
from dataclasses import dataclass, field
|
| 14 |
+
|
| 15 |
+
try:
|
| 16 |
+
from . import constants as C
|
| 17 |
+
except ImportError:
|
| 18 |
+
import constants as C
|
| 19 |
+
|
| 20 |
+
GATE_ORDER = ("R1", "R2", "R3", "R4", "R5", "M_T5", "W_窗数")
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
@dataclass(frozen=True)
|
| 24 |
+
class RejectDecision:
|
| 25 |
+
reject: bool
|
| 26 |
+
reject_code: str | None
|
| 27 |
+
reject_text: str | None
|
| 28 |
+
triggered: tuple[str, ...] = ()
|
| 29 |
+
|
| 30 |
+
def to_dict(self) -> dict:
|
| 31 |
+
return {
|
| 32 |
+
"reject": bool(self.reject),
|
| 33 |
+
"reject_code": self.reject_code,
|
| 34 |
+
"reject_text": self.reject_text,
|
| 35 |
+
}
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
@dataclass(frozen=True)
|
| 39 |
+
class GateInputs:
|
| 40 |
+
t_voiced_s: float
|
| 41 |
+
snr_est_db: float
|
| 42 |
+
voiced_ok: bool
|
| 43 |
+
vowel_coverage: float
|
| 44 |
+
miss_rate: float
|
| 45 |
+
key3_missing: bool
|
| 46 |
+
n_blocks: int
|
| 47 |
+
bypass_severe: bool
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def evaluate_gates(inputs: GateInputs) -> RejectDecision:
|
| 51 |
+
if inputs.bypass_severe:
|
| 52 |
+
return RejectDecision(False, None, None, ())
|
| 53 |
+
|
| 54 |
+
triggered: list[str] = []
|
| 55 |
+
if inputs.t_voiced_s < C.VOICED_MIN_S:
|
| 56 |
+
triggered.append("R1")
|
| 57 |
+
if not (inputs.snr_est_db >= C.SNR_MIN_DB): # NaN fails closed
|
| 58 |
+
triggered.append("R2")
|
| 59 |
+
if not inputs.voiced_ok:
|
| 60 |
+
triggered.append("R3")
|
| 61 |
+
if not (inputs.vowel_coverage >= C.VOWEL_COVERAGE_MIN):
|
| 62 |
+
triggered.append("R4")
|
| 63 |
+
if inputs.miss_rate > C.MISS_RATE_MAX:
|
| 64 |
+
triggered.append("R5")
|
| 65 |
+
if inputs.key3_missing:
|
| 66 |
+
triggered.append("M_T5")
|
| 67 |
+
if inputs.n_blocks < C.MIN_BLOCKS:
|
| 68 |
+
triggered.append("W_窗数")
|
| 69 |
+
|
| 70 |
+
ordered = tuple(g for g in GATE_ORDER if g in triggered)
|
| 71 |
+
if not ordered:
|
| 72 |
+
return RejectDecision(False, None, None, ())
|
| 73 |
+
return RejectDecision(
|
| 74 |
+
reject=True,
|
| 75 |
+
reject_code=ordered[0],
|
| 76 |
+
reject_text=C.REJECT_TEXT[ordered[0]],
|
| 77 |
+
triggered=ordered,
|
| 78 |
+
)
|
inference/stage0/missing.py
ADDED
|
@@ -0,0 +1,122 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Feature-availability interface and the 3-bit missingness audit.
|
| 2 |
+
|
| 3 |
+
R5 / M_T5 / miss_rate consume the 14-feature availability mask that the
|
| 4 |
+
feature layer (M2) will produce. M1 does not yet compute those features, so it
|
| 5 |
+
takes availability from a provider; the default provider derived here answers
|
| 6 |
+
only the questions stage 0 can answer on its own and is PLACEHOLDER-scoped.
|
| 7 |
+
|
| 8 |
+
The interface exists so the M2 feature layer drops in without touching the
|
| 9 |
+
schema: any object with ``availability(x_trim, segment_a) -> Sequence[bool]``
|
| 10 |
+
of length 14 can be passed to the pipeline.
|
| 11 |
+
"""
|
| 12 |
+
|
| 13 |
+
from __future__ import annotations
|
| 14 |
+
|
| 15 |
+
from dataclasses import dataclass
|
| 16 |
+
from typing import Protocol, Sequence
|
| 17 |
+
|
| 18 |
+
import numpy as np
|
| 19 |
+
|
| 20 |
+
try:
|
| 21 |
+
from . import constants as C
|
| 22 |
+
except ImportError:
|
| 23 |
+
import constants as C
|
| 24 |
+
|
| 25 |
+
try:
|
| 26 |
+
from .acoustic_missingness import ACOUSTIC_FEATURES
|
| 27 |
+
except ImportError: # stage0 used standalone
|
| 28 |
+
ACOUSTIC_FEATURES = (
|
| 29 |
+
"hnr_db", "cpps_db", "jitter_local", "jitter_rap", "jitter_ppq5",
|
| 30 |
+
"shimmer_local", "shimmer_apq3", "shimmer_apq5", "f0_median",
|
| 31 |
+
"f0_iqr", "f0_cv", "voiced_fraction", "intensity_median", "intensity_iqr",
|
| 32 |
+
)
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
class AvailabilityProvider(Protocol):
|
| 36 |
+
def availability(self, segment_a: np.ndarray,
|
| 37 |
+
voiced_ratio: float) -> Sequence[bool]:
|
| 38 |
+
"""Length-14 boolean availability for the acoustic feature layout."""
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
@dataclass
|
| 42 |
+
class Stage0Availability:
|
| 43 |
+
"""PLACEHOLDER provider: derives what stage 0 evidence can support.
|
| 44 |
+
|
| 45 |
+
Rules (all provisional, replaced by the M2 feature extractor):
|
| 46 |
+
|
| 47 |
+
* recorded audio below 60 s cannot feed a long-window cepstral estimate;
|
| 48 |
+
* F0 family (jitter, f0 statistics, voiced_fraction) needs at least 2 s of
|
| 49 |
+
periodic voiced audio;
|
| 50 |
+
* HNR / intensity need at least 1 s of periodic voiced audio;
|
| 51 |
+
* spectral family (CPPs, shimmer) needs at least 3 s of periodic voiced
|
| 52 |
+
audio and a non-silent segment.
|
| 53 |
+
|
| 54 |
+
Duration-only rules are exactly the kind of proxy the red-line "gates must
|
| 55 |
+
not target the voice itself" rule forbids if left in place, so this
|
| 56 |
+
provider is marked provisional in every report it feeds.
|
| 57 |
+
"""
|
| 58 |
+
|
| 59 |
+
name: str = "stage0-availability-provisional"
|
| 60 |
+
|
| 61 |
+
def availability(self, segment_a: np.ndarray, voiced_ratio: float,
|
| 62 |
+
t_voiced_s: float | None = None) -> np.ndarray:
|
| 63 |
+
t_voiced = (float(t_voiced_s) if t_voiced_s is not None
|
| 64 |
+
else len(segment_a) / C.TARGET_SR)
|
| 65 |
+
silent = segment_a.size == 0 or float(np.max(np.abs(segment_a))) < 1e-6
|
| 66 |
+
f0_ok = (not silent) and t_voiced >= 2.0
|
| 67 |
+
energy_ok = (not silent) and t_voiced >= 1.0
|
| 68 |
+
spectral_ok = (not silent) and t_voiced >= 3.0
|
| 69 |
+
groups = {
|
| 70 |
+
"f0": f0_ok,
|
| 71 |
+
"energy": energy_ok,
|
| 72 |
+
"spectral": spectral_ok,
|
| 73 |
+
}
|
| 74 |
+
mapping = {}
|
| 75 |
+
for name in C.MISS_GROUP_F0:
|
| 76 |
+
mapping[name] = groups["f0"]
|
| 77 |
+
for name in C.MISS_GROUP_ENERGY:
|
| 78 |
+
mapping[name] = groups["energy"]
|
| 79 |
+
for name in C.MISS_GROUP_SPECTRAL:
|
| 80 |
+
mapping[name] = groups["spectral"]
|
| 81 |
+
mask = np.array([bool(mapping.get(name, energy_ok))
|
| 82 |
+
for name in ACOUSTIC_FEATURES], dtype=bool)
|
| 83 |
+
return mask
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
def miss_rate(mask: Sequence[bool]) -> float:
|
| 87 |
+
mask = np.asarray(mask, dtype=bool)
|
| 88 |
+
if mask.size != C.N_ACOUSTIC_FEATURES:
|
| 89 |
+
raise ValueError(
|
| 90 |
+
f"availability mask must have {C.N_ACOUSTIC_FEATURES} entries, got {mask.size}")
|
| 91 |
+
return float(1.0 - mask.mean())
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
def miss_code(mask: Sequence[bool]) -> int:
|
| 95 |
+
"""3-bit coarse pattern: [F0 family | energy/HNR family | spectral family].
|
| 96 |
+
|
| 97 |
+
A bit is set when *every* feature in that family is missing, mirroring the
|
| 98 |
+
SSOT wording "#{...}全缺".
|
| 99 |
+
"""
|
| 100 |
+
mask = np.asarray(mask, dtype=bool)
|
| 101 |
+
index = {name: i for i, name in enumerate(ACOUSTIC_FEATURES)}
|
| 102 |
+
groups = (C.MISS_GROUP_F0, C.MISS_GROUP_ENERGY, C.MISS_GROUP_SPECTRAL)
|
| 103 |
+
code = 0
|
| 104 |
+
for bit, names in enumerate(groups):
|
| 105 |
+
members = [index[n] for n in names if n in index]
|
| 106 |
+
if members and not mask[members].any():
|
| 107 |
+
code |= (1 << bit)
|
| 108 |
+
return code
|
| 109 |
+
|
| 110 |
+
|
| 111 |
+
def key3_missing(mask: Sequence[bool]) -> bool:
|
| 112 |
+
"""CPPs / HNR / F0-or-GNE: ALL three unavailable -> hard reject M_T5."""
|
| 113 |
+
mask = np.asarray(mask, dtype=bool)
|
| 114 |
+
index = {name: i for i, name in enumerate(ACOUSTIC_FEATURES)}
|
| 115 |
+
cpps = index.get("cpps_db")
|
| 116 |
+
hnr = index.get("hnr_db")
|
| 117 |
+
f0_members = [index[n] for n in ("f0_median", "f0_iqr", "f0_cv", "voiced_fraction")
|
| 118 |
+
if n in index]
|
| 119 |
+
cpps_missing = cpps is not None and not mask[cpps]
|
| 120 |
+
hnr_missing = hnr is not None and not mask[hnr]
|
| 121 |
+
f0_missing = bool(f0_members) and not mask[f0_members].any()
|
| 122 |
+
return bool(cpps_missing and hnr_missing and f0_missing)
|
inference/stage0/pipeline.py
ADDED
|
@@ -0,0 +1,163 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Stage 0 end-to-end: waveform (+ metadata) -> segment A + quality report.
|
| 2 |
+
|
| 3 |
+
Order follows SSOT §3.1 exactly: segment, estimate the bypass, score quality,
|
| 4 |
+
resolve missingness, evaluate gates, build the 21-field report. Nothing here
|
| 5 |
+
writes files; callers decide persistence and reruns stay deterministic because
|
| 6 |
+
every step is a pure function of the input PCM.
|
| 7 |
+
"""
|
| 8 |
+
|
| 9 |
+
from __future__ import annotations
|
| 10 |
+
|
| 11 |
+
from dataclasses import dataclass, field
|
| 12 |
+
|
| 13 |
+
import numpy as np
|
| 14 |
+
|
| 15 |
+
try:
|
| 16 |
+
from . import constants as C
|
| 17 |
+
from .audio import LoadedAudio
|
| 18 |
+
from .bypass import bypass_evidence
|
| 19 |
+
from .gates import GateInputs, evaluate_gates
|
| 20 |
+
from .missing import (AvailabilityProvider, Stage0Availability, key3_missing,
|
| 21 |
+
miss_code, miss_rate)
|
| 22 |
+
from .quality import quality_vectors, vowel_like_mask
|
| 23 |
+
from .report import Stage0Report, build_report
|
| 24 |
+
from .segment import SegmentationResult, segment
|
| 25 |
+
from .vad import SileroVad, VadSegment
|
| 26 |
+
except ImportError:
|
| 27 |
+
import constants as C
|
| 28 |
+
from audio import LoadedAudio
|
| 29 |
+
from bypass import bypass_evidence
|
| 30 |
+
from gates import GateInputs, evaluate_gates
|
| 31 |
+
from missing import (AvailabilityProvider, Stage0Availability, key3_missing,
|
| 32 |
+
miss_code, miss_rate)
|
| 33 |
+
from quality import quality_vectors, vowel_like_mask
|
| 34 |
+
from report import Stage0Report, build_report
|
| 35 |
+
from segment import SegmentationResult, segment
|
| 36 |
+
from vad import SileroVad, VadSegment
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
@dataclass
|
| 40 |
+
class Stage0Result:
|
| 41 |
+
report: dict
|
| 42 |
+
segment_a: np.ndarray
|
| 43 |
+
segmentation: SegmentationResult
|
| 44 |
+
diagnostics: dict = field(default_factory=dict)
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
def run_stage0(audio: LoadedAudio,
|
| 48 |
+
vad_segments: list[VadSegment] | None = None,
|
| 49 |
+
*,
|
| 50 |
+
vad: SileroVad | None = None,
|
| 51 |
+
availability: AvailabilityProvider | None = None,
|
| 52 |
+
device_id: str | None = None,
|
| 53 |
+
distance_bin: str | None = None,
|
| 54 |
+
dnsmos: dict | None = None) -> Stage0Result:
|
| 55 |
+
"""Run stage 0 on one recording.
|
| 56 |
+
|
| 57 |
+
``vad_segments`` may be supplied directly (tests, cached runs); otherwise a
|
| 58 |
+
``vad`` detector is required.
|
| 59 |
+
"""
|
| 60 |
+
if vad_segments is None:
|
| 61 |
+
if vad is None:
|
| 62 |
+
raise ValueError("run_stage0 needs either vad_segments or a vad detector")
|
| 63 |
+
vad_segments = vad.segments(audio.pcm)
|
| 64 |
+
|
| 65 |
+
seg = segment(audio.pcm, vad_segments)
|
| 66 |
+
evidence = bypass_evidence(seg.x_trim, seg.t_voiced_s)
|
| 67 |
+
|
| 68 |
+
# Median RMS over periodic frames, in the same frame grid as the bypass.
|
| 69 |
+
frame = int(round(C.BYPASS_FRAME_S * C.TARGET_SR))
|
| 70 |
+
voiced_frames = _voiced_frame_rms(seg, frame)
|
| 71 |
+
signal_rms = float(np.median(voiced_frames)) if voiced_frames.size else float("nan")
|
| 72 |
+
|
| 73 |
+
provider = availability or Stage0Availability()
|
| 74 |
+
availability_mask = np.asarray(
|
| 75 |
+
provider.availability(seg.segment_a, seg.voiced_ratio, seg.t_voiced_s),
|
| 76 |
+
dtype=bool)
|
| 77 |
+
rate = miss_rate(availability_mask)
|
| 78 |
+
code = miss_code(availability_mask)
|
| 79 |
+
key3_missing_flag = key3_missing(availability_mask)
|
| 80 |
+
|
| 81 |
+
vowels = vowel_like_mask(seg.segment_a, _voiced_mask_on_a(seg))
|
| 82 |
+
vowel_cov = (float(vowels.sum()) / len(vowels)) if len(vowels) else 0.0
|
| 83 |
+
|
| 84 |
+
quality = quality_vectors(
|
| 85 |
+
x_pcm=seg.x_pcm, x_trim=seg.x_trim, segment_a=seg.segment_a,
|
| 86 |
+
voiced_mask=seg.voiced_mask, voiced_ratio=seg.voiced_ratio,
|
| 87 |
+
e_noise=evidence.e_noise, n_missing=int((~availability_mask).sum()),
|
| 88 |
+
signal_rms=signal_rms)
|
| 89 |
+
|
| 90 |
+
voiced_ok = bool(seg.t_voiced_s >= C.VOICED_MIN_S
|
| 91 |
+
and seg.voiced_ratio >= C.VOICED_RATIO_MIN)
|
| 92 |
+
decision = evaluate_gates(GateInputs(
|
| 93 |
+
t_voiced_s=seg.t_voiced_s,
|
| 94 |
+
snr_est_db=quality.snr_est_db,
|
| 95 |
+
voiced_ok=voiced_ok,
|
| 96 |
+
vowel_coverage=vowel_cov,
|
| 97 |
+
miss_rate=rate,
|
| 98 |
+
key3_missing=key3_missing_flag,
|
| 99 |
+
n_blocks=seg.n_blocks,
|
| 100 |
+
bypass_severe=evidence.trigger,
|
| 101 |
+
))
|
| 102 |
+
|
| 103 |
+
report = build_report(
|
| 104 |
+
device_id=device_id,
|
| 105 |
+
distance_bin=distance_bin,
|
| 106 |
+
input_spec=audio.input_spec(),
|
| 107 |
+
t_raw_s=seg.t_raw_s,
|
| 108 |
+
t_speech_s=seg.t_speech_s,
|
| 109 |
+
t_voiced_s=seg.t_voiced_s,
|
| 110 |
+
voiced_ratio=seg.voiced_ratio,
|
| 111 |
+
voiced_ok=voiced_ok,
|
| 112 |
+
bypass_severe=evidence.trigger,
|
| 113 |
+
vowel_coverage=vowel_cov,
|
| 114 |
+
q_raw=quality.q_raw,
|
| 115 |
+
q_dmg=quality.q_dmg,
|
| 116 |
+
d=quality.d,
|
| 117 |
+
snr_db=quality.snr_est_db,
|
| 118 |
+
dnsmos=dnsmos,
|
| 119 |
+
miss_rate=rate,
|
| 120 |
+
miss_code=code,
|
| 121 |
+
key3_missing=key3_missing_flag,
|
| 122 |
+
decision=decision,
|
| 123 |
+
)
|
| 124 |
+
|
| 125 |
+
diagnostics = {
|
| 126 |
+
"triggered_gates": list(decision.triggered),
|
| 127 |
+
"subsegments": len(seg.subsegments),
|
| 128 |
+
"dropped_short": seg.dropped_short,
|
| 129 |
+
"n_blocks": seg.n_blocks,
|
| 130 |
+
"e_noise": evidence.e_noise,
|
| 131 |
+
"t_active_s": evidence.t_active_s,
|
| 132 |
+
"snr_active_db": evidence.snr_active_db,
|
| 133 |
+
"availability_provider": getattr(provider, "name", type(provider).__name__),
|
| 134 |
+
"availability_mask": availability_mask.tolist(),
|
| 135 |
+
"snr_components": quality.diagnostics,
|
| 136 |
+
"periodicity_peak_median": (float(np.median(seg.periodicity.peak))
|
| 137 |
+
if seg.periodicity.peak.size else None),
|
| 138 |
+
"periodicity_frames": int(seg.periodicity.peak.size),
|
| 139 |
+
}
|
| 140 |
+
return Stage0Result(report=report, segment_a=seg.segment_a,
|
| 141 |
+
segmentation=seg, diagnostics=diagnostics)
|
| 142 |
+
|
| 143 |
+
|
| 144 |
+
def _voiced_frame_rms(seg: SegmentationResult, frame: int) -> np.ndarray:
|
| 145 |
+
"""RMS of the bypass-grid frames whose centre lies in a periodic region."""
|
| 146 |
+
n_frames = len(seg.x_trim) // frame
|
| 147 |
+
if n_frames == 0:
|
| 148 |
+
return np.zeros(0)
|
| 149 |
+
usable = seg.x_trim[:n_frames * frame].reshape(n_frames, frame)
|
| 150 |
+
rms = np.sqrt(np.mean(usable ** 2, axis=1))
|
| 151 |
+
centres = (np.arange(n_frames) * frame + frame // 2).astype(int)
|
| 152 |
+
return rms[seg.voiced_mask[centres]]
|
| 153 |
+
|
| 154 |
+
|
| 155 |
+
def _voiced_mask_on_a(seg: SegmentationResult) -> np.ndarray:
|
| 156 |
+
"""Rebuild the per-sample periodic mask for the concatenated A.
|
| 157 |
+
|
| 158 |
+
A is built from contiguous periodic runs, so its time base is simply all
|
| 159 |
+
True by construction; keeping the function explicit documents that the
|
| 160 |
+
vowel detector and the periodicity test share a time base only via this
|
| 161 |
+
construction, never by index coincidence.
|
| 162 |
+
"""
|
| 163 |
+
return np.ones(len(seg.segment_a), dtype=bool)
|
inference/stage0/quality.py
ADDED
|
@@ -0,0 +1,303 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""q_raw (8 physical dimensions), q_dmg (4 normalized damage dimensions), d.
|
| 2 |
+
|
| 3 |
+
SSOT §3.1.3. Two of the four damage detectors have no formula anywhere in the
|
| 4 |
+
SSOT (``T60_est`` and ``codec_artifact_score``); they are implemented here as
|
| 5 |
+
explicit PLACEHOLDER estimators and repeated verbatim in the milestone report.
|
| 6 |
+
``vowel_coverage`` likewise has only a ratio in the SSOT — the detector below
|
| 7 |
+
follows the De Jong & Wempe (2009) syllable-nucleus idea the older scheme
|
| 8 |
+
document cites, with every free parameter marked PLACEHOLDER.
|
| 9 |
+
|
| 10 |
+
``snr_est_db`` per SSOT is ``min(20 log10(signal_rms / E_noise), WADA_SNR(A))``.
|
| 11 |
+
The paper's lookup table is not in the repository, so WADA-SNR here is the
|
| 12 |
+
paper's statistic G = log E|x| - E log|x| inverted through a table rebuilt by
|
| 13 |
+
simulation from the paper's model (Gamma(0.4) speech amplitudes, Gaussian
|
| 14 |
+
noise, fixed seed). That is a deviation from "use the published table".
|
| 15 |
+
"""
|
| 16 |
+
|
| 17 |
+
from __future__ import annotations
|
| 18 |
+
|
| 19 |
+
from dataclasses import dataclass
|
| 20 |
+
|
| 21 |
+
import numpy as np
|
| 22 |
+
|
| 23 |
+
try:
|
| 24 |
+
from . import constants as C
|
| 25 |
+
except ImportError:
|
| 26 |
+
import constants as C
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
# --------------------------------------------------------------------------
|
| 30 |
+
# WADA-SNR (Kim & Stern, Interspeech 2008)
|
| 31 |
+
# --------------------------------------------------------------------------
|
| 32 |
+
_WADA_TABLE: tuple[np.ndarray, np.ndarray] | None = None
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def _wada_statistic(signal: np.ndarray) -> float:
|
| 36 |
+
"""Kim & Stern G = log(E|x|) - E(log|x|) over the whole waveform."""
|
| 37 |
+
magnitude = np.maximum(np.abs(np.asarray(signal, dtype=np.float64)),
|
| 38 |
+
C.WADA_AMPLITUDE_FLOOR)
|
| 39 |
+
if magnitude.size < 2:
|
| 40 |
+
return float("nan")
|
| 41 |
+
return float(np.log(magnitude.mean()) - np.log(magnitude).mean())
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def wada_lookup_table():
|
| 45 |
+
"""Rebuild the paper's G -> SNR table by simulation (deterministic seed).
|
| 46 |
+
|
| 47 |
+
DEVIATION from "use the published table": the Kim & Stern (2008) table is
|
| 48 |
+
not in the repository. It is regenerated with the paper's own model: clean
|
| 49 |
+
speech amplitudes ~ Gamma(shape 0.4) with random sign, unit power; additive
|
| 50 |
+
unit Gaussian noise scaled to each grid SNR. G rises monotonically with
|
| 51 |
+
SNR; ``np.maximum.accumulate`` removes Monte-Carlo wiggle at the noisy end.
|
| 52 |
+
"""
|
| 53 |
+
global _WADA_TABLE
|
| 54 |
+
if _WADA_TABLE is not None:
|
| 55 |
+
return _WADA_TABLE
|
| 56 |
+
rng = np.random.default_rng(C.WADA_SIM_SEED)
|
| 57 |
+
n = C.WADA_SIM_SAMPLES
|
| 58 |
+
speech = rng.gamma(C.WADA_GAMMA_SHAPE, 1.0, n) * rng.choice([-1.0, 1.0], n)
|
| 59 |
+
speech /= np.sqrt(np.mean(speech ** 2))
|
| 60 |
+
noise = rng.standard_normal(n)
|
| 61 |
+
grid = np.arange(C.WADA_GRID_DB[0], C.WADA_GRID_DB[1] + 0.5, 1.0)
|
| 62 |
+
stats = np.array([_wada_statistic(speech + noise * 10.0 ** (-snr / 20.0))
|
| 63 |
+
for snr in grid])
|
| 64 |
+
stats = np.maximum.accumulate(stats)
|
| 65 |
+
_WADA_TABLE = (stats, grid)
|
| 66 |
+
return _WADA_TABLE
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def wada_snr_db(signal: np.ndarray, sample_rate: int = C.TARGET_SR) -> float:
|
| 70 |
+
"""Invert the G -> SNR table; values outside clamp to the grid ends."""
|
| 71 |
+
value = _wada_statistic(signal)
|
| 72 |
+
if not np.isfinite(value):
|
| 73 |
+
return float("nan")
|
| 74 |
+
stats, grid = wada_lookup_table()
|
| 75 |
+
estimate = float(np.interp(value, stats, grid))
|
| 76 |
+
return float(np.clip(estimate, C.SNR_REPORT_MIN_DB, C.SNR_REPORT_MAX_DB))
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
# --------------------------------------------------------------------------
|
| 80 |
+
# Damage detectors written as explicit placeholders
|
| 81 |
+
# --------------------------------------------------------------------------
|
| 82 |
+
def t60_estimate(x_trim: np.ndarray, sample_rate: int = C.TARGET_SR) -> float:
|
| 83 |
+
"""PLACEHOLDER T60 from a Schroeder backward integral on the quietest tail.
|
| 84 |
+
|
| 85 |
+
The SSOT never defines how T60_est is measured. This estimator takes the
|
| 86 |
+
last ``T60_WINDOW_S`` of the recording, integrates energy backwards, fits
|
| 87 |
+
a line to the -5..-25 dB part of the decay, and returns ``3 * 60 / slope``.
|
| 88 |
+
Poor fits (R^2 < T60_MIN_R2) return 0.0 so q_reverb stays silent rather
|
| 89 |
+
than inventing reverb. Replace with a calibrated detector before release.
|
| 90 |
+
"""
|
| 91 |
+
window = int(round(C.T60_WINDOW_S * sample_rate))
|
| 92 |
+
if len(x_trim) < window or window <= 0:
|
| 93 |
+
return 0.0
|
| 94 |
+
tail = np.asarray(x_trim[-window:], dtype=np.float64)
|
| 95 |
+
energy = np.cumsum((tail ** 2)[::-1])[::-1]
|
| 96 |
+
if energy[0] <= C.RMS_EPSILON:
|
| 97 |
+
return 0.0
|
| 98 |
+
edc_db = 10.0 * np.log10(np.maximum(energy / energy[0], 1e-20))
|
| 99 |
+
times = np.arange(len(edc_db)) / sample_rate
|
| 100 |
+
mask = (edc_db <= -5.0) & (edc_db >= -25.0)
|
| 101 |
+
if mask.sum() < 10:
|
| 102 |
+
return 0.0
|
| 103 |
+
slope, intercept = np.polyfit(times[mask], edc_db[mask], 1)
|
| 104 |
+
fitted = slope * times[mask] + intercept
|
| 105 |
+
residual = edc_db[mask] - fitted
|
| 106 |
+
ss_tot = float(np.sum((edc_db[mask] - edc_db[mask].mean()) ** 2))
|
| 107 |
+
r2 = 1.0 - float(np.sum(residual ** 2)) / ss_tot if ss_tot > 0 else 0.0
|
| 108 |
+
if slope >= 0 or r2 < C.T60_MIN_R2:
|
| 109 |
+
return 0.0
|
| 110 |
+
return float(np.clip(-60.0 / slope, 0.0, 2.0))
|
| 111 |
+
|
| 112 |
+
|
| 113 |
+
def codec_artifact_score(spectrum: np.ndarray, freqs: np.ndarray) -> float:
|
| 114 |
+
"""PLACEHOLDER codec score: high-band energy relative to the speech band.
|
| 115 |
+
|
| 116 |
+
Low-bitrate codecs and aggressive noise suppression cut the 6.5-7.8 kHz
|
| 117 |
+
shelf while leaving 300-3400 Hz mostly intact. The SSOT gives no formula,
|
| 118 |
+
so this is a placeholder ratio mapped by ``CODEC_RATIO_REF``; 0 = clean.
|
| 119 |
+
"""
|
| 120 |
+
total = float(spectrum.sum())
|
| 121 |
+
if total <= C.RMS_EPSILON:
|
| 122 |
+
return 0.0
|
| 123 |
+
high = float(spectrum[(freqs >= C.CODEC_HIGH_BAND_HZ[0]) &
|
| 124 |
+
(freqs <= C.CODEC_HIGH_BAND_HZ[1])].sum())
|
| 125 |
+
mid = float(spectrum[(freqs >= C.CODEC_MID_BAND_HZ[0]) &
|
| 126 |
+
(freqs <= C.CODEC_MID_BAND_HZ[1])].sum())
|
| 127 |
+
if mid <= C.RMS_EPSILON:
|
| 128 |
+
return 1.0
|
| 129 |
+
ratio = high / mid
|
| 130 |
+
return float(np.clip(1.0 - ratio / C.CODEC_RATIO_REF, 0.0, 1.0))
|
| 131 |
+
|
| 132 |
+
|
| 133 |
+
# --------------------------------------------------------------------------
|
| 134 |
+
# Clipping (original PCM, absolute full-scale, plus flat-top detection)
|
| 135 |
+
# --------------------------------------------------------------------------
|
| 136 |
+
def clipping_ratio(x_pcm: np.ndarray) -> float:
|
| 137 |
+
"""SSOT clipping definition on the original, un-resampled-scale PCM.
|
| 138 |
+
|
| 139 |
+
``|x| >= 0.99`` (hard limit) OR ``|x| >= 0.50`` with two consecutive
|
| 140 |
+
near-zero sample differences (flat rail). Never relative to max|x|.
|
| 141 |
+
"""
|
| 142 |
+
x = np.asarray(x_pcm, dtype=np.float64)
|
| 143 |
+
if x.size == 0:
|
| 144 |
+
return 0.0
|
| 145 |
+
hard = np.abs(x) >= C.CLIP_HARD_LIMIT
|
| 146 |
+
if x.size <= C.CLIP_FLAT_LOOKBACK:
|
| 147 |
+
return float(hard.mean())
|
| 148 |
+
diffs = [np.abs(x[C.CLIP_FLAT_LOOKBACK:] - x[C.CLIP_FLAT_LOOKBACK - k:x.size - k])
|
| 149 |
+
for k in range(1, C.CLIP_FLAT_LOOKBACK + 1)]
|
| 150 |
+
flat = np.maximum.reduce(diffs) < C.CLIP_FLAT_DELTA
|
| 151 |
+
loud = np.abs(x[C.CLIP_FLAT_LOOKBACK:]) >= C.CLIP_FLAT_LIMIT
|
| 152 |
+
flat_top = np.concatenate((np.zeros(C.CLIP_FLAT_LOOKBACK, dtype=bool), flat & loud))
|
| 153 |
+
return float(np.mean(hard | flat_top))
|
| 154 |
+
|
| 155 |
+
|
| 156 |
+
# --------------------------------------------------------------------------
|
| 157 |
+
# Vowel-like coverage
|
| 158 |
+
# --------------------------------------------------------------------------
|
| 159 |
+
def _intensity_envelope(signal: np.ndarray, sample_rate: int) -> np.ndarray:
|
| 160 |
+
frame = int(round(0.010 * sample_rate))
|
| 161 |
+
hop = frame // 2
|
| 162 |
+
n_frames = max(1, 1 + (len(signal) - frame) // hop) if len(signal) >= frame else 1
|
| 163 |
+
env = np.empty(n_frames)
|
| 164 |
+
for index in range(n_frames):
|
| 165 |
+
a = index * hop
|
| 166 |
+
env[index] = np.sqrt(np.mean(signal[a:a + frame] ** 2))
|
| 167 |
+
return env
|
| 168 |
+
|
| 169 |
+
|
| 170 |
+
def vowel_like_mask(segment_a: np.ndarray, voiced_mask: np.ndarray,
|
| 171 |
+
sample_rate: int = C.TARGET_SR) -> np.ndarray:
|
| 172 |
+
"""PLACEHOLDER vowel-like regions on A (De Jong & Wempe syllable nuclei).
|
| 173 |
+
|
| 174 |
+
Peaks in the 10 ms intensity envelope with >= 2 dB prominence and >= 150 ms
|
| 175 |
+
spacing are expanded to 250 ms regions clipped to A, required to overlap a
|
| 176 |
+
periodic frame, and merged. Every constant is a placeholder; the SSOT only
|
| 177 |
+
fixes the ratio ``vowel_coverage = vowel-like duration / T_voiced``.
|
| 178 |
+
"""
|
| 179 |
+
n = len(segment_a)
|
| 180 |
+
out = np.zeros(n, dtype=bool)
|
| 181 |
+
if n == 0:
|
| 182 |
+
return out
|
| 183 |
+
env = _intensity_envelope(segment_a, sample_rate)
|
| 184 |
+
if len(env) < 3:
|
| 185 |
+
return out
|
| 186 |
+
env_db = 20.0 * np.log10(np.maximum(env, C.RMS_EPSILON))
|
| 187 |
+
frame = int(round(0.010 * sample_rate))
|
| 188 |
+
hop = frame // 2
|
| 189 |
+
min_gap = int(round(C.VOWEL_PEAK_MIN_GAP_S / (hop / sample_rate)))
|
| 190 |
+
half = int(round(C.VOWEL_REGION_S / 2.0 * sample_rate))
|
| 191 |
+
last_peak = -min_gap
|
| 192 |
+
for index in range(1, len(env_db) - 1):
|
| 193 |
+
if not (env_db[index] > env_db[index - 1] and env_db[index] >= env_db[index + 1]):
|
| 194 |
+
continue
|
| 195 |
+
neighbourhood = env_db[max(0, index - 15):index + 16]
|
| 196 |
+
if env_db[index] - np.median(neighbourhood) < C.VOWEL_PEAK_PROMINENCE_DB:
|
| 197 |
+
continue
|
| 198 |
+
if index - last_peak < min_gap:
|
| 199 |
+
continue
|
| 200 |
+
center = index * hop
|
| 201 |
+
a = max(0, center - half)
|
| 202 |
+
b = min(n, center + half)
|
| 203 |
+
if b <= a:
|
| 204 |
+
continue
|
| 205 |
+
region = np.zeros(n, dtype=bool)
|
| 206 |
+
region[a:b] = True
|
| 207 |
+
if (region & voiced_mask).sum() < (b - a) // 2:
|
| 208 |
+
continue
|
| 209 |
+
out |= region
|
| 210 |
+
last_peak = index
|
| 211 |
+
return out
|
| 212 |
+
|
| 213 |
+
|
| 214 |
+
# --------------------------------------------------------------------------
|
| 215 |
+
# Quality vector assembly
|
| 216 |
+
# --------------------------------------------------------------------------
|
| 217 |
+
@dataclass(frozen=True)
|
| 218 |
+
class QualityVectors:
|
| 219 |
+
q_raw: np.ndarray # 8 physical dimensions, SSOT order
|
| 220 |
+
q_dmg: np.ndarray # [q_clip, q_reverb, q_snr, q_codec]
|
| 221 |
+
d: float
|
| 222 |
+
snr_est_db: float
|
| 223 |
+
diagnostics: dict
|
| 224 |
+
|
| 225 |
+
|
| 226 |
+
def quality_vectors(x_pcm: np.ndarray, x_trim: np.ndarray, segment_a: np.ndarray,
|
| 227 |
+
voiced_mask: np.ndarray, voiced_ratio: float,
|
| 228 |
+
e_noise: float, n_missing: int,
|
| 229 |
+
signal_rms: float | None = None,
|
| 230 |
+
sample_rate: int = C.TARGET_SR) -> QualityVectors:
|
| 231 |
+
"""Assemble q_raw / q_dmg / d exactly in SSOT §3.1.3 order.
|
| 232 |
+
|
| 233 |
+
``voiced_mask`` is the per-sample periodic mask on the same time base as
|
| 234 |
+
``x_trim``; ``signal_rms`` is the median RMS over periodic frames and is
|
| 235 |
+
passed in by the caller that already computed frame RMS.
|
| 236 |
+
"""
|
| 237 |
+
if signal_rms is None or not np.isfinite(signal_rms):
|
| 238 |
+
signal_rms = float(np.sqrt(np.mean(segment_a ** 2))) if segment_a.size else 0.0
|
| 239 |
+
|
| 240 |
+
from scipy.signal import welch
|
| 241 |
+
|
| 242 |
+
if segment_a.size >= C.SPECTRAL_WELCH_SEGMENT:
|
| 243 |
+
freqs, spectrum = welch(segment_a, fs=sample_rate,
|
| 244 |
+
nperseg=C.SPECTRAL_WELCH_SEGMENT)
|
| 245 |
+
elif segment_a.size:
|
| 246 |
+
freqs, spectrum = welch(segment_a, fs=sample_rate,
|
| 247 |
+
nperseg=max(2, segment_a.size))
|
| 248 |
+
else:
|
| 249 |
+
freqs, spectrum = np.zeros(1), np.zeros(1)
|
| 250 |
+
total_energy = float(spectrum.sum())
|
| 251 |
+
if total_energy > C.RMS_EPSILON:
|
| 252 |
+
flatness = float(np.exp(np.mean(np.log(np.maximum(spectrum, 1e-20)))) /
|
| 253 |
+
np.mean(spectrum))
|
| 254 |
+
lf_ratio = float(spectrum[freqs < C.LF_RUMBLE_HZ].sum() / total_energy)
|
| 255 |
+
tilt = 10.0 * np.log10(
|
| 256 |
+
max(float(spectrum[(freqs >= C.TILT_HIGH_HZ[0]) &
|
| 257 |
+
(freqs <= C.TILT_HIGH_HZ[1])].sum()), 1e-20) /
|
| 258 |
+
max(float(spectrum[(freqs >= C.TILT_LOW_HZ[0]) &
|
| 259 |
+
(freqs <= C.TILT_LOW_HZ[1])].sum()), 1e-20))
|
| 260 |
+
else:
|
| 261 |
+
flatness, lf_ratio, tilt = float("nan"), float("nan"), float("nan")
|
| 262 |
+
|
| 263 |
+
# snr_est_db = min(energy ratio, WADA-SNR) — the stricter of the two.
|
| 264 |
+
if e_noise > C.RMS_EPSILON and signal_rms > 0:
|
| 265 |
+
energy_snr = 20.0 * np.log10(signal_rms / e_noise)
|
| 266 |
+
else:
|
| 267 |
+
energy_snr = float("nan")
|
| 268 |
+
wada = wada_snr_db(segment_a, sample_rate) if segment_a.size else float("nan")
|
| 269 |
+
candidates = [v for v in (energy_snr, wada) if np.isfinite(v)]
|
| 270 |
+
snr_est = min(candidates) if candidates else float("nan")
|
| 271 |
+
|
| 272 |
+
clip_ratio = clipping_ratio(x_pcm)
|
| 273 |
+
recording_level = (20.0 * np.log10(np.sqrt(np.mean(x_trim ** 2)))
|
| 274 |
+
if x_trim.size else float("-inf"))
|
| 275 |
+
|
| 276 |
+
q_raw = np.array([
|
| 277 |
+
snr_est,
|
| 278 |
+
flatness,
|
| 279 |
+
lf_ratio,
|
| 280 |
+
clip_ratio,
|
| 281 |
+
recording_level,
|
| 282 |
+
tilt,
|
| 283 |
+
voiced_ratio,
|
| 284 |
+
float(n_missing),
|
| 285 |
+
], dtype=np.float64)
|
| 286 |
+
|
| 287 |
+
q_clip = float(np.clip(clip_ratio / C.Q_CLIP_SCALE, 0.0, 1.0))
|
| 288 |
+
t60 = t60_estimate(x_trim, sample_rate)
|
| 289 |
+
q_reverb = float(np.clip((t60 - C.Q_REVERB_T60_ONSET_S) / C.Q_REVERB_T60_SCALE_S,
|
| 290 |
+
0.0, 1.0))
|
| 291 |
+
q_snr = (float(np.clip((C.Q_SNR_CLEAN_DB - snr_est) / C.Q_SNR_SCALE_DB, 0.0, 1.0))
|
| 292 |
+
if np.isfinite(snr_est) else 1.0)
|
| 293 |
+
q_codec = codec_artifact_score(spectrum, freqs)
|
| 294 |
+
q_dmg = np.array([q_clip, q_reverb, q_snr, q_codec], dtype=np.float64)
|
| 295 |
+
d = float(1.0 - np.prod(1.0 - q_dmg))
|
| 296 |
+
return QualityVectors(
|
| 297 |
+
q_raw=q_raw,
|
| 298 |
+
q_dmg=q_dmg,
|
| 299 |
+
d=d,
|
| 300 |
+
snr_est_db=float(snr_est) if np.isfinite(snr_est) else float("nan"),
|
| 301 |
+
diagnostics={"energy_snr_db": energy_snr, "wada_snr_db": wada,
|
| 302 |
+
"t60_est_s": t60, "clipping_ratio": clip_ratio},
|
| 303 |
+
)
|
inference/stage0/report.py
ADDED
|
@@ -0,0 +1,147 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Quality report Schema v0.2.1: exactly 21 fields, 5 groups (SSOT §2.1).
|
| 2 |
+
|
| 3 |
+
Field names and order are frozen. ``validate_report`` refuses missing keys and
|
| 4 |
+
unknown keys, so a schema drift cannot pass silently. Anything stage 0 computes
|
| 5 |
+
that is not a schema field (bypass evidence, gate list, periodicity stats) goes
|
| 6 |
+
into a separate diagnostics object that is never merged into the report.
|
| 7 |
+
"""
|
| 8 |
+
|
| 9 |
+
from __future__ import annotations
|
| 10 |
+
|
| 11 |
+
import json
|
| 12 |
+
from dataclasses import dataclass, field
|
| 13 |
+
|
| 14 |
+
import numpy as np
|
| 15 |
+
|
| 16 |
+
try:
|
| 17 |
+
from . import constants as C
|
| 18 |
+
from .gates import RejectDecision
|
| 19 |
+
except ImportError:
|
| 20 |
+
import constants as C
|
| 21 |
+
from gates import RejectDecision
|
| 22 |
+
|
| 23 |
+
SCHEMA_VERSION = "v0.2.1"
|
| 24 |
+
|
| 25 |
+
SCHEMA_GROUPS: dict[str, tuple[str, ...]] = {
|
| 26 |
+
"G1_collection": ("device_id", "distance_bin", "input_spec"),
|
| 27 |
+
"G2_segmentation": ("T_raw", "T_speech", "T_voiced", "voiced_ratio",
|
| 28 |
+
"voiced_ok", "bypass_severe", "vowel_coverage",
|
| 29 |
+
"seg_ver"),
|
| 30 |
+
"G3_quality_fingerprint": ("q_raw", "q_dmg", "d", "snr_db", "dnsmos"),
|
| 31 |
+
"G4_feature_missingness": ("miss_rate", "miss_code", "key3_missing"),
|
| 32 |
+
# SSOT row 21 is one field with two components: reject_code / reject_text.
|
| 33 |
+
"G5_rejection": ("reject", ("reject_code", "reject_text")),
|
| 34 |
+
}
|
| 35 |
+
# One report key per SSOT row. Row 21 renders as two JSON keys (reject_code,
|
| 36 |
+
# reject_text) but is a single schema entry; the slash keeps that visible.
|
| 37 |
+
ROW_KEYS = {"reject_code / reject_text": ("reject_code", "reject_text")}
|
| 38 |
+
# Rows as listed in SSOT §2.1, in order: 21 rows, one of which expands into
|
| 39 |
+
# two JSON keys. ``SCHEMA_FIELDS`` is therefore 21 rows; ``SCHEMA_KEYS`` is the
|
| 40 |
+
# 22 JSON keys that must all be present.
|
| 41 |
+
SCHEMA_FIELDS = tuple(name for group in SCHEMA_GROUPS.values() for name in group)
|
| 42 |
+
SCHEMA_KEYS = tuple(
|
| 43 |
+
key
|
| 44 |
+
for name in SCHEMA_FIELDS
|
| 45 |
+
for key in ((name,) if isinstance(name, str) else ROW_KEYS[" / ".join(name)]))
|
| 46 |
+
assert len(SCHEMA_FIELDS) == 21, len(SCHEMA_FIELDS)
|
| 47 |
+
assert len(SCHEMA_KEYS) == 22, len(SCHEMA_KEYS)
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
@dataclass
|
| 51 |
+
class Stage0Report:
|
| 52 |
+
"""The 21 fields plus the audio A the report describes."""
|
| 53 |
+
|
| 54 |
+
report: dict
|
| 55 |
+
segment_a: np.ndarray
|
| 56 |
+
diagnostics: dict = field(default_factory=dict)
|
| 57 |
+
|
| 58 |
+
def to_json(self, indent: int | None = None) -> str:
|
| 59 |
+
validate_report(self.report)
|
| 60 |
+
return json.dumps(self.report, ensure_ascii=False, indent=indent,
|
| 61 |
+
allow_nan=False)
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
def validate_report(report: dict) -> None:
|
| 65 |
+
keys = set(report)
|
| 66 |
+
expected = set(SCHEMA_KEYS)
|
| 67 |
+
missing = expected - keys
|
| 68 |
+
unknown = keys - expected
|
| 69 |
+
if missing:
|
| 70 |
+
raise ValueError(f"report is missing schema fields: {sorted(missing)}")
|
| 71 |
+
if unknown:
|
| 72 |
+
raise ValueError(f"report has unknown keys outside Schema v0.2.1: {sorted(unknown)}")
|
| 73 |
+
q_raw = report["q_raw"]
|
| 74 |
+
if not isinstance(q_raw, (list, tuple)) or len(q_raw) != 8:
|
| 75 |
+
raise ValueError("q_raw must have 8 entries")
|
| 76 |
+
q_dmg = report["q_dmg"]
|
| 77 |
+
if not isinstance(q_dmg, (list, tuple)) or len(q_dmg) != 4:
|
| 78 |
+
raise ValueError("q_dmg must have 4 entries")
|
| 79 |
+
if not all(0.0 <= float(v) <= 1.0 for v in q_dmg):
|
| 80 |
+
raise ValueError("q_dmg entries must lie in [0, 1]")
|
| 81 |
+
if not 0.0 <= float(report["d"]) <= 1.0:
|
| 82 |
+
raise ValueError("d must lie in [0, 1]")
|
| 83 |
+
if report["distance_bin"] not in (None, "15cm", "30cm", "50cm"):
|
| 84 |
+
raise ValueError("distance_bin must be null or one of 15cm/30cm/50cm")
|
| 85 |
+
if report["reject"] and not (report["reject_code"] and report["reject_text"]):
|
| 86 |
+
raise ValueError("a rejected recording must carry both reject_code and reject_text")
|
| 87 |
+
|
| 88 |
+
|
| 89 |
+
def build_report(*, device_id: str | None,
|
| 90 |
+
distance_bin: str | None,
|
| 91 |
+
input_spec: dict,
|
| 92 |
+
t_raw_s: float, t_speech_s: float, t_voiced_s: float,
|
| 93 |
+
voiced_ratio: float, voiced_ok: bool, bypass_severe: bool,
|
| 94 |
+
vowel_coverage: float,
|
| 95 |
+
q_raw: np.ndarray, q_dmg: np.ndarray, d: float,
|
| 96 |
+
snr_db: float, dnsmos: dict | None,
|
| 97 |
+
miss_rate: float, miss_code: int, key3_missing: bool,
|
| 98 |
+
decision: RejectDecision) -> dict:
|
| 99 |
+
"""Assemble the 21 fields. ``dnsmos`` stays null until offline backfill."""
|
| 100 |
+
report = {
|
| 101 |
+
"device_id": device_id,
|
| 102 |
+
"distance_bin": distance_bin,
|
| 103 |
+
"input_spec": input_spec,
|
| 104 |
+
"T_raw": float(t_raw_s),
|
| 105 |
+
"T_speech": float(t_speech_s),
|
| 106 |
+
"T_voiced": float(t_voiced_s),
|
| 107 |
+
"voiced_ratio": float(voiced_ratio),
|
| 108 |
+
"voiced_ok": bool(voiced_ok),
|
| 109 |
+
"bypass_severe": bool(bypass_severe),
|
| 110 |
+
"vowel_coverage": float(vowel_coverage),
|
| 111 |
+
"seg_ver": C.SEG_VER,
|
| 112 |
+
"q_raw": [float(v) for v in q_raw],
|
| 113 |
+
"q_dmg": [float(v) for v in q_dmg],
|
| 114 |
+
"d": float(d),
|
| 115 |
+
"snr_db": float(snr_db),
|
| 116 |
+
"dnsmos": dnsmos,
|
| 117 |
+
"miss_rate": float(miss_rate),
|
| 118 |
+
"miss_code": int(miss_code),
|
| 119 |
+
"key3_missing": bool(key3_missing),
|
| 120 |
+
"reject": bool(decision.reject),
|
| 121 |
+
"reject_code": decision.reject_code,
|
| 122 |
+
"reject_text": decision.reject_text,
|
| 123 |
+
}
|
| 124 |
+
validate_report(report)
|
| 125 |
+
return report
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def field_completeness(report: dict) -> dict:
|
| 129 |
+
"""Per-field presence audit for the milestone report.
|
| 130 |
+
|
| 131 |
+
A field counts as complete when it is not null and, for numerics, finite.
|
| 132 |
+
``device_id`` / ``distance_bin`` are legitimately null for PVQD, which
|
| 133 |
+
carries neither; the audit shows that instead of inventing values.
|
| 134 |
+
"""
|
| 135 |
+
result = {}
|
| 136 |
+
for group, names in SCHEMA_GROUPS.items():
|
| 137 |
+
for name in names:
|
| 138 |
+
value = report.get(name)
|
| 139 |
+
complete = value is not None
|
| 140 |
+
if isinstance(value, float):
|
| 141 |
+
complete = complete and np.isfinite(value)
|
| 142 |
+
if isinstance(value, (list, tuple)):
|
| 143 |
+
complete = complete and all(
|
| 144 |
+
isinstance(v, float) and np.isfinite(v) or isinstance(v, (int, str))
|
| 145 |
+
for v in value)
|
| 146 |
+
result[name] = {"group": group, "complete": bool(complete)}
|
| 147 |
+
return result
|
inference/stage0/run_corpus.py
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Batch stage-0 runner over a corpus of recordings (PVQD first).
|
| 2 |
+
|
| 3 |
+
Outputs, all under ``--out``:
|
| 4 |
+
|
| 5 |
+
``reports/<stem>.json`` the 21-field Schema v0.2.1 report per recording
|
| 6 |
+
``diagnostics/<stem>.json`` gate list, bypass evidence, periodicity stats
|
| 7 |
+
``segments/<stem>.wav`` segment A at 16 kHz mono (``--write-audio``)
|
| 8 |
+
``summary.json`` aggregate counts the milestone report cites
|
| 9 |
+
``env.json`` interpreter, library versions, constant table, hashes
|
| 10 |
+
|
| 11 |
+
The runner never touches the locked test speakers unless ``--include-locked``
|
| 12 |
+
is passed explicitly; the default reads the lock list from ``--lock-test`` when
|
| 13 |
+
one is supplied.
|
| 14 |
+
"""
|
| 15 |
+
|
| 16 |
+
from __future__ import annotations
|
| 17 |
+
|
| 18 |
+
import argparse
|
| 19 |
+
import json
|
| 20 |
+
import platform
|
| 21 |
+
import sys
|
| 22 |
+
import time
|
| 23 |
+
from collections import Counter
|
| 24 |
+
from pathlib import Path
|
| 25 |
+
|
| 26 |
+
import numpy as np
|
| 27 |
+
|
| 28 |
+
try:
|
| 29 |
+
from . import constants as C
|
| 30 |
+
from .audio import load_wav
|
| 31 |
+
from .pipeline import run_stage0
|
| 32 |
+
from .report import SCHEMA_FIELDS, field_completeness
|
| 33 |
+
from .vad import SileroVad
|
| 34 |
+
except ImportError:
|
| 35 |
+
import constants as C
|
| 36 |
+
from audio import load_wav
|
| 37 |
+
from pipeline import run_stage0
|
| 38 |
+
from report import SCHEMA_FIELDS, field_completeness
|
| 39 |
+
from vad import SileroVad
|
| 40 |
+
|
| 41 |
+
|
| 42 |
+
def _read_lock_list(path: Path | None) -> set[str]:
|
| 43 |
+
if path is None or not path.exists():
|
| 44 |
+
return set()
|
| 45 |
+
return {line.strip() for line in path.read_text().splitlines()
|
| 46 |
+
if line.strip() and not line.startswith("#")}
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def speaker_from_stem(stem: str) -> str:
|
| 50 |
+
"""PVQD speaker id is the stem up to the first separator."""
|
| 51 |
+
for sep in (" ", "_"):
|
| 52 |
+
if sep in stem:
|
| 53 |
+
return stem.split(sep, 1)[0]
|
| 54 |
+
return stem
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
def environment() -> dict:
|
| 58 |
+
import onnxruntime
|
| 59 |
+
import scipy
|
| 60 |
+
import soundfile
|
| 61 |
+
|
| 62 |
+
return {
|
| 63 |
+
"python": sys.version,
|
| 64 |
+
"platform": platform.platform(),
|
| 65 |
+
"machine": platform.machine(),
|
| 66 |
+
"numpy": np.__version__,
|
| 67 |
+
"scipy": scipy.__version__,
|
| 68 |
+
"soundfile": soundfile.__version__,
|
| 69 |
+
"onnxruntime": onnxruntime.__version__,
|
| 70 |
+
"constant_table": C.constant_table(),
|
| 71 |
+
"vad_model_sha256": C.VAD_MODEL_SHA256,
|
| 72 |
+
"seg_ver": C.SEG_VER,
|
| 73 |
+
}
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
def main(argv: list[str] | None = None) -> int:
|
| 77 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 78 |
+
parser.add_argument("--audio-dir", required=True, type=Path)
|
| 79 |
+
parser.add_argument("--out", required=True, type=Path)
|
| 80 |
+
parser.add_argument("--model", type=Path, default=None)
|
| 81 |
+
parser.add_argument("--lock-test", type=Path, default=None,
|
| 82 |
+
help="speaker-per-line lock list to exclude")
|
| 83 |
+
parser.add_argument("--include-locked", action="store_true")
|
| 84 |
+
parser.add_argument("--write-audio", action="store_true")
|
| 85 |
+
parser.add_argument("--limit", type=int, default=None)
|
| 86 |
+
args = parser.parse_args(argv)
|
| 87 |
+
|
| 88 |
+
files = sorted(args.audio_dir.glob("*.wav"))
|
| 89 |
+
if args.limit:
|
| 90 |
+
files = files[:args.limit]
|
| 91 |
+
locked = _read_lock_list(args.lock_test)
|
| 92 |
+
|
| 93 |
+
args.out.mkdir(parents=True, exist_ok=True)
|
| 94 |
+
for sub in ("reports", "diagnostics", "segments"):
|
| 95 |
+
(args.out / sub).mkdir(exist_ok=True)
|
| 96 |
+
(args.out / "env.json").write_text(
|
| 97 |
+
json.dumps(environment(), ensure_ascii=False, indent=2))
|
| 98 |
+
|
| 99 |
+
vad_kwargs = {"model_path": args.model} if args.model else {}
|
| 100 |
+
detector = SileroVad(**vad_kwargs)
|
| 101 |
+
|
| 102 |
+
rows = []
|
| 103 |
+
started = time.time()
|
| 104 |
+
skipped_locked = 0
|
| 105 |
+
for index, path in enumerate(files, 1):
|
| 106 |
+
stem = path.stem
|
| 107 |
+
speaker = speaker_from_stem(stem)
|
| 108 |
+
if locked and speaker in locked and not args.include_locked:
|
| 109 |
+
skipped_locked += 1
|
| 110 |
+
continue
|
| 111 |
+
audio = load_wav(path)
|
| 112 |
+
result = run_stage0(audio, vad=detector)
|
| 113 |
+
(args.out / "reports" / f"{stem}.json").write_text(
|
| 114 |
+
json.dumps(result.report, ensure_ascii=False, indent=2, allow_nan=False))
|
| 115 |
+
(args.out / "diagnostics" / f"{stem}.json").write_text(
|
| 116 |
+
json.dumps({**result.diagnostics, "source_sha256": audio.source_sha256},
|
| 117 |
+
ensure_ascii=False, indent=2, allow_nan=False, default=str))
|
| 118 |
+
if args.write_audio:
|
| 119 |
+
import soundfile as sf
|
| 120 |
+
|
| 121 |
+
sf.write(str(args.out / "segments" / f"{stem}.wav"),
|
| 122 |
+
result.segment_a.astype(np.float32), C.TARGET_SR)
|
| 123 |
+
rows.append({
|
| 124 |
+
"stem": stem,
|
| 125 |
+
"speaker_id": speaker,
|
| 126 |
+
"report": result.report,
|
| 127 |
+
"diagnostics": result.diagnostics,
|
| 128 |
+
})
|
| 129 |
+
print(f"[{index}/{len(files)}] {stem} T_voiced={result.report['T_voiced']:.2f}s "
|
| 130 |
+
f"reject={result.report['reject_code']}")
|
| 131 |
+
|
| 132 |
+
summary = summarize(rows, skipped_locked=skipped_locked,
|
| 133 |
+
elapsed_s=time.time() - started)
|
| 134 |
+
(args.out / "summary.json").write_text(
|
| 135 |
+
json.dumps(summary, ensure_ascii=False, indent=2, allow_nan=False, default=str))
|
| 136 |
+
print(json.dumps({k: v for k, v in summary.items()
|
| 137 |
+
if k not in ("per_file",)}, ensure_ascii=False, indent=2))
|
| 138 |
+
return 0
|
| 139 |
+
|
| 140 |
+
|
| 141 |
+
def summarize(rows: list[dict], *, skipped_locked: int, elapsed_s: float) -> dict:
|
| 142 |
+
"""Aggregate the milestone-report numbers from per-file rows."""
|
| 143 |
+
n = len(rows)
|
| 144 |
+
codes = Counter()
|
| 145 |
+
for row in rows:
|
| 146 |
+
if row["report"]["reject"]:
|
| 147 |
+
codes[row["report"]["reject_code"]] += 1
|
| 148 |
+
else:
|
| 149 |
+
codes["pass"] += 1
|
| 150 |
+
triggered = Counter()
|
| 151 |
+
for row in rows:
|
| 152 |
+
for gate in row["diagnostics"]["triggered_gates"]:
|
| 153 |
+
triggered[gate] += 1
|
| 154 |
+
|
| 155 |
+
completeness = Counter()
|
| 156 |
+
for row in rows:
|
| 157 |
+
for name, info in field_completeness(row["report"]).items():
|
| 158 |
+
if info["complete"]:
|
| 159 |
+
completeness[name] += 1
|
| 160 |
+
|
| 161 |
+
def values(key):
|
| 162 |
+
return np.asarray([row["report"][key] for row in rows], dtype=np.float64)
|
| 163 |
+
|
| 164 |
+
return {
|
| 165 |
+
"files": n,
|
| 166 |
+
"skipped_locked": skipped_locked,
|
| 167 |
+
"elapsed_s": elapsed_s,
|
| 168 |
+
"bypass_severe": int(sum(row["report"]["bypass_severe"] for row in rows)),
|
| 169 |
+
"reject_distribution": dict(sorted(codes.items())),
|
| 170 |
+
"gate_trigger_counts": dict(sorted(triggered.items())),
|
| 171 |
+
"field_complete_fraction": {name: completeness[name] / n if n else 0.0
|
| 172 |
+
for name in SCHEMA_FIELDS},
|
| 173 |
+
"T_raw": _describe(values("T_raw")),
|
| 174 |
+
"T_speech": _describe(values("T_speech")),
|
| 175 |
+
"T_voiced": _describe(values("T_voiced")),
|
| 176 |
+
"voiced_ratio": _describe(values("voiced_ratio")),
|
| 177 |
+
"vowel_coverage": _describe(values("vowel_coverage")),
|
| 178 |
+
"snr_db": _describe(values("snr_db")),
|
| 179 |
+
"d": _describe(values("d")),
|
| 180 |
+
"miss_rate": _describe(values("miss_rate")),
|
| 181 |
+
"per_file": rows,
|
| 182 |
+
}
|
| 183 |
+
|
| 184 |
+
|
| 185 |
+
def _describe(values: np.ndarray) -> dict:
|
| 186 |
+
finite = values[np.isfinite(values)]
|
| 187 |
+
if finite.size == 0:
|
| 188 |
+
return {"n": int(values.size), "finite": 0}
|
| 189 |
+
return {
|
| 190 |
+
"n": int(values.size),
|
| 191 |
+
"finite": int(finite.size),
|
| 192 |
+
"mean": float(finite.mean()),
|
| 193 |
+
"median": float(np.median(finite)),
|
| 194 |
+
"p10": float(np.percentile(finite, 10)),
|
| 195 |
+
"p90": float(np.percentile(finite, 90)),
|
| 196 |
+
"min": float(finite.min()),
|
| 197 |
+
"max": float(finite.max()),
|
| 198 |
+
}
|
| 199 |
+
|
| 200 |
+
|
| 201 |
+
if __name__ == "__main__":
|
| 202 |
+
raise SystemExit(main())
|
inference/stage0/run_requests.py
ADDED
|
@@ -0,0 +1,143 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Run stage 0 over the G6 feature-request list (one row per endpoint).
|
| 2 |
+
|
| 3 |
+
Reads ``requests.jsonl`` (see docs/m2/README.md), resolves each ``audio_path``
|
| 4 |
+
against ``--dataset``, verifies the SHA-256 before touching the audio, and
|
| 5 |
+
writes under ``--out``:
|
| 6 |
+
|
| 7 |
+
``reports.jsonl`` one line per endpoint: request keys + 21-field report
|
| 8 |
+
``diagnostics.jsonl`` gate list, bypass evidence, periodicity stats
|
| 9 |
+
``segments/<path>`` segment A as 16 kHz float32 WAV (``--write-audio``)
|
| 10 |
+
``env.json`` interpreter, libraries, constant table, input hashes
|
| 11 |
+
|
| 12 |
+
Rows keep request order, so reruns diff line by line. The output directory
|
| 13 |
+
must not already exist.
|
| 14 |
+
"""
|
| 15 |
+
|
| 16 |
+
from __future__ import annotations
|
| 17 |
+
|
| 18 |
+
import argparse
|
| 19 |
+
import hashlib
|
| 20 |
+
import json
|
| 21 |
+
import multiprocessing as mp
|
| 22 |
+
import sys
|
| 23 |
+
import time
|
| 24 |
+
from pathlib import Path
|
| 25 |
+
|
| 26 |
+
import numpy as np
|
| 27 |
+
|
| 28 |
+
sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
|
| 29 |
+
|
| 30 |
+
from stage0 import constants as C # noqa: E402
|
| 31 |
+
from stage0.audio import load_wav # noqa: E402
|
| 32 |
+
from stage0.pipeline import run_stage0 # noqa: E402
|
| 33 |
+
from stage0.run_corpus import environment # noqa: E402
|
| 34 |
+
from stage0.vad import DEFAULT_MODEL_PATH, SileroVad # noqa: E402
|
| 35 |
+
|
| 36 |
+
_VAD: SileroVad | None = None
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def _sha256(path: Path) -> str:
|
| 40 |
+
return hashlib.sha256(path.read_bytes()).hexdigest()
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
def _init(model_path: str) -> None:
|
| 44 |
+
global _VAD
|
| 45 |
+
_VAD = SileroVad(model_path=model_path)
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
def _finite_or_null(value):
|
| 49 |
+
"""JSON has no NaN/Infinity; the G6 contract wants explicit null instead."""
|
| 50 |
+
if isinstance(value, dict):
|
| 51 |
+
return {k: _finite_or_null(v) for k, v in value.items()}
|
| 52 |
+
if isinstance(value, (list, tuple)):
|
| 53 |
+
return [_finite_or_null(v) for v in value]
|
| 54 |
+
if isinstance(value, (float, np.floating)):
|
| 55 |
+
return float(value) if np.isfinite(value) else None
|
| 56 |
+
if isinstance(value, np.integer):
|
| 57 |
+
return int(value)
|
| 58 |
+
if isinstance(value, np.bool_):
|
| 59 |
+
return bool(value)
|
| 60 |
+
return value
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
def _distance_bin(request: dict) -> str | None:
|
| 64 |
+
"""Degraded endpoints carry a simulated 15/30/50 cm bin; clean is 6 cm source."""
|
| 65 |
+
value = f"{int(request['distance_cm'])}cm"
|
| 66 |
+
return value if value in ("15cm", "30cm", "50cm") else None
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def _process(job: tuple[int, dict, str, str | None]) -> tuple[int, dict, dict, np.ndarray | None]:
|
| 70 |
+
index, request, dataset, segment_dir = job
|
| 71 |
+
path = Path(dataset) / request["audio_path"]
|
| 72 |
+
if _sha256(path) != request["audio_sha256"]:
|
| 73 |
+
raise ValueError(f"sha256 mismatch for {request['audio_path']}")
|
| 74 |
+
audio = load_wav(path)
|
| 75 |
+
result = run_stage0(audio, vad=_VAD, distance_bin=_distance_bin(request))
|
| 76 |
+
diagnostics = _finite_or_null(result.diagnostics)
|
| 77 |
+
segment = result.segment_a.astype(np.float32) if segment_dir else None
|
| 78 |
+
return index, _finite_or_null(result.report), diagnostics, segment
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def main(argv: list[str] | None = None) -> int:
|
| 82 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 83 |
+
parser.add_argument("--requests", required=True, type=Path)
|
| 84 |
+
parser.add_argument("--dataset", required=True, type=Path)
|
| 85 |
+
parser.add_argument("--out", required=True, type=Path)
|
| 86 |
+
parser.add_argument("--model", type=Path, default=DEFAULT_MODEL_PATH)
|
| 87 |
+
parser.add_argument("--workers", type=int, default=1)
|
| 88 |
+
parser.add_argument("--write-audio", action="store_true")
|
| 89 |
+
parser.add_argument("--limit", type=int, default=None)
|
| 90 |
+
args = parser.parse_args(argv)
|
| 91 |
+
|
| 92 |
+
if args.out.exists():
|
| 93 |
+
raise SystemExit(f"refusing to overwrite existing output {args.out}")
|
| 94 |
+
requests = [json.loads(line) for line in args.requests.read_text().splitlines()
|
| 95 |
+
if line.strip()]
|
| 96 |
+
if args.limit:
|
| 97 |
+
requests = requests[:args.limit]
|
| 98 |
+
args.out.mkdir(parents=True)
|
| 99 |
+
segment_dir = args.out / "segments" if args.write_audio else None
|
| 100 |
+
|
| 101 |
+
env = environment()
|
| 102 |
+
env.update({
|
| 103 |
+
"requests_path": str(args.requests),
|
| 104 |
+
"requests_sha256": _sha256(args.requests),
|
| 105 |
+
"dataset": str(args.dataset),
|
| 106 |
+
"vad_model_path": str(args.model),
|
| 107 |
+
"vad_model_sha256_actual": _sha256(args.model),
|
| 108 |
+
})
|
| 109 |
+
if env["vad_model_sha256_actual"] != C.VAD_MODEL_SHA256:
|
| 110 |
+
raise SystemExit("VAD model hash differs from the pinned constant table")
|
| 111 |
+
(args.out / "env.json").write_text(json.dumps(env, ensure_ascii=False, indent=2))
|
| 112 |
+
|
| 113 |
+
jobs = [(i, r, str(args.dataset), str(segment_dir) if segment_dir else None)
|
| 114 |
+
for i, r in enumerate(requests)]
|
| 115 |
+
results: list[tuple[dict, dict] | None] = [None] * len(jobs)
|
| 116 |
+
started = time.time()
|
| 117 |
+
with mp.get_context("spawn").Pool(args.workers, initializer=_init,
|
| 118 |
+
initargs=(str(args.model),)) as pool:
|
| 119 |
+
for done, (index, report, diagnostics, segment) in enumerate(
|
| 120 |
+
pool.imap_unordered(_process, jobs, chunksize=4), 1):
|
| 121 |
+
results[index] = (report, diagnostics)
|
| 122 |
+
if segment is not None:
|
| 123 |
+
import soundfile as sf
|
| 124 |
+
|
| 125 |
+
target = segment_dir / requests[index]["audio_path"]
|
| 126 |
+
target.parent.mkdir(parents=True, exist_ok=True)
|
| 127 |
+
sf.write(str(target), segment, C.TARGET_SR, subtype="FLOAT")
|
| 128 |
+
if done % 50 == 0 or done == len(jobs):
|
| 129 |
+
print(f"[{done}/{len(jobs)}] {time.time() - started:.0f}s", flush=True)
|
| 130 |
+
|
| 131 |
+
with (args.out / "reports.jsonl").open("w") as rep, \
|
| 132 |
+
(args.out / "diagnostics.jsonl").open("w") as diag:
|
| 133 |
+
for request, (report, diagnostics) in zip(requests, results):
|
| 134 |
+
rep.write(json.dumps({**request, "quality_report": report},
|
| 135 |
+
ensure_ascii=False, allow_nan=False) + "\n")
|
| 136 |
+
diag.write(json.dumps({"audio_path": request["audio_path"], **diagnostics},
|
| 137 |
+
ensure_ascii=False, allow_nan=False, default=str) + "\n")
|
| 138 |
+
print(f"done {len(jobs)} endpoints in {time.time() - started:.0f}s")
|
| 139 |
+
return 0
|
| 140 |
+
|
| 141 |
+
|
| 142 |
+
if __name__ == "__main__":
|
| 143 |
+
raise SystemExit(main())
|
inference/stage0/segment.py
ADDED
|
@@ -0,0 +1,228 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Two-level boundary trim, periodic voiced confirmation and segment A.
|
| 2 |
+
|
| 3 |
+
SSOT §3.1.1, implemented in the order the spec writes it:
|
| 4 |
+
|
| 5 |
+
Step 1 trim 250 ms from the head and tail of the whole recording
|
| 6 |
+
Step 2 Silero VAD on the trimmed signal, merge pauses < 100 ms
|
| 7 |
+
Step 3 trim 25 ms from the head and tail of each surviving VAD sub-segment
|
| 8 |
+
Step 4 per-frame periodicity (autocorrelation peak > 0.45) -> concatenate
|
| 9 |
+
the periodic regions into A
|
| 10 |
+
|
| 11 |
+
The periodicity check runs inside the already-trimmed sub-segments, so T_speech
|
| 12 |
+
and T_voiced are both measured on the same step-3 output. Sub-segments shorter
|
| 13 |
+
than 2 x 25 ms vanish at step 3; that is a consequence of the spec, not an
|
| 14 |
+
extra gate, and ``dropped_short`` records how many.
|
| 15 |
+
"""
|
| 16 |
+
|
| 17 |
+
from __future__ import annotations
|
| 18 |
+
|
| 19 |
+
from dataclasses import dataclass, field
|
| 20 |
+
|
| 21 |
+
import numpy as np
|
| 22 |
+
|
| 23 |
+
try:
|
| 24 |
+
from . import constants as C
|
| 25 |
+
from .vad import VadSegment
|
| 26 |
+
except ImportError:
|
| 27 |
+
import constants as C
|
| 28 |
+
from vad import VadSegment
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
@dataclass
|
| 32 |
+
class Periodicity:
|
| 33 |
+
"""Frame-level periodicity for one signal.
|
| 34 |
+
|
| 35 |
+
``voiced_mask`` is a per-sample boolean: a sample is periodic when any
|
| 36 |
+
voiced frame covers it. That keeps trailing partial frames represented
|
| 37 |
+
instead of silently truncating the tail of a segment.
|
| 38 |
+
"""
|
| 39 |
+
|
| 40 |
+
frame_rms: np.ndarray
|
| 41 |
+
peak: np.ndarray
|
| 42 |
+
voiced_frame: np.ndarray
|
| 43 |
+
voiced_mask: np.ndarray
|
| 44 |
+
frame_bounds: list[tuple[int, int]] = field(default_factory=list)
|
| 45 |
+
|
| 46 |
+
@property
|
| 47 |
+
def voiced_s(self) -> float:
|
| 48 |
+
return float(self.voiced_mask.sum()) / C.TARGET_SR
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def trim_edges(samples: np.ndarray, trim_s: float,
|
| 52 |
+
sample_rate: int = C.TARGET_SR) -> np.ndarray:
|
| 53 |
+
"""Drop ``trim_s`` from both ends; return an empty array if too short."""
|
| 54 |
+
cut = int(round(trim_s * sample_rate))
|
| 55 |
+
if len(samples) <= 2 * cut:
|
| 56 |
+
return samples[:0]
|
| 57 |
+
return samples[cut:len(samples) - cut]
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def frame_bounds(n_samples: int, frame: int, hop: int) -> list[tuple[int, int]]:
|
| 61 |
+
if n_samples < frame:
|
| 62 |
+
return [(0, n_samples)] if n_samples else []
|
| 63 |
+
bounds = []
|
| 64 |
+
start = 0
|
| 65 |
+
while start + frame <= n_samples:
|
| 66 |
+
bounds.append((start, start + frame))
|
| 67 |
+
start += hop
|
| 68 |
+
if bounds and bounds[-1][1] < n_samples:
|
| 69 |
+
bounds.append((n_samples - frame, n_samples))
|
| 70 |
+
return bounds
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
def periodicity(samples: np.ndarray,
|
| 74 |
+
sample_rate: int = C.TARGET_SR,
|
| 75 |
+
frame_s: float = C.PERIOD_FRAME_S,
|
| 76 |
+
hop_s: float = C.PERIOD_HOP_S,
|
| 77 |
+
f0_min: float = C.F0_MIN_HZ,
|
| 78 |
+
f0_max: float = C.F0_MAX_HZ,
|
| 79 |
+
peak_min: float = C.PERIODICITY_PEAK_MIN) -> Periodicity:
|
| 80 |
+
"""Normalized autocorrelation peak per frame (SSOT Step 4).
|
| 81 |
+
|
| 82 |
+
The normalization is the unbiased-overlap form ``r(l) = sum(x[n] x[n+l]) /
|
| 83 |
+
sqrt(E0 * El)`` restricted to overlapping samples, so a decaying tail cannot
|
| 84 |
+
inflate the peak. Lags span the full F0 search range and the maximum over
|
| 85 |
+
that range is the frame's periodicity evidence.
|
| 86 |
+
"""
|
| 87 |
+
samples = np.asarray(samples, dtype=np.float64)
|
| 88 |
+
n = len(samples)
|
| 89 |
+
frame = int(round(frame_s * sample_rate))
|
| 90 |
+
hop = int(round(hop_s * sample_rate))
|
| 91 |
+
lag_min = max(1, int(np.floor(sample_rate / f0_max)))
|
| 92 |
+
lag_max = int(np.ceil(sample_rate / f0_min))
|
| 93 |
+
bounds = frame_bounds(n, frame, hop)
|
| 94 |
+
if not bounds:
|
| 95 |
+
empty = np.zeros(0)
|
| 96 |
+
return Periodicity(empty, empty, empty, np.zeros(n, dtype=bool), [])
|
| 97 |
+
|
| 98 |
+
rms = np.asarray([np.sqrt(np.mean(samples[a:b] ** 2)) for a, b in bounds])
|
| 99 |
+
loud_floor = C.PERIOD_RMS_FLOOR_RATIO * (rms.max() if rms.size else 0.0)
|
| 100 |
+
peaks = np.zeros(len(bounds))
|
| 101 |
+
voiced = np.zeros(len(bounds), dtype=bool)
|
| 102 |
+
voiced_mask = np.zeros(n, dtype=bool)
|
| 103 |
+
for index, (a, b) in enumerate(bounds):
|
| 104 |
+
block = samples[a:b]
|
| 105 |
+
if rms[index] <= max(loud_floor, C.RMS_EPSILON):
|
| 106 |
+
continue
|
| 107 |
+
block = block - block.mean()
|
| 108 |
+
energy = float(np.dot(block, block))
|
| 109 |
+
if energy <= 0.0:
|
| 110 |
+
continue
|
| 111 |
+
best = 0.0
|
| 112 |
+
for lag in range(lag_min, min(lag_max, len(block) - 1) + 1):
|
| 113 |
+
left = block[:-lag]
|
| 114 |
+
right = block[lag:]
|
| 115 |
+
denom = np.sqrt(float(np.dot(left, left)) * float(np.dot(right, right)))
|
| 116 |
+
if denom <= C.RMS_EPSILON:
|
| 117 |
+
continue
|
| 118 |
+
value = float(np.dot(left, right)) / denom
|
| 119 |
+
if value > best:
|
| 120 |
+
best = value
|
| 121 |
+
peaks[index] = best
|
| 122 |
+
if best > peak_min:
|
| 123 |
+
voiced[index] = True
|
| 124 |
+
voiced_mask[a:b] = True
|
| 125 |
+
return Periodicity(rms, peaks, voiced, voiced_mask, bounds)
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def contiguous_runs(mask: np.ndarray) -> list[tuple[int, int]]:
|
| 129 |
+
"""Half-open [start, end) runs of True."""
|
| 130 |
+
if not mask.any():
|
| 131 |
+
return []
|
| 132 |
+
padded = np.concatenate(([False], mask.astype(bool), [False]))
|
| 133 |
+
diff = np.diff(padded.astype(np.int8))
|
| 134 |
+
starts = np.flatnonzero(diff == 1)
|
| 135 |
+
ends = np.flatnonzero(diff == -1)
|
| 136 |
+
return [(int(a), int(b)) for a, b in zip(starts, ends)]
|
| 137 |
+
|
| 138 |
+
|
| 139 |
+
@dataclass
|
| 140 |
+
class SegmentationResult:
|
| 141 |
+
"""Everything stage 0 derives from the waveform before quality scoring."""
|
| 142 |
+
|
| 143 |
+
x_pcm: np.ndarray # original mono 16 kHz PCM (for clipping)
|
| 144 |
+
x_trim: np.ndarray # step 1 output
|
| 145 |
+
subsegments: list[tuple[int, int]] # step 3 intervals on x_trim
|
| 146 |
+
dropped_short: int
|
| 147 |
+
speech_mask: np.ndarray # per-sample bool on x_trim, step 3 output
|
| 148 |
+
voiced_mask: np.ndarray # per-sample bool on x_trim, step 4 output
|
| 149 |
+
segment_a: np.ndarray # concatenated periodic voiced audio
|
| 150 |
+
periodicity: Periodicity
|
| 151 |
+
|
| 152 |
+
@property
|
| 153 |
+
def t_raw_s(self) -> float:
|
| 154 |
+
return len(self.x_pcm) / C.TARGET_SR
|
| 155 |
+
|
| 156 |
+
@property
|
| 157 |
+
def t_speech_s(self) -> float:
|
| 158 |
+
return float(self.speech_mask.sum()) / C.TARGET_SR
|
| 159 |
+
|
| 160 |
+
@property
|
| 161 |
+
def t_voiced_s(self) -> float:
|
| 162 |
+
return float(self.voiced_mask.sum()) / C.TARGET_SR
|
| 163 |
+
|
| 164 |
+
@property
|
| 165 |
+
def voiced_ratio(self) -> float:
|
| 166 |
+
return self.t_voiced_s / self.t_speech_s if self.t_speech_s > 0 else 0.0
|
| 167 |
+
|
| 168 |
+
@property
|
| 169 |
+
def n_blocks(self) -> int:
|
| 170 |
+
"""Count of 500 ms analysis blocks on A (SSOT W_窗数)."""
|
| 171 |
+
return int(np.floor(self.t_voiced_s / C.CVA_BLOCK_S))
|
| 172 |
+
|
| 173 |
+
|
| 174 |
+
def segment(x_pcm: np.ndarray, vad_segments: list[VadSegment],
|
| 175 |
+
sample_rate: int = C.TARGET_SR) -> SegmentationResult:
|
| 176 |
+
"""Run the four SSOT steps on one recording."""
|
| 177 |
+
x_pcm = np.asarray(x_pcm, dtype=np.float64)
|
| 178 |
+
x_trim = trim_edges(x_pcm, C.EDGE_TRIM_S, sample_rate)
|
| 179 |
+
|
| 180 |
+
# Step 3: per-sub-segment 25 ms trim, measured on x_trim.
|
| 181 |
+
kept: list[tuple[int, int]] = []
|
| 182 |
+
dropped = 0
|
| 183 |
+
for seg in vad_segments:
|
| 184 |
+
start = np.clip(seg.start, 0, len(x_trim))
|
| 185 |
+
end = np.clip(seg.end, 0, len(x_trim))
|
| 186 |
+
cut = int(round(C.SUBBOUNDARY_TRIM_S * sample_rate))
|
| 187 |
+
if end - start <= 2 * cut:
|
| 188 |
+
dropped += 1
|
| 189 |
+
continue
|
| 190 |
+
kept.append((int(start + cut), int(end - cut)))
|
| 191 |
+
|
| 192 |
+
speech_mask = np.zeros(len(x_trim), dtype=bool)
|
| 193 |
+
voiced_mask = np.zeros(len(x_trim), dtype=bool)
|
| 194 |
+
pieces: list[np.ndarray] = []
|
| 195 |
+
peak_parts, rms_parts, frame_parts, bounds_parts = [], [], [], []
|
| 196 |
+
for a, b in kept:
|
| 197 |
+
speech_mask[a:b] = True
|
| 198 |
+
per = periodicity(x_trim[a:b], sample_rate=sample_rate)
|
| 199 |
+
voiced_mask[a:b] = per.voiced_mask
|
| 200 |
+
for s, e in contiguous_runs(per.voiced_mask):
|
| 201 |
+
pieces.append(x_trim[a + s:a + e])
|
| 202 |
+
peak_parts.append(per.peak)
|
| 203 |
+
rms_parts.append(per.frame_rms)
|
| 204 |
+
frame_parts.append(per.voiced_frame.astype(bool))
|
| 205 |
+
bounds_parts.extend((a + s, a + e) for s, e in per.frame_bounds)
|
| 206 |
+
|
| 207 |
+
if pieces:
|
| 208 |
+
segment_a = np.concatenate(pieces)
|
| 209 |
+
else:
|
| 210 |
+
segment_a = np.zeros(0, dtype=np.float64)
|
| 211 |
+
|
| 212 |
+
combined = Periodicity(
|
| 213 |
+
frame_rms=np.concatenate(rms_parts) if rms_parts else np.zeros(0),
|
| 214 |
+
peak=np.concatenate(peak_parts) if peak_parts else np.zeros(0),
|
| 215 |
+
voiced_frame=np.concatenate(frame_parts) if frame_parts else np.zeros(0, dtype=bool),
|
| 216 |
+
voiced_mask=voiced_mask,
|
| 217 |
+
frame_bounds=bounds_parts,
|
| 218 |
+
)
|
| 219 |
+
return SegmentationResult(
|
| 220 |
+
x_pcm=x_pcm,
|
| 221 |
+
x_trim=x_trim,
|
| 222 |
+
subsegments=kept,
|
| 223 |
+
dropped_short=dropped,
|
| 224 |
+
speech_mask=speech_mask,
|
| 225 |
+
voiced_mask=voiced_mask,
|
| 226 |
+
segment_a=segment_a,
|
| 227 |
+
periodicity=combined,
|
| 228 |
+
)
|
inference/stage0/vad.py
ADDED
|
@@ -0,0 +1,139 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Silero VAD wrapper (SSOT §3.1.1 Step 2, Q11-D1).
|
| 2 |
+
|
| 3 |
+
Only the ONNX detector is supported here; the WebRTC comparison arm and the
|
| 4 |
+
Sohn energy fallback belong to the evaluation harness, not this pipeline.
|
| 5 |
+
|
| 6 |
+
Two deliberate simplifications relative to the upstream Silero demo:
|
| 7 |
+
|
| 8 |
+
* One threshold (0.5) for both speech start and end, because the SSOT writes a
|
| 9 |
+
single ``speech_prob >= 0.5``.
|
| 10 |
+
* No ``speech_pad_ms`` and no ``min_speech_duration`` — the SSOT's only
|
| 11 |
+
boundary rules are the 250 ms whole-recording trim and the 25 ms per-segment
|
| 12 |
+
trim. Adding padding here would silently change T_speech.
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
from __future__ import annotations
|
| 16 |
+
|
| 17 |
+
import hashlib
|
| 18 |
+
from dataclasses import dataclass
|
| 19 |
+
from pathlib import Path
|
| 20 |
+
from typing import Protocol, Sequence
|
| 21 |
+
|
| 22 |
+
import numpy as np
|
| 23 |
+
|
| 24 |
+
try:
|
| 25 |
+
from . import constants as C
|
| 26 |
+
except ImportError:
|
| 27 |
+
import constants as C
|
| 28 |
+
|
| 29 |
+
DEFAULT_MODEL_PATH = Path(
|
| 30 |
+
"/Users/dannyfeng/Documents/Pitchee/Dependencies/PitcheeCore/models/SileroVAD.onnx")
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
class ProbabilityModel(Protocol):
|
| 34 |
+
"""Minimal contract needed from a detector; makes the merge logic testable."""
|
| 35 |
+
|
| 36 |
+
def probabilities(self, pcm: np.ndarray) -> np.ndarray:
|
| 37 |
+
"""Per-chunk speech probability for ``len(pcm) / VAD_CHUNK_SAMPLES`` chunks."""
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
@dataclass(frozen=True)
|
| 41 |
+
class VadSegment:
|
| 42 |
+
"""Half-open sample interval [start, end) on the VAD input."""
|
| 43 |
+
start: int
|
| 44 |
+
end: int
|
| 45 |
+
|
| 46 |
+
@property
|
| 47 |
+
def length(self) -> int:
|
| 48 |
+
return self.end - self.start
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def merge_segments(segments: Sequence[tuple[int, int]],
|
| 52 |
+
min_silence_samples: int) -> list[tuple[int, int]]:
|
| 53 |
+
"""Merge runs separated by less than ``min_silence_samples``.
|
| 54 |
+
|
| 55 |
+
The SSOT merges pauses shorter than 100 ms; a gap exactly equal to the
|
| 56 |
+
threshold is kept as a real pause (strict inequality).
|
| 57 |
+
"""
|
| 58 |
+
merged: list[list[int]] = []
|
| 59 |
+
for start, end in segments:
|
| 60 |
+
if end <= start:
|
| 61 |
+
continue
|
| 62 |
+
if merged and start - merged[-1][1] < min_silence_samples:
|
| 63 |
+
merged[-1][1] = max(merged[-1][1], end)
|
| 64 |
+
else:
|
| 65 |
+
merged.append([int(start), int(end)])
|
| 66 |
+
return [(a, b) for a, b in merged]
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
def segments_from_probabilities(probabilities: np.ndarray,
|
| 70 |
+
n_samples: int,
|
| 71 |
+
chunk: int = C.VAD_CHUNK_SAMPLES,
|
| 72 |
+
threshold: float = C.VAD_THRESHOLD,
|
| 73 |
+
min_silence_s: float = C.VAD_MIN_SILENCE_S,
|
| 74 |
+
sample_rate: int = C.TARGET_SR) -> list[VadSegment]:
|
| 75 |
+
"""Turn per-chunk probabilities into sample intervals, then merge pauses."""
|
| 76 |
+
probs = np.asarray(probabilities, dtype=np.float64).reshape(-1)
|
| 77 |
+
active = probs >= threshold
|
| 78 |
+
runs: list[tuple[int, int]] = []
|
| 79 |
+
start = None
|
| 80 |
+
for index, flag in enumerate(active):
|
| 81 |
+
if flag and start is None:
|
| 82 |
+
start = index
|
| 83 |
+
elif not flag and start is not None:
|
| 84 |
+
runs.append((start * chunk, min((index) * chunk, n_samples)))
|
| 85 |
+
start = None
|
| 86 |
+
if start is not None:
|
| 87 |
+
runs.append((start * chunk, min(len(probs) * chunk, n_samples)))
|
| 88 |
+
merged = merge_segments(runs, int(round(min_silence_s * sample_rate)))
|
| 89 |
+
return [VadSegment(a, b) for a, b in merged if b > a]
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
class SileroVad:
|
| 93 |
+
"""Silero v5 ONNX detector driven one 512-sample chunk at a time."""
|
| 94 |
+
|
| 95 |
+
def __init__(self, model_path: str | Path = DEFAULT_MODEL_PATH,
|
| 96 |
+
session=None, threads: int = 1):
|
| 97 |
+
self.model_path = Path(model_path)
|
| 98 |
+
if session is None:
|
| 99 |
+
import onnxruntime as ort
|
| 100 |
+
|
| 101 |
+
options = ort.SessionOptions()
|
| 102 |
+
options.intra_op_num_threads = threads
|
| 103 |
+
options.inter_op_num_threads = 1
|
| 104 |
+
options.log_severity_level = 3
|
| 105 |
+
session = ort.InferenceSession(
|
| 106 |
+
str(self.model_path),
|
| 107 |
+
sess_options=options,
|
| 108 |
+
providers=["CPUExecutionProvider"])
|
| 109 |
+
self.session = session
|
| 110 |
+
self.model_sha256 = (hashlib.sha256(self.model_path.read_bytes()).hexdigest()
|
| 111 |
+
if self.model_path.exists() else "")
|
| 112 |
+
|
| 113 |
+
def probabilities(self, pcm: np.ndarray, reset: bool = True) -> np.ndarray:
|
| 114 |
+
"""Per-chunk speech probability.
|
| 115 |
+
|
| 116 |
+
Silero v5 ONNX expects each 512-sample chunk prefixed by the previous
|
| 117 |
+
64 samples (``VAD_CONTEXT_SAMPLES``), exactly as the upstream
|
| 118 |
+
``OnnxWrapper`` does. Feeding bare 512-sample chunks runs without error
|
| 119 |
+
but collapses probabilities towards zero on clean speech.
|
| 120 |
+
"""
|
| 121 |
+
pcm = np.asarray(pcm, dtype=np.float32)
|
| 122 |
+
chunk = C.VAD_CHUNK_SAMPLES
|
| 123 |
+
state = np.zeros((2, 1, 128), dtype=np.float32)
|
| 124 |
+
context = np.zeros(C.VAD_CONTEXT_SAMPLES, dtype=np.float32)
|
| 125 |
+
sr = np.array(C.TARGET_SR, dtype=np.int64)
|
| 126 |
+
probs = []
|
| 127 |
+
for start in range(0, len(pcm), chunk):
|
| 128 |
+
block = pcm[start:start + chunk]
|
| 129 |
+
if len(block) < chunk:
|
| 130 |
+
block = np.pad(block, (0, chunk - len(block)))
|
| 131 |
+
framed = np.concatenate([context, block]).reshape(1, -1)
|
| 132 |
+
out, state = self.session.run(
|
| 133 |
+
None, {"input": framed, "state": state, "sr": sr})
|
| 134 |
+
probs.append(float(out.reshape(-1)[0]))
|
| 135 |
+
context = block[-C.VAD_CONTEXT_SAMPLES:]
|
| 136 |
+
return np.asarray(probs, dtype=np.float64)
|
| 137 |
+
|
| 138 |
+
def segments(self, pcm: np.ndarray) -> list[VadSegment]:
|
| 139 |
+
return segments_from_probabilities(self.probabilities(pcm), len(pcm))
|
model_card.md
CHANGED
|
@@ -1,24 +1,150 @@
|
|
| 1 |
-
# Aria v0.
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
| 4 |
|
| 5 |
-
|
| 6 |
|
| 7 |
-
##
|
| 8 |
|
| 9 |
-
|
| 10 |
-
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
|
|
|
|
|
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Aria v0.2.3.3 冷启动模型包(正式包):Model Card
|
| 2 |
|
| 3 |
+
- model_ver **`v0.2.3.3`**(refit L3,g 版本 3);feature_engine **native**(numpy/scipy 复刻,不依赖 Praat);seg_ver `stage0-v0.2.1.2`。
|
| 4 |
+
- 链路:阶段 0 段 A → native 9 变量 → L3 系数 → clip [0,10] → G6 恒等(ŵ = 0)→ g([weights/g_table.json](weights/g_table.json))→ S。常数见 [weights/model.json](weights/model.json),4 位有效数字。
|
| 5 |
+
- 取代 v0.2.3.2(草稿,praat-script)与 v0.2.0.1(breathiness 标签),两者的包存档在 `aria-training/artifacts/m4-model-package-v0.2.3.2/` 与 `aria-training/artifacts/m4-model-package/`。上一代发布包 v0.1-pre 见 [history/v0.1-pre/](history/v0.1-pre/model_card.md)。规格修订见 `aria-training/docs/spec-revisions.md`。
|
| 6 |
|
| 7 |
+
> 本卡与 `aria-training/artifacts/m4-model-package-v0.2.3.3/MODEL_CARD.md` 同源同版本。为适应本发布仓的目录布局,卡内指向训练/评估产物的链接改写为 `aria-training` 仓内路径的文字引用,并新增或改写本包专有内容(新增「本包内容」一节与「发布包自洽门」一小节,「复现」改为只列本包命令)。模型本身的数字、门结论与判定一字未改;Limitations 的条目、顺序与结论不变,仅在两条上补了本包专有说明(第 8 条:本包同时输出训练口径与产品口径两个 status;第 12 条:锁定集上未达显著)。本包不含训练与评估代码。
|
| 8 |
|
| 9 |
+
## 本包内容
|
| 10 |
|
| 11 |
+
| 路径 | 内容 |
|
| 12 |
+
|---|---|
|
| 13 |
+
| `weights/model.json` | L3 系数、clip、g 版本、冷启动阈值、dev 指标与包身份 |
|
| 14 |
+
| `weights/g_table.json` | 量纲链 g 的常数表(knots / p / σ)与验收记录 |
|
| 15 |
+
| `inference/predict.py` | 入口:`--audio`(原始录音)、`--segment-a`(段 A)、`--requests`(批量)|
|
| 16 |
+
| `inference/stage0/` | 阶段 0(seg_ver `stage0-v0.2.1.2`,13 个模块,与冻结预测所用代码逐字节相同)|
|
| 17 |
+
| `inference/native_features/` | native 九特征引擎(5 个模块,与冻结预测所用代码逐字节相同)|
|
| 18 |
|
| 19 |
+
依赖:numpy、scipy、soundfile、onnxruntime(只有阶段 0 的 VAD 需要)。阶段 0 的 Silero VAD ONNX 不随包分发,需自备并校验 SHA-256 `1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3`(`--vad-model` / `$ARIA_VAD_MODEL` / `inference/models/SileroVAD.onnx`);只走 `--segment-a` 时无需 VAD。
|
| 20 |
|
| 21 |
+
## 适用范围
|
| 22 |
|
| 23 |
+
本包是**测量内核**,不是完整的录音端产品链路。输入契约是**段 A**(16 kHz mono 波形):从段 A 往后依次计算九项特征、ABI'、S。段 A 之前的环节——从任意录音中定位并切出持续元音——不属于本包。
|
| 24 |
|
| 25 |
+
**已验证范围仅限 PVQD 协议复合录音经阶段 0 产出的段 A**:拟合池 212 人与 hold-out 12 人的全部样本都是这一口径下的段 A——PVQD 每个文件是持续 /a/、/i/ 加 CAPE-V 六句的复合录音(无人工句切),整文件过现行阶段 0 离线产出段 A(全部周期性区域拼接,训练口径)。**任意录音场景不在本包验证范围内**:面向任意录音的元音切割前端尚未建立,也未经验证;本包没有、也无法据此对任意录音的切分正确性、噪声鲁棒性或端到端输出作出任何声明。
|
| 26 |
+
|
| 27 |
+
## 变更追溯
|
| 28 |
+
|
| 29 |
+
| # | 变更 | 从 → 到 | 依据 |
|
| 30 |
+
|---|---|---|---|
|
| 31 |
+
| 1 | 目标标签 | CAPE-V breathiness / 10 → **CAPE-V 总体严重度均值 / 10** | Ryo 2026-10-18 第 1 项 |
|
| 32 |
+
| 2 | G3 CV 极差判据 | ≤ 0.10 → ≤ τ(噪声校准,本池 τ = 0.369) | Ryo 批准提案 A(规格修订 R1) |
|
| 33 |
+
| 3 | 「不劣于 L0」 | 严格比较 hold-out → 选级以 CV 为准,hold-out 只核 ≥ 0.70 | Ryo 方案 (iii)(规格修订 R1) |
|
| 34 |
+
| 4 | T_diag | 3.44 原文 → **未改,占位,待裁定** | Ryo 第 3 项挂起 |
|
| 35 |
+
| 5 | 特征引擎 | praat-script → **native** | 原生线审计 + 本包回归 |
|
| 36 |
+
|
| 37 |
+
## 开发指标(dev,非锁定测试;训练口径,R1/W 不排除)
|
| 38 |
+
|
| 39 |
+
| 项 | 主标签:CAPE-V 严重度 | 对照:CAPE-V breathiness(v0.2.0.1) |
|
| 40 |
+
|---|---|---|
|
| 41 |
+
| 拟合池 / hold-out | 212(normal 84 / mild 79 / moderate 31 / severe 18)/ 12 人 | 同 |
|
| 42 |
+
| 选中级 | **L3**(λ = 100,向已发表系数收缩) | L0 |
|
| 43 |
+
| CV rs 均值 L0 → L3 | 0.588 → 0.642(Δ +0.054) | 0.635 → 0.636(Δ +0.001) |
|
| 44 |
+
| CV 各折 rs(选中级) | 0.490 / 0.677 / 0.676 / 0.697 / 0.668 | 0.519 / 0.621 / 0.631 / 0.540 / 0.866 |
|
| 45 |
+
| CV rs 极差(选中级) | 0.207 | 0.348 |
|
| 46 |
+
| hold-out rs(L0 对照) | **0.888**(0.895) | 0.860(0.860) |
|
| 47 |
+
| hold-out MAE(ABI' 空间) | 1.255 | 2.146 |
|
| 48 |
+
| 同一模型对其他标签的 rs(拟合池 / hold-out) | breathiness 0.647 / 0.846;GRBAS-B 0.653 / 0.875 | GRBAS-B 0.664 / 0.883 |
|
| 49 |
+
|
| 50 |
+
L3 系数:b0 = 12.47;cpps −0.5062,jitter −0.3403,GNE −5.915,Hfno −1.061,HNR-D 0.06068,H1-H2 −0.004161,ShdB 4.759,Shim −0.1942,PSD 10.64。
|
| 51 |
+
|
| 52 |
+
## 锁定 Benchmark 结果(§2.5,2026-10-18 一次性执行)
|
| 53 |
+
|
| 54 |
+
比较对象:v1 冻结 CAPE-V severity 头(原生 mm,未重算,只按冻结文件哈希复用)。两侧同在 n = 57 的配对集上计算,零映射、零拟合。预测文件冻结 SHA-256 `3ff22e6a…44d69`;完整方法、终点与合规披露见 `aria-training/docs/v0233-locked-benchmark-result.md`(预测冻结记录:`aria-training/docs/v0233-locked-benchmark-prediction-freeze.md`)。
|
| 55 |
+
|
| 56 |
+
**判定:未达显著。** 主检验单侧 Wilcoxon signed-rank(方向 d > 0)W = 725,**p = 0.790005**(α = 0.05);主终点 median d = **+0.693 mm**,95% CI **[−2.408, +3.012]**(跨 0);配对符号检验 30/57,单侧 p = 0.395683。**因此不得写「v0.2.3.3 在锁定集上显著优于 v1 severity 头」。**
|
| 57 |
+
|
| 58 |
+
| 侧 | Spearman rs [95% CI] | MAE mm [95% CI] | 中位 \|误差\| mm |
|
| 59 |
+
|---|---|---|---|
|
| 60 |
+
| v1 CAPE-V severity 头 | 0.8130 [0.6983, 0.8826] | 10.951 [8.645, 13.508] | 8.593 |
|
| 61 |
+
| Aria v0.2.3.3(ABI′ × 10) | 0.7164 [0.5338, 0.8429] | 11.971 [9.553, 14.482] | 9.247 |
|
| 62 |
+
|
| 63 |
+
点估计上 v1 略优(mean d = −1.020 mm;v0.2.3.3 更准 30 人 / v1 更准 27 人 / ties 0),但两个独立 ρ 之差未作显著性检验,两列数字**只作描述**,不构成任何优势或劣势的显著性宣称。按作者 Category 分层同样只是描述:**Normal/Mild 两层两侧相当**(Normal n = 23,MAE 8.649 vs 8.366;Mild n = 17,MAE 7.390 vs 8.322),**Moderate/Severe 两层 v0.2.3.3 描述性偏弱**(Moderate n = 13,22.019 vs 18.532;Severe n = 4,17.883 vs 12.348);分层子样本很小(Severe 仅 4 人),不作分层结论。
|
| 64 |
+
|
| 65 |
+
intent-to-screen:原始分配 60 → 成功评分 58 → VAD 拒检 2(PT084、PT134)。v0.2.3.3 侧 **PT097 在正式包(训练口径)下 status = `rejected:R2|R5`**(SNR 9.93 dB < 10、miss_rate > 0.5),不产出预测;配对集因此为 n = 57,不插补、不替换人选。产品口径(七门全过)下同一批 58 人中 22 人可评分、36 人 `rejected:R1`,该表仅描述性、不参与任何终点。
|
| 66 |
+
|
| 67 |
+
**本结果不改变 v0.2.3.3 的适用范围表述,也不决定是否发布**(v0.2.3.3 按计划上线发布)。
|
| 68 |
+
|
| 69 |
+
### 发布包自洽门(2026-10-18)
|
| 70 |
+
|
| 71 |
+
用**本发布包自身的入口**(`inference/predict.py` 批模式)复算同一批 58 人,与冻结预测文件 `3ff22e6a…44d69` 逐人比对:`predictions.csv` **逐字节一致**(同为 SHA-256 `3ff22e6a…44d69`),逐人 ABI′ 最大 |Δ| = **0.0**(容差 1e-6);status 58/58 一致(scored 57 / `rejected:R2|R5` 1)。同时逐模块校验本包 `inference/` 的 13 + 5 个模块哈希与冻结运行的 `generation_manifest.json` 记录逐字相同,权重哈希与 §2.5 报告页眉所载包身份逐字相同。报告:`aria-training/artifacts/release-selfcheck-v0.2.3.3/selfcheck_report.md`。
|
| 72 |
+
|
| 73 |
+
## G3 重拟合门:修订后 PASS,原门 FAIL 并排披露
|
| 74 |
+
|
| 75 |
+
| 判据 | 原门(v0.2.1) | 修订门 R1 | 本包数值 |
|
| 76 |
+
|---|---|---|---|
|
| 77 |
+
| hold-out rs ≥ 0.70 | 通过 | 通过 | 0.888 |
|
| 78 |
+
| 不劣于原系数 | 严格 ≥ L0:**不过**(−0.007) | 不作门,仅披露 | 差 −0.007,bootstrap 95% CI [−0.26, +0.22],记为噪声内 |
|
| 79 |
+
| CV 极差 | ≤ 0.10:**不过** | ≤ τ = 0.369:**通过** | 0.207 |
|
| 80 |
+
| **结论** | **FAIL** | **PASS** | |
|
| 81 |
+
|
| 82 |
+
- τ 的定义:零模型下 5 折 rs 极差的第 95 百分位,零模型取 ρ₀ = 0.70、每折 42.4 ��,按 Fisher z 抽样(Bonett–Wright SE),蒙特卡洛 10⁶ 次。观测极差 0.207 落在重新分折噪声分布的第 50 百分位。
|
| 83 |
+
- 原结论留档:v0.2.0.1(breathiness 标签)CV 极差 0.348,原门 FAIL。这个结论没有被重判,也没有被覆盖。
|
| 84 |
+
- Step 0 对等审计:native 对 praat-script 九项特征 r ≥ 0.95(|r − 1| < 1e-12)。原生线报告 §5.1 记录段 A 对原始录音的特征偏移(H1-H2 0.65、PSD 0.82),属阶段 0 层面的问题。**比较对象口径已裁定**(Ryo 2026-10-18 六项裁决第 4 项):Step 0 只判引擎对等,已过线;段 A 对原始录音的偏移记为已知限制,归阶段 0 范畴,不在本次引擎对等范围内,不回炉。
|
| 85 |
+
|
| 86 |
+
## G4 量纲链:通过
|
| 87 |
+
|
| 88 |
+
参考样本:拟合池中作者类别 Normal 的 84 人,用 L3 打分(1 个值裁剪到 0,无并列)。中位数 1.592,σ̃ = 0.884。三段 C0 连续(接点差 1.9e-9),解析严格单调;纯查表实现与参考实现最大差 1.4e-14。
|
| 89 |
+
|
| 90 |
+
## 冷启动阈值:未标定、待裁定
|
| 91 |
+
|
| 92 |
+
> **S_diag 与冷启动红线未标定,不得作为已标定结论引用。**
|
| 93 |
+
|
| 94 |
+
T_diag 按规格原文 3.44 占位(L0 ABI 诊断阈值)。在严重度标签与 L3 下,ABI' 的含义是「预测 CAPE-V 严重度 / 10」,3.44 对应 34.4 mm,这个阈值在新口径下没有定义。由此算出的数值只是占位:S_diag = g(4.128) = 99.87,T_red^cold = min(90, 99.87) = 90,T_yellow^cold = 75,防倒挂锁不触发。待 Ryo 裁定 T_diag^(3)。本包因此只输出 ABI′、y_hat_mm 与 S 三项测量值,不输出产品档位判定。
|
| 95 |
+
|
| 96 |
+
## 引擎替换回归(praat-script → native)
|
| 97 |
+
|
| 98 |
+
对照包:同一标签、同一 R1 规则、同一 g 版本,用 praat-script 特征构建(`aria-training/artifacts/m5-engine-regression/control-praat-script/`,仅作回归对照,不发布)。容差 1e-6(审计误差量级,远低于 4 位有效数字的发布精度)。
|
| 99 |
+
|
| 100 |
+
| 检查 | 结果 |
|
| 101 |
+
|---|---|
|
| 102 |
+
| 选中级 / G3 判定 | L3 / PASS,两引擎相同 |
|
| 103 |
+
| 发布系数(4 位有效数字) | 完全相同 |
|
| 104 |
+
| g 常数表、冷启动阈值 | 完全相同 |
|
| 105 |
+
| refit 全部数值(CV、hold-out、λ 曲线等) | 最大差 5.6e-8 |
|
| 106 |
+
| dev 指标 | 最大差 2.4e-11 |
|
| 107 |
+
| 逐端点 ABI'(2,352 端点,2,167 两边可评分) | 最大 \|Δ\| 5.1e-9 |
|
| 108 |
+
| 逐端点 S | 最大 \|Δ\| 5.3e-8,中位数 0 |
|
| 109 |
+
| 拟合池 234 段(233 可评分) | ABI' 最大 \|Δ\| 8.9e-10,S 最大 \|Δ\| 6.4e-8 |
|
| 110 |
+
| 只一边可评分 | 0 |
|
| 111 |
+
|
| 112 |
+
单元测试:126/126 通过(`aria-training/artifacts/m5-engine-regression/unit-tests.txt`),包括原生线 18 项,以及 abi_praat 空段 A 修复的 2 项回归测试。
|
| 113 |
+
|
| 114 |
+
## Limitations 与偏差清单
|
| 115 |
+
|
| 116 |
+
1. **适用范围限于 PVQD 协议复合录音经阶段 0 产出的段 A**(见上「适用范围」)。本包验证过的输入只有这一口径下的段 A;面向任意录音的元音切割前端未建、未验证,任意录音场景不在本包验证范围内。本项不改变模型数值、门结论与发布状态。
|
| 117 |
+
2. **T_diag 未标定**(见上);冷启动红线不能用于产品结论。
|
| 118 |
+
3. G3 按修订门 R1 通过;按原门 0.10 不通过。两者并排披露。
|
| 119 |
+
4. hold-out 只有 12 人,rs 的 CI 很宽;L3 与 L0 在 hold-out 上的差在噪声内。
|
| 120 |
+
5. 段 A 与原始录音之间的特征偏移(12 人上 ABI r = 0.954)。Step 0 的比较对象已裁定(2026-10-18 第 4 项):只判引擎对等,已过线;本项偏移记为已知限制,属阶段 0 范畴,不回炉。
|
| 121 |
+
6. native 引擎复刻的是 arm64 上 parselmouth 0.4.7(Praat 6.1.38,FMA 舍入)的数值;对 x86 Praat 基准未验证,预期 CPPS 有约 1e-3 dB 量级的平台差。
|
| 122 |
+
7. 段 A 是 16 kHz,经 sinc 重采样到 44.1 kHz;Hfno-6000 的 6–10 kHz 分母只有重采样残余。
|
| 123 |
+
8. 训练口径:R1/W_窗数 不排除(裁决),产品口径不变。本包同时输出两个口径的 status(`status` = 训练口径,`status_product_caliber` = 七门全过),两个口径下可评分人数差别很大(§2.5 锁定 58 人:57 vs 22)。
|
| 124 |
+
9. G6 修正器未激活(ŵ = 0)。
|
| 125 |
+
10. 阶段 0 已知偏差:WADA 模拟重建表、q_codec 停用、距离档占位、T60/元音样检测器为占位实现。
|
| 126 |
+
11. 个人化参数(σ_floor、σ_CVA、w_base)未出。
|
| 127 |
+
12. **§2.5 已运行**(2026-10-18 一次性锁定测试,结果见上「锁定 Benchmark 结果」);60 名锁定说话人未进入任何拟合、调参或参考集。锁定集上未达显著:v0.2.3.3 与 v1 冻结 severity 头的精度差异不显著。
|
| 128 |
+
13. 非医疗产品:输出为嗓音状态参考,不是诊断。
|
| 129 |
+
|
| 130 |
+
## 复现
|
| 131 |
+
|
| 132 |
+
只用本包(无训练依赖):
|
| 133 |
+
|
| 134 |
+
```bash
|
| 135 |
+
python -m pip install numpy scipy soundfile onnxruntime
|
| 136 |
+
|
| 137 |
+
# 已切好的段 A(本包验证过的输入口径)
|
| 138 |
+
python inference/predict.py --segment-a segment_a.wav --out out.json
|
| 139 |
+
|
| 140 |
+
# 原始录音(阶段 0 + 段 A + 特征 + ABI' + S;需自备 VAD ONNX)
|
| 141 |
+
python inference/predict.py --audio rec.wav --vad-model /path/SileroVAD.onnx --out out.json
|
| 142 |
+
|
| 143 |
+
# 批量:requests.jsonl 每行 {"audio_path","audio_sha256","distance_cm","speaker_id"}
|
| 144 |
+
python inference/predict.py --requests requests.jsonl --dataset /path/PVQD \
|
| 145 |
+
--vad-model /path/SileroVAD.onnx --outdir out/
|
| 146 |
+
```
|
| 147 |
+
|
| 148 |
+
训练、重拟合与锁定评估的复现命令在 `aria-training` 仓(`docs/m1/README.md`、`docs/v0233-locked-benchmark-result.md` §9);本包不含训练代码与任何测试标签。
|
| 149 |
+
|
| 150 |
+
SHA-256:`weights/model.json` `4415d3fd…56c0`,`weights/g_table.json` `4cc7d521…a385`(见 `SHA256SUMS`)。
|
weights/g_table.json
ADDED
|
@@ -0,0 +1,295 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"g_ver": 3,
|
| 3 |
+
"space_in": "ABI_prime_corr_0_10",
|
| 4 |
+
"space_out": "S_0_100",
|
| 5 |
+
"reference": "PVQD author-category Normal, M3 fit pool, clean recording, relaxed training caliber, refit L3, G6 identity",
|
| 6 |
+
"reference_speakers": [
|
| 7 |
+
"BL04",
|
| 8 |
+
"BL07",
|
| 9 |
+
"BL08",
|
| 10 |
+
"BL10",
|
| 11 |
+
"BL14",
|
| 12 |
+
"LA1001",
|
| 13 |
+
"LA1002",
|
| 14 |
+
"LA1005",
|
| 15 |
+
"LA2001",
|
| 16 |
+
"LA90026",
|
| 17 |
+
"LA9003",
|
| 18 |
+
"LA9011",
|
| 19 |
+
"LA9012",
|
| 20 |
+
"LA9014",
|
| 21 |
+
"LA9015",
|
| 22 |
+
"LA9016",
|
| 23 |
+
"LA9017",
|
| 24 |
+
"LA9018",
|
| 25 |
+
"LA9021",
|
| 26 |
+
"LA9022",
|
| 27 |
+
"LA9025",
|
| 28 |
+
"LA9026",
|
| 29 |
+
"NYU1008",
|
| 30 |
+
"NYU1009",
|
| 31 |
+
"NYU1010",
|
| 32 |
+
"NYU1013",
|
| 33 |
+
"NYU1014",
|
| 34 |
+
"NYU1020",
|
| 35 |
+
"NYU1024",
|
| 36 |
+
"NYU1027",
|
| 37 |
+
"NYU1028",
|
| 38 |
+
"PT003",
|
| 39 |
+
"PT005",
|
| 40 |
+
"PT010",
|
| 41 |
+
"PT023",
|
| 42 |
+
"PT033",
|
| 43 |
+
"PT042",
|
| 44 |
+
"PT052",
|
| 45 |
+
"PT056",
|
| 46 |
+
"PT062",
|
| 47 |
+
"PT063",
|
| 48 |
+
"PT067",
|
| 49 |
+
"PT070",
|
| 50 |
+
"PT071",
|
| 51 |
+
"PT078",
|
| 52 |
+
"PT093",
|
| 53 |
+
"PT096",
|
| 54 |
+
"PT098",
|
| 55 |
+
"PT101",
|
| 56 |
+
"PT111",
|
| 57 |
+
"PT112",
|
| 58 |
+
"PT113",
|
| 59 |
+
"PT114",
|
| 60 |
+
"PT115",
|
| 61 |
+
"PT124",
|
| 62 |
+
"PT129",
|
| 63 |
+
"PT132",
|
| 64 |
+
"PT133",
|
| 65 |
+
"SJ1003",
|
| 66 |
+
"SJ1004",
|
| 67 |
+
"SJ2001",
|
| 68 |
+
"SJ2003",
|
| 69 |
+
"SJ2005",
|
| 70 |
+
"SJ2006",
|
| 71 |
+
"SJ2012",
|
| 72 |
+
"SJ2013",
|
| 73 |
+
"SJ2014",
|
| 74 |
+
"SJ2015",
|
| 75 |
+
"SJ30004",
|
| 76 |
+
"SJ30005",
|
| 77 |
+
"SJ30008",
|
| 78 |
+
"SJ4004",
|
| 79 |
+
"SJ5003",
|
| 80 |
+
"SJ5004",
|
| 81 |
+
"SJ5007",
|
| 82 |
+
"SJ5008",
|
| 83 |
+
"SJ5009",
|
| 84 |
+
"SJ5010",
|
| 85 |
+
"SJ5015",
|
| 86 |
+
"SJ5017",
|
| 87 |
+
"SJ5018",
|
| 88 |
+
"SJ5020",
|
| 89 |
+
"SJ6008",
|
| 90 |
+
"SJ7001"
|
| 91 |
+
],
|
| 92 |
+
"knots": [
|
| 93 |
+
0.0,
|
| 94 |
+
0.08031,
|
| 95 |
+
0.1203,
|
| 96 |
+
0.1581,
|
| 97 |
+
0.2741,
|
| 98 |
+
0.3333,
|
| 99 |
+
0.3355,
|
| 100 |
+
0.3778,
|
| 101 |
+
0.3945,
|
| 102 |
+
0.4719,
|
| 103 |
+
0.6089,
|
| 104 |
+
0.6113,
|
| 105 |
+
0.6899,
|
| 106 |
+
0.7032,
|
| 107 |
+
0.7706,
|
| 108 |
+
0.7896,
|
| 109 |
+
0.7951,
|
| 110 |
+
0.8179,
|
| 111 |
+
0.8578,
|
| 112 |
+
0.8795,
|
| 113 |
+
0.9309,
|
| 114 |
+
1.035,
|
| 115 |
+
1.073,
|
| 116 |
+
1.085,
|
| 117 |
+
1.135,
|
| 118 |
+
1.139,
|
| 119 |
+
1.14,
|
| 120 |
+
1.155,
|
| 121 |
+
1.182,
|
| 122 |
+
1.217,
|
| 123 |
+
1.246,
|
| 124 |
+
1.256,
|
| 125 |
+
1.258,
|
| 126 |
+
1.268,
|
| 127 |
+
1.285,
|
| 128 |
+
1.341,
|
| 129 |
+
1.375,
|
| 130 |
+
1.413,
|
| 131 |
+
1.416,
|
| 132 |
+
1.533,
|
| 133 |
+
1.539,
|
| 134 |
+
1.588,
|
| 135 |
+
1.596,
|
| 136 |
+
1.623,
|
| 137 |
+
1.708,
|
| 138 |
+
1.716,
|
| 139 |
+
1.752,
|
| 140 |
+
1.759,
|
| 141 |
+
1.815,
|
| 142 |
+
1.844,
|
| 143 |
+
1.868,
|
| 144 |
+
1.87,
|
| 145 |
+
1.872,
|
| 146 |
+
1.886,
|
| 147 |
+
2.002,
|
| 148 |
+
2.006,
|
| 149 |
+
2.011,
|
| 150 |
+
2.025,
|
| 151 |
+
2.045,
|
| 152 |
+
2.068,
|
| 153 |
+
2.153,
|
| 154 |
+
2.156,
|
| 155 |
+
2.187,
|
| 156 |
+
2.19,
|
| 157 |
+
2.272,
|
| 158 |
+
2.324,
|
| 159 |
+
2.428,
|
| 160 |
+
2.448,
|
| 161 |
+
2.541,
|
| 162 |
+
2.698,
|
| 163 |
+
2.734,
|
| 164 |
+
2.764,
|
| 165 |
+
2.794,
|
| 166 |
+
2.85,
|
| 167 |
+
2.853,
|
| 168 |
+
2.879,
|
| 169 |
+
2.912,
|
| 170 |
+
2.98,
|
| 171 |
+
3.038,
|
| 172 |
+
3.06,
|
| 173 |
+
3.119,
|
| 174 |
+
3.228,
|
| 175 |
+
3.415,
|
| 176 |
+
3.693
|
| 177 |
+
],
|
| 178 |
+
"p": [
|
| 179 |
+
0.005952380952380952,
|
| 180 |
+
0.017857142857142856,
|
| 181 |
+
0.02976190476190476,
|
| 182 |
+
0.041666666666666664,
|
| 183 |
+
0.05357142857142857,
|
| 184 |
+
0.06547619047619048,
|
| 185 |
+
0.07738095238095238,
|
| 186 |
+
0.08928571428571429,
|
| 187 |
+
0.10119047619047619,
|
| 188 |
+
0.1130952380952381,
|
| 189 |
+
0.125,
|
| 190 |
+
0.13690476190476192,
|
| 191 |
+
0.1488095238095238,
|
| 192 |
+
0.16071428571428573,
|
| 193 |
+
0.17261904761904762,
|
| 194 |
+
0.18452380952380953,
|
| 195 |
+
0.19642857142857142,
|
| 196 |
+
0.20833333333333334,
|
| 197 |
+
0.22023809523809523,
|
| 198 |
+
0.23214285714285715,
|
| 199 |
+
0.24404761904761904,
|
| 200 |
+
0.25595238095238093,
|
| 201 |
+
0.26785714285714285,
|
| 202 |
+
0.27976190476190477,
|
| 203 |
+
0.2916666666666667,
|
| 204 |
+
0.30357142857142855,
|
| 205 |
+
0.31547619047619047,
|
| 206 |
+
0.3273809523809524,
|
| 207 |
+
0.3392857142857143,
|
| 208 |
+
0.35119047619047616,
|
| 209 |
+
0.3630952380952381,
|
| 210 |
+
0.375,
|
| 211 |
+
0.3869047619047619,
|
| 212 |
+
0.39880952380952384,
|
| 213 |
+
0.4107142857142857,
|
| 214 |
+
0.4226190476190476,
|
| 215 |
+
0.43452380952380953,
|
| 216 |
+
0.44642857142857145,
|
| 217 |
+
0.4583333333333333,
|
| 218 |
+
0.47023809523809523,
|
| 219 |
+
0.48214285714285715,
|
| 220 |
+
0.49404761904761907,
|
| 221 |
+
0.5059523809523809,
|
| 222 |
+
0.5178571428571429,
|
| 223 |
+
0.5297619047619048,
|
| 224 |
+
0.5416666666666666,
|
| 225 |
+
0.5535714285714286,
|
| 226 |
+
0.5654761904761905,
|
| 227 |
+
0.5773809523809523,
|
| 228 |
+
0.5892857142857143,
|
| 229 |
+
0.6011904761904762,
|
| 230 |
+
0.6130952380952381,
|
| 231 |
+
0.625,
|
| 232 |
+
0.6369047619047619,
|
| 233 |
+
0.6488095238095238,
|
| 234 |
+
0.6607142857142857,
|
| 235 |
+
0.6726190476190477,
|
| 236 |
+
0.6845238095238095,
|
| 237 |
+
0.6964285714285714,
|
| 238 |
+
0.7083333333333334,
|
| 239 |
+
0.7202380952380952,
|
| 240 |
+
0.7321428571428571,
|
| 241 |
+
0.7440476190476191,
|
| 242 |
+
0.7559523809523809,
|
| 243 |
+
0.7678571428571429,
|
| 244 |
+
0.7797619047619048,
|
| 245 |
+
0.7916666666666666,
|
| 246 |
+
0.8035714285714286,
|
| 247 |
+
0.8154761904761905,
|
| 248 |
+
0.8273809523809523,
|
| 249 |
+
0.8392857142857143,
|
| 250 |
+
0.8511904761904762,
|
| 251 |
+
0.8630952380952381,
|
| 252 |
+
0.875,
|
| 253 |
+
0.8869047619047619,
|
| 254 |
+
0.8988095238095238,
|
| 255 |
+
0.9107142857142857,
|
| 256 |
+
0.9226190476190477,
|
| 257 |
+
0.9345238095238095,
|
| 258 |
+
0.9464285714285714,
|
| 259 |
+
0.9583333333333334,
|
| 260 |
+
0.9702380952380952,
|
| 261 |
+
0.9821428571428571,
|
| 262 |
+
0.9940476190476191
|
| 263 |
+
],
|
| 264 |
+
"n": 84,
|
| 265 |
+
"n_knots": 84,
|
| 266 |
+
"collapsed_ties": 0,
|
| 267 |
+
"sigma": 0.884,
|
| 268 |
+
"median": 1.592,
|
| 269 |
+
"mad": 0.5962,
|
| 270 |
+
"acceptance": {
|
| 271 |
+
"c0_left_gap": 1.90986271242366e-09,
|
| 272 |
+
"c0_right_gap": 1.909853608594858e-09,
|
| 273 |
+
"c0_left_analytic_gap": 7.771561172376096e-16,
|
| 274 |
+
"c0_right_analytic_gap": 0.0,
|
| 275 |
+
"c0_continuous": true,
|
| 276 |
+
"strictly_increasing_analytic": true,
|
| 277 |
+
"strictly_increasing_on_grid": true,
|
| 278 |
+
"grid_points_resolvable": 64307,
|
| 279 |
+
"nondecreasing_everywhere_on_grid": true,
|
| 280 |
+
"grid_points": 100001,
|
| 281 |
+
"grid_range": [
|
| 282 |
+
0.0,
|
| 283 |
+
10.0
|
| 284 |
+
],
|
| 285 |
+
"interior_segment_slopes_min": 4.28228845495035,
|
| 286 |
+
"sigma_positive": true,
|
| 287 |
+
"pass": true,
|
| 288 |
+
"device_lookup_max_abs_diff": 1.4210854715202004e-14
|
| 289 |
+
},
|
| 290 |
+
"fallback_g0": {
|
| 291 |
+
"center": 2.32,
|
| 292 |
+
"scale": 1.1
|
| 293 |
+
},
|
| 294 |
+
"formula": "Hazen p_i=(i-0.5)/n; interior linear; tails probit with sigma=1.4826*MAD; 4 significant digits"
|
| 295 |
+
}
|
weights/model.json
ADDED
|
@@ -0,0 +1,149 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_ver": "v0.2.3.3",
|
| 3 |
+
"refit_level": "L3",
|
| 4 |
+
"g_ver": 3,
|
| 5 |
+
"feature_engine": "native",
|
| 6 |
+
"feature_engine_source": "training/native_features/abi_native.py (numpy/scipy port of abi_praat.py PART 2; no Praat dependency)",
|
| 7 |
+
"label": "cape_v_severity",
|
| 8 |
+
"seg_ver": "stage0-v0.2.1.2",
|
| 9 |
+
"feature_names": [
|
| 10 |
+
"cpps",
|
| 11 |
+
"jitter_local",
|
| 12 |
+
"gne_max",
|
| 13 |
+
"hfno6000",
|
| 14 |
+
"hnrd",
|
| 15 |
+
"h1h2",
|
| 16 |
+
"shimmer_local_db",
|
| 17 |
+
"shimmer_local",
|
| 18 |
+
"psd"
|
| 19 |
+
],
|
| 20 |
+
"abi_prime_raw": {
|
| 21 |
+
"level": "L3",
|
| 22 |
+
"coefficients": {
|
| 23 |
+
"b0": 12.47,
|
| 24 |
+
"b": [
|
| 25 |
+
-0.5062,
|
| 26 |
+
-0.3403,
|
| 27 |
+
-5.915,
|
| 28 |
+
-1.061,
|
| 29 |
+
0.06068,
|
| 30 |
+
-0.004161,
|
| 31 |
+
4.759,
|
| 32 |
+
-0.1942,
|
| 33 |
+
10.64
|
| 34 |
+
],
|
| 35 |
+
"lambda": 100.0
|
| 36 |
+
},
|
| 37 |
+
"formula": "b0 + sum_j b_j x_j",
|
| 38 |
+
"abi_orig_coefficients": {
|
| 39 |
+
"intercept": 5.0447730915,
|
| 40 |
+
"scale": 2.9257400394,
|
| 41 |
+
"cpps": -0.172,
|
| 42 |
+
"jitter_local": -0.193,
|
| 43 |
+
"gne_max": -1.283,
|
| 44 |
+
"hfno6000": -0.396,
|
| 45 |
+
"hnrd": 0.01,
|
| 46 |
+
"h1h2": 0.017,
|
| 47 |
+
"shimmer_local_db": 1.473,
|
| 48 |
+
"shimmer_local": -0.088,
|
| 49 |
+
"psd": -68.295
|
| 50 |
+
},
|
| 51 |
+
"clip": [
|
| 52 |
+
0.0,
|
| 53 |
+
10.0
|
| 54 |
+
]
|
| 55 |
+
},
|
| 56 |
+
"g6_correction": {
|
| 57 |
+
"active": false,
|
| 58 |
+
"weights": [
|
| 59 |
+
0.0,
|
| 60 |
+
0.0,
|
| 61 |
+
0.0,
|
| 62 |
+
0.0,
|
| 63 |
+
0.0,
|
| 64 |
+
0.0,
|
| 65 |
+
0.0,
|
| 66 |
+
0.0,
|
| 67 |
+
0.0,
|
| 68 |
+
0.0
|
| 69 |
+
],
|
| 70 |
+
"reason": "G6 finalized under product strict caliber: real_g6_status=FAIL (253 pairs: 209 train / 44 eval); corrector held bypass, identity fallback"
|
| 71 |
+
},
|
| 72 |
+
"g_table": {
|
| 73 |
+
"path": "g_table.json",
|
| 74 |
+
"sha256": "4cc7d521f6989c87b2355f49381081541860c6f60aa9e170f2393d727c8da385"
|
| 75 |
+
},
|
| 76 |
+
"cold_start_thresholds": {
|
| 77 |
+
"T_diag_L0": 3.44,
|
| 78 |
+
"S_diag": 99.87,
|
| 79 |
+
"T_red_cold": 90.0,
|
| 80 |
+
"T_yellow_cold": 75.0,
|
| 81 |
+
"T_yellow_effective_n_lt_5": 75.0,
|
| 82 |
+
"inversion_lock_active": false,
|
| 83 |
+
"reference_share_above_T_diag": 0.0
|
| 84 |
+
},
|
| 85 |
+
"dev_metrics": {
|
| 86 |
+
"target": "PVQD cape_v_severity average / 10 (ABI 0-10 scale)",
|
| 87 |
+
"caliber": "relaxed",
|
| 88 |
+
"holdout_n": 12,
|
| 89 |
+
"holdout_rs": 0.8881118881118882,
|
| 90 |
+
"holdout_mae_abi": 1.254975337236816,
|
| 91 |
+
"holdout_rs_L0": 0.8951048951048951,
|
| 92 |
+
"cv_mean_rs": 0.64148287217087,
|
| 93 |
+
"cv_rs_range": 0.20672451853288648,
|
| 94 |
+
"g3_refit_gate": {
|
| 95 |
+
"rule": "G3 revision R1 (Ryo 2026-10-18): level chosen by CV; hold-out rs >= 0.70; CV rs range <= tau, tau = P95 of the 5-fold range under a rho0 = 0.70 Fisher-z null at the per-fold speaker count",
|
| 96 |
+
"holdout_rs": 0.8881118881118882,
|
| 97 |
+
"holdout_rs_ge_0_70": true,
|
| 98 |
+
"cv_rs_range": 0.20672451853288648,
|
| 99 |
+
"n_per_fold": 42.4,
|
| 100 |
+
"tau": 0.3692486604860881,
|
| 101 |
+
"tau_params": {
|
| 102 |
+
"rho0": 0.7,
|
| 103 |
+
"quantile": 0.95,
|
| 104 |
+
"draws": 1000000,
|
| 105 |
+
"seed": 20261018
|
| 106 |
+
},
|
| 107 |
+
"cv_range_le_tau": true,
|
| 108 |
+
"holdout_vs_L0": {
|
| 109 |
+
"difference": -0.006993006993006867,
|
| 110 |
+
"bootstrap_ci95": [
|
| 111 |
+
-0.2643073593073595,
|
| 112 |
+
0.21900010353574575
|
| 113 |
+
],
|
| 114 |
+
"bootstrap_valid_draws": 4000,
|
| 115 |
+
"status": "disclosed only; not a gate under ruling (iii)"
|
| 116 |
+
},
|
| 117 |
+
"pass": true
|
| 118 |
+
},
|
| 119 |
+
"g3_refit_gate_original": {
|
| 120 |
+
"rule": "v0.2.1 original: hold-out rs >= 0.70, hold-out rs >= L0 (strict), CV rs range <= 0.10",
|
| 121 |
+
"holdout_rs": 0.8881118881118882,
|
| 122 |
+
"holdout_rs_ge_0_70": true,
|
| 123 |
+
"not_worse_than_L0": false,
|
| 124 |
+
"cv_rs_range": 0.20672451853288648,
|
| 125 |
+
"cv_range_le_0_10": false,
|
| 126 |
+
"pass": false
|
| 127 |
+
}
|
| 128 |
+
},
|
| 129 |
+
"inputs_sha256": {
|
| 130 |
+
"m3_refit": "8bf40e7004188584c845284ca0f72ecb162de63b1131ae230a182505bcba5186",
|
| 131 |
+
"pool_requests": "d2edadb7308c324d60cb9b92d0e1b397ce69994f113a84832b04fda7385e8a20",
|
| 132 |
+
"pool_reports": "a20e9629e0944c7065b4c346d1d08a5bfc98e238c0be16af68a92b18b33ca30a",
|
| 133 |
+
"pool_abi": "6ae7987b79bae861272375d3cd6b782ed232ed87d5d38f4e3e5e0f8942e77b23"
|
| 134 |
+
},
|
| 135 |
+
"release_status": "formal cold-start release; G3 PASS under revision R1; T_diag uncalibrated (pending ruling)",
|
| 136 |
+
"label_change": {
|
| 137 |
+
"from": "cape_v_breathiness (v0.2.0.1)",
|
| 138 |
+
"to": "cape_v_severity",
|
| 139 |
+
"ruling": "Ryo 2026-10-18 item 1"
|
| 140 |
+
},
|
| 141 |
+
"engine_change": {
|
| 142 |
+
"from": "praat-script (v0.2.3.2)",
|
| 143 |
+
"to": "native",
|
| 144 |
+
"basis": "training/native_features/REPORT.md: 9/9 features r >= 0.95 (|r-1| < 1e-12), ABI L0 max |delta| 5.2e-9 over 2,352 endpoints"
|
| 145 |
+
},
|
| 146 |
+
"g3_revision": "R1 (Ryo 2026-10-18): proposal A noise-calibrated tau; ruling (iii) on not-worse-than-L0",
|
| 147 |
+
"t_diag_status": "PLACEHOLDER 3.44 from spec; S_diag and cold red line UNCALIBRATED, pending Ryo ruling",
|
| 148 |
+
"supersedes": "v0.2.3.2 (draft, archived); v0.2.0.1 (archived)"
|
| 149 |
+
}
|