Instructions to use kugu/AIAMD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Keras
How to use kugu/AIAMD with Keras:
# !pip install -U keras tensorflow huggingface_hub # Keras needs TensorFlow installed to read "hf://" paths, so the tensorflow backend is selected here; # "jax" and "torch" also work for computation once TensorFlow is installed. import os os.environ["KERAS_BACKEND"] = "tensorflow" import keras model = keras.saving.load_model("hf://kugu/AIAMD") - Notebooks
- Google Colab
- Kaggle
AIAMD GRU (epoch 4), calibration, inference script and model card
Browse files- .gitattributes +1 -0
- README.md +112 -0
- amd_gru.keras +3 -0
- amd_gru_calibration.json +24 -0
- amd_gru_history.csv +15 -0
- config.yaml +22 -0
- evaluation.json +271 -0
- inference.py +80 -0
- training_summary.json +34 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
amd_gru.keras filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,112 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: keras
|
| 3 |
+
pipeline_tag: audio-classification
|
| 4 |
+
tags:
|
| 5 |
+
- answering-machine-detection
|
| 6 |
+
- telephony
|
| 7 |
+
- voicemail-detection
|
| 8 |
+
- yamnet
|
| 9 |
+
- gru
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# AIAMD: answering machine detection (YAMNet + GRU)
|
| 13 |
+
|
| 14 |
+
Decides whether an outbound call was answered by a **human** or a **machine** (voicemail, call screener,
|
| 15 |
+
IVR, network tones), from the first ≤ 3 s of the callee's audio.
|
| 16 |
+
|
| 17 |
+
An implementation of *A Recurrent Neural Network Approach to the Answering Machine Detection Problem*
|
| 18 |
+
(Altwlkany et al., 2024, [arXiv:2410.08235](https://arxiv.org/abs/2410.08235)): a frozen
|
| 19 |
+
[YAMNet](https://tfhub.dev/google/yamnet/1) turns every 0.96 s of audio (every 0.48 s) into 1,024 numbers,
|
| 20 |
+
and a small GRU gives `p(machine)` after each window, so it can decide while the call is still going.
|
| 21 |
+
|
| 22 |
+
```
|
| 23 |
+
8 kHz audio -> fill digital silence + 3.6 kHz low-pass -> 16 kHz -> YAMNet (frozen)
|
| 24 |
+
-> Dense(32, tanh) -> GRU(48) -> Dense(1, sigmoid) = p(machine) after every 0.48 s
|
| 25 |
+
```
|
| 26 |
+
|
| 27 |
+
44,657 trainable parameters.
|
| 28 |
+
|
| 29 |
+
## Files
|
| 30 |
+
|
| 31 |
+
| File | What |
|
| 32 |
+
|---|---|
|
| 33 |
+
| `amd_gru.keras` | the GRU (input: YAMNet embeddings, shape `(windows, 1024)`) |
|
| 34 |
+
| `amd_gru_calibration.json` | temperature (1.156) and decision threshold (0.452), fitted on validation data |
|
| 35 |
+
| `inference.py` | self-contained preprocessing + YAMNet + GRU + decision |
|
| 36 |
+
| `config.yaml` | preprocessing, model and decision settings |
|
| 37 |
+
| `training_summary.json`, `amd_gru_history.csv` | training run (best epoch 4 by validation loss) |
|
| 38 |
+
| `evaluation.json` | full test / holdout results |
|
| 39 |
+
|
| 40 |
+
## Usage
|
| 41 |
+
|
| 42 |
+
```bash
|
| 43 |
+
pip install tensorflow tensorflow-hub "setuptools<81" soundfile scipy numpy huggingface_hub
|
| 44 |
+
hf download kugu/AIAMD --local-dir AIAMD
|
| 45 |
+
python AIAMD/inference.py call.wav
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
```python
|
| 49 |
+
from inference import AmdDetector
|
| 50 |
+
detector = AmdDetector() # folder with the downloaded files
|
| 51 |
+
result = detector.predict(samples, sample_rate) # mono float audio, any sample rate
|
| 52 |
+
# {"decision": "machine", "p_machine": 0.99, "threshold": 0.452, "p_machine_per_window": [...], ...}
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
**Preprocessing must match training**: runs of exact-zero samples ≥ 10 ms are replaced by -70 dBFS noise,
|
| 56 |
+
then a causal 8th-order Butterworth low-pass at 3.6 kHz, then resampling to 16 kHz, then the first 3 s.
|
| 57 |
+
`inference.py` does all of this.
|
| 58 |
+
|
| 59 |
+
## Training data (not released)
|
| 60 |
+
|
| 61 |
+
| | Clips | Description |
|
| 62 |
+
|---|---|---|
|
| 63 |
+
| Human | 2,175 | the callee's first response on outbound calls, ≤ 3 s, 8 kHz |
|
| 64 |
+
| Machine | 3,000 for training (+ 7,000 holdout) | vendor-labelled recordings: voicemail, call screeners, IVR, network tones, beeps |
|
| 65 |
+
|
| 66 |
+
- Splits 80 / 10 / 10 by call, so no call appears in two splits.
|
| 67 |
+
- Both classes get the same random leading (0–1 s) and trailing (0–0.5 s) silence, made from the clip's own
|
| 68 |
+
background noise, and machine clips are cut to lengths drawn from the human clips. Clip length and
|
| 69 |
+
silence therefore don't reveal the class.
|
| 70 |
+
- Windows quieter than -50 dBFS get zero weight in training.
|
| 71 |
+
|
| 72 |
+
## Results
|
| 73 |
+
|
| 74 |
+
Test: 212 human + 300 machine clips. Holdout: 7,000 machine clips. Decision after the whole clip.
|
| 75 |
+
|
| 76 |
+
| Threshold | Human recall | Machine recall (test) | Machine recall (holdout) | Balanced acc. | AUC |
|
| 77 |
+
|---|---|---|---|---|---|
|
| 78 |
+
| 0.5 | 0.972 | 0.950 | 0.962 | 0.961 | 0.993 |
|
| 79 |
+
| 0.452 (calibrated, target 99 % human recall on val) | 0.962 | 0.963 | 0.965 | 0.963 (95 % CI 0.947–0.979) | 0.993 |
|
| 80 |
+
|
| 81 |
+
Decision after k windows (test, threshold 0.452):
|
| 82 |
+
|
| 83 |
+
| Audio heard | 0.98 s | 1.46 s | 1.94 s | 2.42 s | 2.90 s |
|
| 84 |
+
|---|---|---|---|---|---|
|
| 85 |
+
| Balanced accuracy | 0.868 | 0.927 | 0.940 | 0.958 | 0.965 |
|
| 86 |
+
|
| 87 |
+
Machine categories (test + holdout): voicemail 0.974, network / tone 0.955, call screening 0.949, IVR 0.948.
|
| 88 |
+
Hardest: custom voicemail, clicks, name-only greetings (~0.86–0.89).
|
| 89 |
+
|
| 90 |
+
## Limitations
|
| 91 |
+
|
| 92 |
+
- **Recording channel.** All human audio comes from one dialer and all machine audio from one vendor, so part
|
| 93 |
+
of what the model learned may be the recording channel. Scores on audio from other systems may be lower.
|
| 94 |
+
- **Strict human-safety targets.** The validation set has 220 humans, too few to set a 99 % human-recall
|
| 95 |
+
threshold reliably: on test the calibrated threshold gives 96.2 %. A few human clips get confident
|
| 96 |
+
`machine` scores, so at 99 % human recall only about 73 % of machines are caught. In the same setting a
|
| 97 |
+
gradient-boosted model on hand-crafted timing / loudness / MFCC features caught 94 %. Choose the threshold on
|
| 98 |
+
your own data.
|
| 99 |
+
- **At most 3 s** of audio (training clips were ≤ 3 s); the paper uses 4 s.
|
| 100 |
+
- Labels were screened automatically, not reviewed by ear.
|
| 101 |
+
- 8 kHz telephony audio only.
|
| 102 |
+
|
| 103 |
+
## Citation
|
| 104 |
+
|
| 105 |
+
```bibtex
|
| 106 |
+
@article{altwlkany2024amd,
|
| 107 |
+
title = {A Recurrent Neural Network Approach to the Answering Machine Detection Problem},
|
| 108 |
+
author = {Altwlkany, Kemal and Delali{\'c}, Sead and Selmanovi{\'c}, Elmedin and Alihod{\v{z}}i{\'c}, Adis and Lovri{\'c}, Ivica},
|
| 109 |
+
journal = {arXiv preprint arXiv:2410.08235},
|
| 110 |
+
year = {2024}
|
| 111 |
+
}
|
| 112 |
+
```
|
amd_gru.keras
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:37284d3df90a251980ca3d4554529b47b3654f13a519515563131109df25f7d6
|
| 3 |
+
size 570776
|
amd_gru_calibration.json
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created": "2026-11-16T09:57:45+00:00",
|
| 3 |
+
"model_path": "models/amd_gru.keras",
|
| 4 |
+
"fitted_on": "val",
|
| 5 |
+
"temperature": 1.1556061176661283,
|
| 6 |
+
"target_human_recall": 0.99,
|
| 7 |
+
"threshold": 0.4520810842514038,
|
| 8 |
+
"val": {
|
| 9 |
+
"window_calibration_error_before": 0.013274187854735818,
|
| 10 |
+
"window_calibration_error_after": 0.016876171816844275,
|
| 11 |
+
"at_0.5": {
|
| 12 |
+
"human_recall": 0.990909090909091,
|
| 13 |
+
"machine_recall": 0.9633333333333334,
|
| 14 |
+
"balanced_acc": 0.9771212121212122,
|
| 15 |
+
"auc": 0.9963939393939394
|
| 16 |
+
},
|
| 17 |
+
"at_threshold": {
|
| 18 |
+
"human_recall": 0.990909090909091,
|
| 19 |
+
"machine_recall": 0.9633333333333334,
|
| 20 |
+
"balanced_acc": 0.9771212121212122,
|
| 21 |
+
"auc": 0.9963939393939394
|
| 22 |
+
}
|
| 23 |
+
}
|
| 24 |
+
}
|
amd_gru_history.csv
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
epoch,loss,val_loss,val_window_acc,window_acc,val_clip_balanced_acc,val_clip_auc
|
| 2 |
+
1,0.14931009709835052,0.09590049088001251,0.9524377584457397,0.923165500164032,0.9736363636363636,0.9961363636363636
|
| 3 |
+
2,0.09785998612642288,0.08416719734668732,0.9622458815574646,0.9499034285545349,0.9715151515151514,0.9966666666666666
|
| 4 |
+
3,0.08525345474481583,0.08037174493074417,0.9662546515464783,0.9559043645858765,0.9760606060606061,0.9964696969696969
|
| 5 |
+
4,0.07630451768636703,0.07769685238599777,0.9693971276283264,0.9608813524246216,0.9771212121212122,0.9963939393939394
|
| 6 |
+
5,0.06992293149232864,0.09264785051345825,0.9621654748916626,0.9647255539894104,0.980909090909091,0.9953939393939393
|
| 7 |
+
6,0.06482290476560593,0.09431156516075134,0.9609526991844177,0.9675842523574829,0.978030303030303,0.995469696969697
|
| 8 |
+
7,0.060456931591033936,0.08757705241441727,0.9651347398757935,0.969741940498352,0.976969696969697,0.9959848484848485
|
| 9 |
+
8,0.05552744120359421,0.0912625715136528,0.9624777436256409,0.972305178642273,0.978030303030303,0.9956969696969697
|
| 10 |
+
9,0.05275537446141243,0.09123143553733826,0.9604672789573669,0.9732204079627991,0.976969696969697,0.9948333333333335
|
| 11 |
+
10,0.047697342932224274,0.08868642151355743,0.9619459509849548,0.9763790369033813,0.9815151515151515,0.9952727272727272
|
| 12 |
+
11,0.0441267304122448,0.092005655169487,0.963944137096405,0.977412223815918,0.9698484848484848,0.9951060606060605
|
| 13 |
+
12,0.039735473692417145,0.09554276615381241,0.9610334634780884,0.9804154634475708,0.9713636363636364,0.9951212121212121
|
| 14 |
+
13,0.03597458079457283,0.09973978251218796,0.9606747627258301,0.9816167950630188,0.9765151515151516,0.9948787878787878
|
| 15 |
+
14,0.03649942949414253,0.11146742850542068,0.9604430794715881,0.9811847805976868,0.9715151515151514,0.9940151515151515
|
config.yaml
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Settings needed for inference (from the training repo's configs/config.yaml)
|
| 2 |
+
audio:
|
| 3 |
+
sample_rate: 16000
|
| 4 |
+
clip_seconds: 3.0
|
| 5 |
+
silence_dbfs: -50.0
|
| 6 |
+
zero_fill_dbfs: -70.0
|
| 7 |
+
zero_fill_min_ms: 10
|
| 8 |
+
lowpass_hz: 3600
|
| 9 |
+
lowpass_order: 8
|
| 10 |
+
features:
|
| 11 |
+
yamnet_handle: https://tfhub.dev/google/yamnet/1
|
| 12 |
+
cache_dir: models/tfhub_modules
|
| 13 |
+
model:
|
| 14 |
+
dense_units: 32
|
| 15 |
+
gru_units: 48
|
| 16 |
+
decision:
|
| 17 |
+
target_human_recall: 0.99
|
| 18 |
+
calibration_path: amd_gru_calibration.json
|
| 19 |
+
stream:
|
| 20 |
+
timeout: 3.0
|
| 21 |
+
confidence: 0.9
|
| 22 |
+
min_detection_time: 1.44
|
evaluation.json
ADDED
|
@@ -0,0 +1,271 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model": {
|
| 3 |
+
"created": "2026-11-16T09:43:43+00:00",
|
| 4 |
+
"model_path": "models/amd_gru.keras",
|
| 5 |
+
"monitor": "val_loss",
|
| 6 |
+
"best_epoch": 4,
|
| 7 |
+
"epochs_run": 14,
|
| 8 |
+
"best_val": {
|
| 9 |
+
"loss": 0.0763,
|
| 10 |
+
"val_loss": 0.0777,
|
| 11 |
+
"val_window_acc": 0.9694,
|
| 12 |
+
"window_acc": 0.9609,
|
| 13 |
+
"val_clip_balanced_acc": 0.9771,
|
| 14 |
+
"val_clip_auc": 0.9964
|
| 15 |
+
},
|
| 16 |
+
"class_weights": {
|
| 17 |
+
"0": 1.2205114570240743,
|
| 18 |
+
"1": 0.846975745580194
|
| 19 |
+
},
|
| 20 |
+
"silence_dbfs": -50.0,
|
| 21 |
+
"model": {
|
| 22 |
+
"dense_units": 32,
|
| 23 |
+
"gru_units": 48
|
| 24 |
+
},
|
| 25 |
+
"train": {
|
| 26 |
+
"epochs": 50,
|
| 27 |
+
"batch_size": 32,
|
| 28 |
+
"learning_rate": 0.001,
|
| 29 |
+
"model_path": "models/amd_gru.keras",
|
| 30 |
+
"monitor": "val_loss",
|
| 31 |
+
"patience": 10,
|
| 32 |
+
"threshold": 0.5
|
| 33 |
+
},
|
| 34 |
+
"params": 44657
|
| 35 |
+
},
|
| 36 |
+
"calibration": {
|
| 37 |
+
"created": "2026-11-16T09:57:45+00:00",
|
| 38 |
+
"model_path": "models/amd_gru.keras",
|
| 39 |
+
"fitted_on": "val",
|
| 40 |
+
"temperature": 1.1556061176661283,
|
| 41 |
+
"target_human_recall": 0.99,
|
| 42 |
+
"threshold": 0.4520810842514038,
|
| 43 |
+
"val": {
|
| 44 |
+
"window_calibration_error_before": 0.013274187854735818,
|
| 45 |
+
"window_calibration_error_after": 0.016876171816844275,
|
| 46 |
+
"at_0.5": {
|
| 47 |
+
"human_recall": 0.990909090909091,
|
| 48 |
+
"machine_recall": 0.9633333333333334,
|
| 49 |
+
"balanced_acc": 0.9771212121212122,
|
| 50 |
+
"auc": 0.9963939393939394
|
| 51 |
+
},
|
| 52 |
+
"at_threshold": {
|
| 53 |
+
"human_recall": 0.990909090909091,
|
| 54 |
+
"machine_recall": 0.9633333333333334,
|
| 55 |
+
"balanced_acc": 0.9771212121212122,
|
| 56 |
+
"auc": 0.9963939393939394
|
| 57 |
+
}
|
| 58 |
+
}
|
| 59 |
+
},
|
| 60 |
+
"threshold": 0.4520810842514038,
|
| 61 |
+
"splits": {
|
| 62 |
+
"test": {
|
| 63 |
+
"clips": 512,
|
| 64 |
+
"gru": {
|
| 65 |
+
"human_recall": 0.9622641509433962,
|
| 66 |
+
"machine_recall": 0.9633333333333334,
|
| 67 |
+
"balanced_acc": 0.9627987421383648,
|
| 68 |
+
"auc": 0.9931761006289308,
|
| 69 |
+
"balanced_acc_95ci": [
|
| 70 |
+
0.9465258535699461,
|
| 71 |
+
0.9794183557051795
|
| 72 |
+
]
|
| 73 |
+
},
|
| 74 |
+
"gru_at_0.5": {
|
| 75 |
+
"human_recall": 0.9716981132075472,
|
| 76 |
+
"machine_recall": 0.95,
|
| 77 |
+
"balanced_acc": 0.9608490566037735,
|
| 78 |
+
"auc": 0.9931761006289308
|
| 79 |
+
},
|
| 80 |
+
"over_time": [
|
| 81 |
+
{
|
| 82 |
+
"windows": 1,
|
| 83 |
+
"seconds": 0.97,
|
| 84 |
+
"human_recall": 0.8821,
|
| 85 |
+
"machine_recall": 0.8533,
|
| 86 |
+
"balanced_acc": 0.8677,
|
| 87 |
+
"auc": 0.9483
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"windows": 2,
|
| 91 |
+
"seconds": 1.46,
|
| 92 |
+
"human_recall": 0.9434,
|
| 93 |
+
"machine_recall": 0.91,
|
| 94 |
+
"balanced_acc": 0.9267,
|
| 95 |
+
"auc": 0.9803
|
| 96 |
+
},
|
| 97 |
+
{
|
| 98 |
+
"windows": 3,
|
| 99 |
+
"seconds": 1.94,
|
| 100 |
+
"human_recall": 0.9434,
|
| 101 |
+
"machine_recall": 0.9367,
|
| 102 |
+
"balanced_acc": 0.94,
|
| 103 |
+
"auc": 0.9885
|
| 104 |
+
},
|
| 105 |
+
{
|
| 106 |
+
"windows": 4,
|
| 107 |
+
"seconds": 2.42,
|
| 108 |
+
"human_recall": 0.967,
|
| 109 |
+
"machine_recall": 0.95,
|
| 110 |
+
"balanced_acc": 0.9585,
|
| 111 |
+
"auc": 0.9926
|
| 112 |
+
},
|
| 113 |
+
{
|
| 114 |
+
"windows": 5,
|
| 115 |
+
"seconds": 2.9,
|
| 116 |
+
"human_recall": 0.967,
|
| 117 |
+
"machine_recall": 0.9633,
|
| 118 |
+
"balanced_acc": 0.9652,
|
| 119 |
+
"auc": 0.9942
|
| 120 |
+
},
|
| 121 |
+
{
|
| 122 |
+
"windows": 6,
|
| 123 |
+
"seconds": 3.38,
|
| 124 |
+
"human_recall": 0.9623,
|
| 125 |
+
"machine_recall": 0.9633,
|
| 126 |
+
"balanced_acc": 0.9628,
|
| 127 |
+
"auc": 0.9932
|
| 128 |
+
}
|
| 129 |
+
]
|
| 130 |
+
},
|
| 131 |
+
"holdout": {
|
| 132 |
+
"clips": 7000,
|
| 133 |
+
"gru": {
|
| 134 |
+
"machine_recall": 0.9654285714285714
|
| 135 |
+
},
|
| 136 |
+
"gru_at_0.5": {
|
| 137 |
+
"machine_recall": 0.962
|
| 138 |
+
},
|
| 139 |
+
"over_time": [
|
| 140 |
+
{
|
| 141 |
+
"windows": 1,
|
| 142 |
+
"seconds": 0.97,
|
| 143 |
+
"machine_recall": 0.8837
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"windows": 2,
|
| 147 |
+
"seconds": 1.46,
|
| 148 |
+
"machine_recall": 0.9166
|
| 149 |
+
},
|
| 150 |
+
{
|
| 151 |
+
"windows": 3,
|
| 152 |
+
"seconds": 1.94,
|
| 153 |
+
"machine_recall": 0.946
|
| 154 |
+
},
|
| 155 |
+
{
|
| 156 |
+
"windows": 4,
|
| 157 |
+
"seconds": 2.42,
|
| 158 |
+
"machine_recall": 0.959
|
| 159 |
+
},
|
| 160 |
+
{
|
| 161 |
+
"windows": 5,
|
| 162 |
+
"seconds": 2.9,
|
| 163 |
+
"machine_recall": 0.9659
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"windows": 6,
|
| 167 |
+
"seconds": 3.38,
|
| 168 |
+
"machine_recall": 0.9654
|
| 169 |
+
}
|
| 170 |
+
]
|
| 171 |
+
}
|
| 172 |
+
},
|
| 173 |
+
"by_category": {
|
| 174 |
+
"other": {
|
| 175 |
+
"accuracy": 0.9179,
|
| 176 |
+
"clips": 134
|
| 177 |
+
},
|
| 178 |
+
"ivr": {
|
| 179 |
+
"accuracy": 0.9481,
|
| 180 |
+
"clips": 135
|
| 181 |
+
},
|
| 182 |
+
"call_screening": {
|
| 183 |
+
"accuracy": 0.9493,
|
| 184 |
+
"clips": 1242
|
| 185 |
+
},
|
| 186 |
+
"network_or_tone": {
|
| 187 |
+
"accuracy": 0.9552,
|
| 188 |
+
"clips": 1117
|
| 189 |
+
},
|
| 190 |
+
"human": {
|
| 191 |
+
"accuracy": 0.9623,
|
| 192 |
+
"clips": 212
|
| 193 |
+
},
|
| 194 |
+
"voicemail": {
|
| 195 |
+
"accuracy": 0.9739,
|
| 196 |
+
"clips": 4669
|
| 197 |
+
},
|
| 198 |
+
"silence": {
|
| 199 |
+
"accuracy": 1.0,
|
| 200 |
+
"clips": 3
|
| 201 |
+
}
|
| 202 |
+
},
|
| 203 |
+
"worst_machine_classes": {
|
| 204 |
+
"CUSTOMVMAMD": {
|
| 205 |
+
"accuracy": 0.8554,
|
| 206 |
+
"clips": 242
|
| 207 |
+
},
|
| 208 |
+
"CLICKAMD": {
|
| 209 |
+
"accuracy": 0.8621,
|
| 210 |
+
"clips": 116
|
| 211 |
+
},
|
| 212 |
+
"HONEYPOTAMD": {
|
| 213 |
+
"accuracy": 0.8789,
|
| 214 |
+
"clips": 190
|
| 215 |
+
},
|
| 216 |
+
"NAMEONLYAMD": {
|
| 217 |
+
"accuracy": 0.8853,
|
| 218 |
+
"clips": 218
|
| 219 |
+
},
|
| 220 |
+
"CALLRECAMD": {
|
| 221 |
+
"accuracy": 0.907,
|
| 222 |
+
"clips": 215
|
| 223 |
+
},
|
| 224 |
+
"OTHERAMD": {
|
| 225 |
+
"accuracy": 0.9179,
|
| 226 |
+
"clips": 134
|
| 227 |
+
},
|
| 228 |
+
"AMD": {
|
| 229 |
+
"accuracy": 0.9277,
|
| 230 |
+
"clips": 235
|
| 231 |
+
},
|
| 232 |
+
"FAXAMD": {
|
| 233 |
+
"accuracy": 0.9412,
|
| 234 |
+
"clips": 136
|
| 235 |
+
},
|
| 236 |
+
"BEEPAMD": {
|
| 237 |
+
"accuracy": 0.9433,
|
| 238 |
+
"clips": 141
|
| 239 |
+
},
|
| 240 |
+
"RINGINGAMD": {
|
| 241 |
+
"accuracy": 0.9437,
|
| 242 |
+
"clips": 142
|
| 243 |
+
}
|
| 244 |
+
},
|
| 245 |
+
"baseline": {
|
| 246 |
+
"threshold": 0.9084879043645666,
|
| 247 |
+
"target_human_recall": 0.99,
|
| 248 |
+
"test": {
|
| 249 |
+
"human_recall": 0.9905660377358491,
|
| 250 |
+
"machine_recall": 0.9266666666666666,
|
| 251 |
+
"balanced_acc": 0.9586163522012578,
|
| 252 |
+
"auc": 0.9968238993710692,
|
| 253 |
+
"balanced_acc_95ci": [
|
| 254 |
+
0.9415466057914559,
|
| 255 |
+
0.9730644354969796
|
| 256 |
+
]
|
| 257 |
+
},
|
| 258 |
+
"test_at_0.5": {
|
| 259 |
+
"human_recall": 0.9622641509433962,
|
| 260 |
+
"machine_recall": 0.99,
|
| 261 |
+
"balanced_acc": 0.9761320754716981,
|
| 262 |
+
"auc": 0.9968238993710692
|
| 263 |
+
},
|
| 264 |
+
"holdout": {
|
| 265 |
+
"machine_recall": 0.9327142857142857
|
| 266 |
+
},
|
| 267 |
+
"holdout_at_0.5": {
|
| 268 |
+
"machine_recall": 0.9812857142857143
|
| 269 |
+
}
|
| 270 |
+
}
|
| 271 |
+
}
|
inference.py
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Self-contained inference for the AIAMD GRU: is this call answered by a human or a machine?
|
| 2 |
+
|
| 3 |
+
pip install tensorflow tensorflow-hub "setuptools<81" soundfile scipy numpy huggingface_hub
|
| 4 |
+
python inference.py call.wav
|
| 5 |
+
|
| 6 |
+
Pipeline (identical to training):
|
| 7 |
+
mono audio -> fill runs of exact zeros (>= 10 ms) with -70 dBFS noise -> causal 3.6 kHz low-pass
|
| 8 |
+
-> resample to 16 kHz -> first 3 s -> YAMNet (frozen) embeddings, one per 0.96 s window every 0.48 s
|
| 9 |
+
-> GRU -> p(machine) after every window -> temperature calibration -> threshold
|
| 10 |
+
"""
|
| 11 |
+
import json
|
| 12 |
+
import sys
|
| 13 |
+
from math import gcd
|
| 14 |
+
from pathlib import Path
|
| 15 |
+
|
| 16 |
+
import numpy as np
|
| 17 |
+
import soundfile as sf
|
| 18 |
+
from scipy.signal import butter, resample_poly, sosfilt
|
| 19 |
+
|
| 20 |
+
HERE = Path(__file__).resolve().parent
|
| 21 |
+
YAMNET_HANDLE = "https://tfhub.dev/google/yamnet/1"
|
| 22 |
+
CLIP_SECONDS = 3.0
|
| 23 |
+
WINDOW_SAMPLES, HOP_SAMPLES = 15600, 7680 # YAMNet window and hop at 16 kHz
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def preprocess(x: np.ndarray, sr: int, seed: int = 0) -> np.ndarray:
|
| 27 |
+
"""Zero fill + low-pass at the original sample rate, then resample to 16 kHz."""
|
| 28 |
+
x = x.astype(np.float32).copy()
|
| 29 |
+
edges = np.diff(np.concatenate([[0], (x == 0).astype(np.int8), [0]]))
|
| 30 |
+
starts, ends = np.where(edges == 1)[0], np.where(edges == -1)[0]
|
| 31 |
+
rng = np.random.default_rng(seed)
|
| 32 |
+
for s, e in zip(starts, ends, strict=True):
|
| 33 |
+
if e - s >= int(0.010 * sr):
|
| 34 |
+
x[s:e] = rng.normal(0, 10 ** (-70 / 20), e - s)
|
| 35 |
+
x = sosfilt(butter(8, 3600, btype="low", fs=sr, output="sos"), x).astype(np.float32)
|
| 36 |
+
g = gcd(sr, 16000)
|
| 37 |
+
return resample_poly(x, 16000 // g, sr // g).astype(np.float32) if sr != 16000 else x
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
def window_dbfs(x16k: np.ndarray, n: int) -> np.ndarray:
|
| 41 |
+
padded = np.zeros((n - 1) * HOP_SAMPLES + WINDOW_SAMPLES, np.float32)
|
| 42 |
+
padded[: len(x16k)] = x16k
|
| 43 |
+
frames = np.lib.stride_tricks.sliding_window_view(padded, WINDOW_SAMPLES)[::HOP_SAMPLES]
|
| 44 |
+
return 20 * np.log10(np.sqrt(np.mean(np.square(frames, dtype=np.float64), axis=1)) + 1e-10)
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
class AmdDetector:
|
| 48 |
+
def __init__(self, folder: Path = HERE):
|
| 49 |
+
import keras
|
| 50 |
+
import tensorflow_hub as hub
|
| 51 |
+
|
| 52 |
+
self.yamnet = hub.load(YAMNET_HANDLE)
|
| 53 |
+
self.model = keras.models.load_model(folder / "amd_gru.keras")
|
| 54 |
+
calibration = json.loads((folder / "amd_gru_calibration.json").read_text())
|
| 55 |
+
self.temperature, self.threshold = calibration["temperature"], calibration["threshold"]
|
| 56 |
+
|
| 57 |
+
def predict(self, x: np.ndarray, sr: int) -> dict:
|
| 58 |
+
x16k = preprocess(x, sr)[: int(CLIP_SECONDS * 16000)]
|
| 59 |
+
_, embeddings, _ = self.yamnet(x16k)
|
| 60 |
+
p = self.model.predict(embeddings.numpy()[None], verbose=0)[0, :, 0]
|
| 61 |
+
logit = np.log(np.clip(p, 1e-6, 1 - 1e-6) / np.clip(1 - p, 1e-6, 1))
|
| 62 |
+
p = 1 / (1 + np.exp(-logit / self.temperature)) # calibrated p(machine) per window
|
| 63 |
+
return {
|
| 64 |
+
"decision": "machine" if p[-1] >= self.threshold else "human",
|
| 65 |
+
"p_machine": float(p[-1]),
|
| 66 |
+
"threshold": self.threshold,
|
| 67 |
+
"p_machine_per_window": [round(float(v), 4) for v in p],
|
| 68 |
+
"window_dbfs": [round(float(v), 1) for v in window_dbfs(x16k, len(p))],
|
| 69 |
+
}
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def main() -> None:
|
| 73 |
+
if len(sys.argv) != 2:
|
| 74 |
+
raise SystemExit("usage: python inference.py call.wav")
|
| 75 |
+
x, sr = sf.read(sys.argv[1], dtype="float32", always_2d=True)
|
| 76 |
+
print(json.dumps(AmdDetector().predict(x.mean(axis=1), sr), indent=2))
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
if __name__ == "__main__":
|
| 80 |
+
main()
|
training_summary.json
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created": "2026-11-16T09:43:43+00:00",
|
| 3 |
+
"model_path": "models/amd_gru.keras",
|
| 4 |
+
"monitor": "val_loss",
|
| 5 |
+
"best_epoch": 4,
|
| 6 |
+
"epochs_run": 14,
|
| 7 |
+
"best_val": {
|
| 8 |
+
"loss": 0.0763,
|
| 9 |
+
"val_loss": 0.0777,
|
| 10 |
+
"val_window_acc": 0.9694,
|
| 11 |
+
"window_acc": 0.9609,
|
| 12 |
+
"val_clip_balanced_acc": 0.9771,
|
| 13 |
+
"val_clip_auc": 0.9964
|
| 14 |
+
},
|
| 15 |
+
"class_weights": {
|
| 16 |
+
"0": 1.2205114570240743,
|
| 17 |
+
"1": 0.846975745580194
|
| 18 |
+
},
|
| 19 |
+
"silence_dbfs": -50.0,
|
| 20 |
+
"model": {
|
| 21 |
+
"dense_units": 32,
|
| 22 |
+
"gru_units": 48
|
| 23 |
+
},
|
| 24 |
+
"train": {
|
| 25 |
+
"epochs": 50,
|
| 26 |
+
"batch_size": 32,
|
| 27 |
+
"learning_rate": 0.001,
|
| 28 |
+
"model_path": "models/amd_gru.keras",
|
| 29 |
+
"monitor": "val_loss",
|
| 30 |
+
"patience": 10,
|
| 31 |
+
"threshold": 0.5
|
| 32 |
+
},
|
| 33 |
+
"params": 44657
|
| 34 |
+
}
|