--- library_name: keras pipeline_tag: audio-classification tags: - answering-machine-detection - telephony - voicemail-detection - yamnet - gru --- # AIAMD: answering machine detection (YAMNet + GRU) Decides whether an outbound call was answered by a **human** or a **machine** (voicemail, call screener, IVR, network tones), from the first ≤ 3 s of the callee's audio. An implementation of *A Recurrent Neural Network Approach to the Answering Machine Detection Problem* (Altwlkany et al., 2024, [arXiv:2410.08235](https://arxiv.org/abs/2410.08235)): a frozen [YAMNet](https://tfhub.dev/google/yamnet/1) turns every 0.96 s of audio (every 0.48 s) into 1,024 numbers, and a small GRU gives `p(machine)` after each window, so it can decide while the call is still going. ``` 8 kHz audio -> fill digital silence + 3.6 kHz low-pass -> 16 kHz -> YAMNet (frozen) -> Dense(32, tanh) -> GRU(48) -> Dense(1, sigmoid) = p(machine) after every 0.48 s ``` 44,657 trainable parameters. ## Files | File | What | |---|---| | `amd_gru.keras` | the GRU (input: YAMNet embeddings, shape `(windows, 1024)`) | | `amd_gru_calibration.json` | temperature (1.156) and decision threshold (0.452), fitted on validation data | | `inference.py` | self-contained preprocessing + YAMNet + GRU + decision | | `config.yaml` | preprocessing, model and decision settings | | `training_summary.json`, `amd_gru_history.csv` | training run (best epoch 4 by validation loss) | | `evaluation.json` | full test / holdout results | ## Usage ```bash pip install tensorflow tensorflow-hub "setuptools<81" soundfile scipy numpy huggingface_hub hf download kugu/AIAMD --local-dir AIAMD python AIAMD/inference.py call.wav ``` ```python from inference import AmdDetector detector = AmdDetector() # folder with the downloaded files result = detector.predict(samples, sample_rate) # mono float audio, any sample rate # {"decision": "machine", "p_machine": 0.99, "threshold": 0.452, "p_machine_per_window": [...], ...} ``` **Preprocessing must match training**: runs of exact-zero samples ≥ 10 ms are replaced by -70 dBFS noise, then a causal 8th-order Butterworth low-pass at 3.6 kHz, then resampling to 16 kHz, then the first 3 s. `inference.py` does all of this. ## Training data (not released) | | Clips | Description | |---|---|---| | Human | 2,175 | the callee's first response on outbound calls, ≤ 3 s, 8 kHz | | Machine | 3,000 for training (+ 7,000 holdout) | vendor-labelled recordings: voicemail, call screeners, IVR, network tones, beeps | - Splits 80 / 10 / 10 by call, so no call appears in two splits. - Both classes get the same random leading (0–1 s) and trailing (0–0.5 s) silence, made from the clip's own background noise, and machine clips are cut to lengths drawn from the human clips. Clip length and silence therefore don't reveal the class. - Windows quieter than -50 dBFS get zero weight in training. ## Results Test: 212 human + 300 machine clips. Holdout: 7,000 machine clips. Decision after the whole clip. | Threshold | Human recall | Machine recall (test) | Machine recall (holdout) | Balanced acc. | AUC | |---|---|---|---|---|---| | 0.5 | 0.972 | 0.950 | 0.962 | 0.961 | 0.993 | | 0.452 (calibrated, target 99 % human recall on val) | 0.962 | 0.963 | 0.965 | 0.963 (95 % CI 0.947–0.979) | 0.993 | Decision after k windows (test, threshold 0.452): | Audio heard | 0.98 s | 1.46 s | 1.94 s | 2.42 s | 2.90 s | |---|---|---|---|---|---| | Balanced accuracy | 0.868 | 0.927 | 0.940 | 0.958 | 0.965 | Machine categories (test + holdout): voicemail 0.974, network / tone 0.955, call screening 0.949, IVR 0.948. Hardest: custom voicemail, clicks, name-only greetings (~0.86–0.89). ## Limitations - **Recording channel.** All human audio comes from one dialer and all machine audio from one vendor, so part of what the model learned may be the recording channel. Scores on audio from other systems may be lower. - **Strict human-safety targets.** The validation set has 220 humans, too few to set a 99 % human-recall threshold reliably: on test the calibrated threshold gives 96.2 %. A few human clips get confident `machine` scores, so at 99 % human recall only about 73 % of machines are caught. In the same setting a gradient-boosted model on hand-crafted timing / loudness / MFCC features caught 94 %. Choose the threshold on your own data. - **At most 3 s** of audio (training clips were ≤ 3 s); the paper uses 4 s. - Labels were screened automatically, not reviewed by ear. - 8 kHz telephony audio only. ## Citation ```bibtex @article{altwlkany2024amd, title = {A Recurrent Neural Network Approach to the Answering Machine Detection Problem}, author = {Altwlkany, Kemal and Delali{\'c}, Sead and Selmanovi{\'c}, Elmedin and Alihod{\v{z}}i{\'c}, Adis and Lovri{\'c}, Ivica}, journal = {arXiv preprint arXiv:2410.08235}, year = {2024} } ```