Canine phonetic alphabet: models

Model weights for Toward Automatic Discovery of a Canine Phonetic Alphabet (ACL 2025). Code: https://github.com/TheronWang/canine-word ยท Paper: https://aclanthology.org/2025.acl-long.451/

File Size What it is Pipeline step
beats_sed_dog.ckpt 788 MB BEATs-CRNN sound event detector, post-trained on more than 9,000 s of manually labelled dog vocalizations. F1 is 0.8556 on the test set; use detection threshold 0.6713. 1, cutting vocal segments
beats_sed_dog_config.yaml 5 KB Configuration of the detector for training and inference (4-layer RNN, 256-dim CNN output, lr 0.001, 200 epochs, batch size 4) 1
dog_hubert_stage2_130k.pt 379 MB Dog-HuBERT (HuBERT Base, fairseq) after stage 2, 130k updates 2, transcription
kmeans_layer11_k250.pkl 0.8 MB MiniBatchKMeans with K = 250 on layer-11 features of the model above. These clusters are the 250 initial vocal units. 2
stage1_targets_mfcc_k100.pkl 17 KB k-means with 100 clusters: the stage-1 HuBERT targets 2 (training only)
stage2_targets_layer6_k200.pkl 0.6 MB k-means with 200 clusters: the stage-2 HuBERT targets 2 (training only)

Dog-HuBERT

  • Training data: six breeds, 306,233 vocal segments (152 h). Clips shorter than 0.35 s were dropped, and clips longer than 5 s were cut into 5 s windows with a 1 s overlap.
  • Three stages: 100 clusters in stage 1, 200 in stage 2, and 250 in stage 3, chosen with the elbow method.
  • Training: learning rate 1e-4. Stages 1 and 2 took 20 and 18 hours on two NVIDIA RTX 4090 GPUs with a batch size of 4.
  • Vocal units: layer-11 features of the stage-2 model, clustered with K-Means (K = 250).

dog_hubert_stage2_130k.pt is checkpoint_105_130000.pt with the optimizer state removed. On 40 vocal segments, it and kmeans_layer11_k250.pkl reproduce the released frame labels exactly.

import joblib, torch
from fairseq import checkpoint_utils

(model,), cfg, task = checkpoint_utils.load_model_ensemble_and_task(["dog_hubert_stage2_130k.pt"])
model.eval()
km = joblib.load("kmeans_layer11_k250.pkl")

wav = torch.zeros(1, 16000)  # 16 kHz mono waveform, not normalised
with torch.no_grad():
    feats, _ = model.extract_features(source=wav, padding_mask=None, mask=False, output_layer=11)
units = km.predict(feats[0].numpy())  # one vocal unit per 20 ms frame

The k-means models were saved with scikit-learn 1.3. Newer versions load them with a version warning.

Sound event detector

beats_sed_dog.ckpt is a PyTorch Lightning checkpoint, configured by beats_sed_dog_config.yaml. To load it you need the ATST-SED code base and the BEATs encoder BEATs_iter3_plus_AS2M.pt. Inference is done by 1_preprocessing/sed/inference/second_cut.py in the code repository, which merges detections less than 0.5 s apart into vocal segments.

Citation

@inproceedings{wang-etal-2025-toward,
    title = "Toward Automatic Discovery of a Canine Phonetic Alphabet",
    author = "Wang, Theron S. and Li, Xingyuan and Lekhak, Hridayesh and Dang, Tuan M. and Wu, Mengyue and Zhu, Kenny Q.",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.acl-long.451/",
    doi = "10.18653/v1/2025.acl-long.451",
    pages = "9207--9219",
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support