androiddrew/google-speech-embedding-pytorch

Google speech_embedding/1, converted to PyTorch

This is a PyTorch/safetensors conversion of Google's speech_embedding/1, a small pretrained convolutional speech encoder (Lin et al., 2020). It maps 16 kHz audio to one 96-dimensional embedding every 80 ms. Those embeddings describe speech sounds; the model does not produce transcripts.

It is the frozen audio feature extractor used by the keyword models in naptrap, such as androiddrew/naptrap-phonmatchnet-en.

This conversion is not affiliated with or endorsed by Google. The original model is published by Google under the Apache License 2.0. This conversion keeps that license; see LICENSE and NOTICE.

Usage

pip install "naptrap[hub]"

from huggingface_hub import snapshot_download
from naptrap import GoogleSpeechEmbedding, load_wav

encoder = GoogleSpeechEmbedding.from_pretrained(
    snapshot_download("androiddrew/google-speech-embedding-pytorch", revision="v1")
)
waveform = load_wav("clip.wav")          # mono 16 kHz, 0.5–2 s
embeddings = encoder(waveform)           # torch.float32 [len(waveform) // 1280, 96]

The runtime needs only PyTorch; TensorFlow is not used.

Preprocessing contract

Step	Value
Input	mono 16 kHz float32 in [-1, 1]; naptrap accepts 0.5–2 s clips
Padding	11,120 leading zero samples, no trimming
STFT	400-sample Hann window, hop 160, 512-point FFT, no centering, magnitude
Mel	32 bins, natural log(mel + 0.001)
Output	floor(samples / 1280) frames × 96

The keyword models were trained on features computed this way, one clip at a time. Slicing features computed over a longer recording gives different values.

Conversion and accuracy

All 96 original variables (329,448 parameters plus BatchNorm statistics) and the original window and mel-matrix constants are kept. The network is re-implemented with standard PyTorch operations. conversion/ documents each step; config.json and NOTICE list the modifications.

The results match the TensorFlow original to within floating-point differences between FFT implementations. They are not bit-identical. Across CPU runtimes from Python 3.8/PyTorch 1.13 to Python 3.14/PyTorch 2.10 (see qualification/):

• the maximum feature difference on real speech was 7.2e-5;
• on 231 downstream keyword-model cases, the probability difference was at most 2.45e-6, and every accept/reject decision matched.

Pure-tone stress inputs show larger, documented FFT differences.

Training data and limitations

Google reports training the original on roughly 200 million two-second YouTube clips covering about 5,000 keywords (Lin et al., 2020). No clip-level inventory is available. This repository adds no training. Behaviour on non-English speech, far-field audio or unusual microphones is unknown.

Citation

@article{lin2020training,
  title   = {Training Keyword Spotters with Limited and Synthesized Speech Data},
  author  = {Lin, James and Kilgour, Kevin and Roblek, Dominik and Sharifi, Matthew},
  journal = {arXiv preprint arXiv:2002.01322},
  year    = {2020}
}