File size: 3,520 Bytes
ad7a3dd | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | ---
license: apache-2.0
library_name: naptrap
pipeline_tag: feature-extraction
tags:
- audio
- speech
- keyword-spotting
- wake-word
- speech-embedding
- pytorch
- safetensors
---
# Google speech_embedding/1, converted to PyTorch
This is a PyTorch/safetensors conversion of Google's
[`speech_embedding/1`](https://www.kaggle.com/models/google/speech-embedding/TensorFlow1/speech-embedding/1),
a small pretrained convolutional speech encoder (Lin et al., 2020). It maps
16 kHz audio to one **96-dimensional embedding every 80 ms**. Those embeddings
describe speech sounds; the model does not produce transcripts.
It is the frozen audio feature extractor used by the keyword models in
[naptrap](https://github.com/androiddrew/naptrap), such as
[`androiddrew/naptrap-phonmatchnet-en`](https://huggingface.co/androiddrew/naptrap-phonmatchnet-en).
> This conversion is not affiliated with or endorsed by Google. The original
> model is published by Google under the Apache License 2.0. This conversion
> keeps that license; see `LICENSE` and `NOTICE`.
## Usage
```bash
pip install "naptrap[hub]"
```
```python
from huggingface_hub import snapshot_download
from naptrap import GoogleSpeechEmbedding, load_wav
encoder = GoogleSpeechEmbedding.from_pretrained(
snapshot_download("androiddrew/google-speech-embedding-pytorch", revision="v1")
)
waveform = load_wav("clip.wav") # mono 16 kHz, 0.5–2 s
embeddings = encoder(waveform) # torch.float32 [len(waveform) // 1280, 96]
```
The runtime needs only PyTorch; TensorFlow is not used.
## Preprocessing contract
| Step | Value |
|---|---|
| Input | mono 16 kHz float32 in [-1, 1]; naptrap accepts 0.5–2 s clips |
| Padding | 11,120 leading zero samples, no trimming |
| STFT | 400-sample Hann window, hop 160, 512-point FFT, no centering, magnitude |
| Mel | 32 bins, natural `log(mel + 0.001)` |
| Output | `floor(samples / 1280)` frames × 96 |
The keyword models were trained on features computed this way, one clip at a
time. Slicing features computed over a longer recording gives different values.
## Conversion and accuracy
All 96 original variables (329,448 parameters plus BatchNorm statistics) and
the original window and mel-matrix constants are kept. The network is
re-implemented with standard PyTorch operations. `conversion/` documents each
step; `config.json` and `NOTICE` list the modifications.
The results match the TensorFlow original to within floating-point
differences between FFT implementations. They are not bit-identical. Across
CPU runtimes from Python 3.8/PyTorch 1.13 to Python 3.14/PyTorch 2.10 (see
`qualification/`):
- the maximum feature difference on real speech was 7.2e-5;
- on 231 downstream keyword-model cases, the probability difference was at most
2.45e-6, and every accept/reject decision matched.
Pure-tone stress inputs show larger, documented FFT differences.
## Training data and limitations
Google reports training the original on roughly 200 million two-second YouTube
clips covering about 5,000 keywords (Lin et al., 2020). No clip-level
inventory is available. This repository adds no training. Behaviour on
non-English speech, far-field audio or unusual microphones is unknown.
## Citation
```bibtex
@article{lin2020training,
title = {Training Keyword Spotters with Limited and Synthesized Speech Data},
author = {Lin, James and Kilgour, Kevin and Roblek, Dominik and Sharifi, Matthew},
journal = {arXiv preprint arXiv:2002.01322},
year = {2020}
}
```
|