harshit2312's picture
stt-gemma_embedding-2: Core ML package (fp16, multifunction), tokenizer, subword 6-gram LM
cf06ce2 verified
Raw History Blame Contribute Delete
2.17 kB
V1 speech recognition on Core ML
Copyright Harshit B.
The repository as a whole is licensed under the Apache License, Version 2.0 (LICENSE).
1. Core ML package (v1.mlpackage), derived from google/embeddinggemma-2
It contains layers 0-11 of the audio tower of EmbeddingGemma 2 by Google DeepMind,
https://huggingface.co/google/embeddinggemma-2 (revision 914f7f89142e33e77833254d9c9b90c3cef7303b),
licensed under the Apache License, Version 2.0
(https://ai.google.dev/gemma/docs/gemma_4_license).
Modifications made by Harshit B.:
- the audio tower truncated after layer 11 and converted to Core ML ML programs,
one function per fixed input window (5, 10, 20, 35 s) sharing one set of weights;
- blocked local attention rewritten as dense banded attention (identical in fp32);
- RMSNorm inputs scaled by powers of two so fp16 norms on the CPU / GPU do not
overflow, and constant padding fill (identical in fp32);
- log-softmax rewritten in a numerically stable form; static shapes;
- a per-frame int8 quantize/dequantize step, standardization and a CTC head appended;
- weights stored in float16.
The upstream model card states that deployments must adhere to the Gemma Prohibited
Use Policy: https://ai.google.dev/gemma/prohibited_use_policy
config.json embeds the upstream preprocessor_config.json unchanged.
2. CTC head (inside v1.mlpackage) and tokenizer (spm_unigram256.model)
Trained by Harshit B. on the LibriSpeech ASR corpus, OpenSLR SLR12,
https://www.openslr.org/12/ (license stated there: CC BY 4.0).
V. Panayotov, G. Chen, D. Povey and S. Khudanpur, "LibriSpeech: an ASR corpus based
on public domain audio books", ICASSP 2015.
3. Language model (sp256_6gram.bin)
Built with KenLM (https://github.com/kpu/kenlm; no KenLM code is included) from
librispeech-lm-norm.txt.gz of "LibriSpeech language models, vocabulary and G2P models",
OpenSLR SLR11, https://www.openslr.org/11/ (license stated there: Public domain).
4. Decoder (not included; used by the usage example)
ctc-beam-decoder 0.2.0, https://pypi.org/project/ctc-beam-decoder/,
Apache License, Version 2.0.