File size: 2,166 Bytes
cf06ce2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | V1 speech recognition on Core ML
Copyright Harshit B.
The repository as a whole is licensed under the Apache License, Version 2.0 (LICENSE).
1. Core ML package (v1.mlpackage), derived from google/embeddinggemma-2
It contains layers 0-11 of the audio tower of EmbeddingGemma 2 by Google DeepMind,
https://huggingface.co/google/embeddinggemma-2 (revision 914f7f89142e33e77833254d9c9b90c3cef7303b),
licensed under the Apache License, Version 2.0
(https://ai.google.dev/gemma/docs/gemma_4_license).
Modifications made by Harshit B.:
- the audio tower truncated after layer 11 and converted to Core ML ML programs,
one function per fixed input window (5, 10, 20, 35 s) sharing one set of weights;
- blocked local attention rewritten as dense banded attention (identical in fp32);
- RMSNorm inputs scaled by powers of two so fp16 norms on the CPU / GPU do not
overflow, and constant padding fill (identical in fp32);
- log-softmax rewritten in a numerically stable form; static shapes;
- a per-frame int8 quantize/dequantize step, standardization and a CTC head appended;
- weights stored in float16.
The upstream model card states that deployments must adhere to the Gemma Prohibited
Use Policy: https://ai.google.dev/gemma/prohibited_use_policy
config.json embeds the upstream preprocessor_config.json unchanged.
2. CTC head (inside v1.mlpackage) and tokenizer (spm_unigram256.model)
Trained by Harshit B. on the LibriSpeech ASR corpus, OpenSLR SLR12,
https://www.openslr.org/12/ (license stated there: CC BY 4.0).
V. Panayotov, G. Chen, D. Povey and S. Khudanpur, "LibriSpeech: an ASR corpus based
on public domain audio books", ICASSP 2015.
3. Language model (sp256_6gram.bin)
Built with KenLM (https://github.com/kpu/kenlm; no KenLM code is included) from
librispeech-lm-norm.txt.gz of "LibriSpeech language models, vocabulary and G2P models",
OpenSLR SLR11, https://www.openslr.org/11/ (license stated there: Public domain).
4. Decoder (not included; used by the usage example)
ctc-beam-decoder 0.2.0, https://pypi.org/project/ctc-beam-decoder/,
Apache License, Version 2.0.
|