| --- |
| license: apache-2.0 |
| base_model: google/embeddinggemma-2 |
| library_name: coreml |
| pipeline_tag: feature-extraction |
| tags: |
| - coreml |
| - apple-neural-engine |
| - embeddings |
| - sentence-similarity |
| - ios |
| language: |
| - multilingual |
| --- |
| |
| # EmbeddingGemma 2 · Core ML for the Apple Neural Engine |
|
|
| Core ML conversion of the **text part** of [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2), |
| used for on-device semantic search in the iOS app *Voce Locale*. It runs on the Apple Neural Engine, so it also |
| works while the app is in the background (where the GPU is not available on iOS). |
|
|
| This is a converted and quantized derivative of Google's model, redistributed under the same Apache 2.0 license |
| (see `LICENSE`). It is not an official Google release. |
|
|
| ## Files |
|
|
| | File | What it is | |
| |---|---| |
| | `EmbeddingGemma2.mlmodelc/` | Compiled Core ML model (8-bit weights, fixed length 512) | |
| | `tokenizer.json`, `tokenizer_config.json` | The original tokenizer of google/embeddinggemma-2, unchanged | |
| | `conversion/` | Scripts used for the conversion (`reference.py`, `convert.py`, `quantize.py`, `check.py`) | |
|
|
| ## Inputs and outputs |
|
|
| - Input `input_ids`: int32 `[1, 512]`, padded with `0`. |
| - Output `embedding`: float32 `[1, 768]`, mean-pooled over the tokens and L2-normalized. |
|
|
| The length is fixed at 512 tokens: with flexible shapes Core ML falls back to the CPU (30–40× slower). At 512 |
| tokens the model's sliding window (512) covers the whole input, so the attention mask only excludes padding. |
|
|
| Prompts, as for the original model: |
| - query: `task: search result | query: …` |
| - document: `title: <title or none> | text: …` |
|
|
| Tokenize with the original `tokenizer.json` (e.g. swift-transformers or Hugging Face `tokenizers`); the token ids |
| match the Python reference exactly. |
|
|
| ## Conversion |
|
|
| Changes made for Core ML, mathematically identical to the original: attention masks built inside the model, |
| attention with batch 1, `rotate_half` with `chunk`, `repeat_kv` with `repeat_interleave`. Then 8-bit linear |
| weight quantization. |
|
|
| ```sh |
| uv venv --python 3.12 .venv |
| uv pip install --python .venv/bin/python "torch==2.7.0" torchvision==0.22.0 transformers sentence-transformers coremltools numpy pillow librosa |
| .venv/bin/python conversion/reference.py # reference vectors (ref.npy) |
| .venv/bin/python conversion/convert.py # EG2fixed512.mlpackage (fp16) |
| .venv/bin/python conversion/quantize.py # EG2-512-int8.mlpackage |
| .venv/bin/python conversion/check.py EG2-512-int8.mlpackage 512 |
| xcrun coremlcompiler compile EG2-512-int8.mlpackage out |
| ``` |
|
|
| ## Measurements (Mac, Apple Silicon, Neural Engine) |
|
|
| | Version | Size | Cosine vs original | Time per 512-token chunk | |
| |---|---|---|---| |
| | fp16, flexible shape | 518 MB | 0.999 | 794 ms (CPU) | |
| | fp16, fixed 512 | 518 MB | 0.999 | 28 ms | |
| | **8-bit, fixed 512 (this repo)** | **262 MB** | **0.998–1.000** | **27 ms** | |
|
|
| ## SHA-256 |
|
|
| ``` |
| 3a4cf264e8322b06dcbfa94f34ff1e5160bbc03d82353db163d2cf623f570e62 EmbeddingGemma2.mlmodelc/analytics/coremldata.bin |
| b7fc9952cf7e7dcf9b3dcb0b9c7e62ba593dd966c3cf57b2680ea0f1f2272fd0 EmbeddingGemma2.mlmodelc/coremldata.bin |
| 2d1b06ab8f080f03c01c7d898ebf6426d8ffaa642129a3f1c4f9e89769414be8 EmbeddingGemma2.mlmodelc/metadata.json |
| 2fd0524dd86ae24f6b5c767beec120dc59254e18a98e410d7749a3721d03a9a0 EmbeddingGemma2.mlmodelc/model.mil |
| 905dafd9e926250284774fe8561dfe41222f4963a987b8dad04a1ed01f162b9f EmbeddingGemma2.mlmodelc/weights/weight.bin |
| 4d777ef5bdc1aa36227abdfb77c3e49e7b9c892d16e1b6bda41c393504828be4 tokenizer.json |
| 17bd5d6e9364ca49a534e1502076593317c298d4a663623091ed45388f004874 tokenizer_config.json |
| ``` |
|
|
| ## License |
|
|
| Apache License 2.0, as the original model. Original model: © Google, [google/embeddinggemma-2](https://huggingface.co/google/embeddinggemma-2). |
|
|
