| --- |
| license: gemma |
| license_link: https://ai.google.dev/gemma/terms |
| base_model: google/embeddinggemma-300m |
| base_model_relation: quantized |
| library_name: coreai |
| pipeline_tag: feature-extraction |
| tags: |
| - core-ai |
| - aimodel |
| - apple-silicon |
| - on-device |
| - coreai-kit |
| - embeddings |
| - sentence-embedding |
| - sentence-similarity |
| - quantized |
| - int8 |
| extra_gated_heading: Access EmbeddingGemma on Hugging Face |
| extra_gated_description: >- |
| This artifact is a derivative of google/embeddinggemma-300m and is governed by |
| the Gemma Terms of Use, the Gemma Prohibited Use Policy and the Gemma license. |
| --- |
| |
| # embeddinggemma-300m — Core AI (.aimodel) |
|
|
| `google/embeddinggemma-300m` exported as a single static Core AI graph for Apple silicon by |
| [visible-cx](https://huggingface.co/visible-cx). **The whole embedding pipeline is in the |
| graph** — transformer → mean pooling → dense stack → L2 normalise — so it is one call in, one |
| 768-d unit vector out, with no host-side pooling to implement. |
|
|
| Two variants ship here: an **fp32** bundle and an **int8** bundle produced by compressing that |
| graph in place. **Their status is not the same** — read [Status](#status) before choosing. |
|
|
| > ⚠️ **Neither bundle has produced an embedding vector on Apple silicon.** Everything below the |
| > Provenance table is a build-and-bytes result, not a runtime result. |
|
|
| ## Contents |
|
|
| | Path | Bytes | Weights | Sequence length | |
| |---|---:|---|---| |
| | `model/` — `embeddinggemma-300m_float32_static.aimodel` + `reference.json` + `tokenizer/` | 1,277,545,730 | fp32 | 256 | |
| | `int8/` — `embeddinggemma-300m_int8_static.aimodel` + `reference.json` + `tokenizer/` | 356,639,960 | int8 (graph-compressed) | 256 | |
|
|
| | File | fp32 (`model/`) | int8 (`int8/`) | |
| |---|---:|---:| |
| | `main.mlirb` | 1,244,157,766 | **323,252,231** (0.26×) | |
| | `main.hash` | 32 | 32 | |
| | asset `metadata.json` | 340 | 105 | |
| | `reference.json` | 1,615 | 1,615 | |
| | `tokenizer/tokenizer.json` | 33,385,272 | 33,385,272 | |
| | `tokenizer/tokenizer_config.json` | 705 | 705 | |
|
|
| `reference.json` and `tokenizer/` in `int8/` are byte-identical copies of the fp32 bundle's, |
| carried so the folder is self-contained. `reference.json` is the exporter's torch oracle: four |
| reference texts and their pairwise cosines, for checking the *source* pipeline end to end. |
|
|
| ## Provenance |
|
|
| | | | |
| |---|---| |
| | Base checkpoint | `google/embeddinggemma-300m` | |
| | Recipe (fp32) | `coreai-model-zoo/conversion/export_embeddinggemma.py --dtype float32 --seq-len 256` | |
| | Recipe index | `models/qwen3-embedding/recipe.toml` → `["embeddinggemma-300m"]`, `status = "verified"` | |
| | Upstream reference bundle | [`mlboydaisuke/embeddinggemma-300m-CoreAI`](https://huggingface.co/mlboydaisuke/embeddinggemma-300m-CoreAI) @ `d9a60a18d384484a8dab809b3070b82fb41458ce` | |
| | Producer fingerprint | `coreai-core 1.0.0b2` on both inner asset `metadata.json` files | |
| | Asset creation dates | `20260817T201458Z` (fp32), `20260817T212021Z` (int8) | |
| | Toolchain | `coreai-torch 0.4.1`, `coreai-core 1.0.0b2`, `coreai-opt 0.2.1`, `torch 2.9.0` | |
| | Asset metadata | `author: "Google DeepMind"`, `license: "Gemma"` (fp32 bundle) | |
|
|
| These are an **independent export from the zoo's verified recipe**, at the same layout as the |
| zoo's own bundle — not a byte mirror of it. Rebuilt from the same recipe, `main.mlirb` differed |
| from the published upstream bundle by **+262 bytes**, and two runs on the same host with an |
| identical command differed from each other by **+32 bytes**. The tokenizer half is identical in |
| every comparison. |
|
|
| **The `int8/` bundle is not a second export.** It is the fp32 bundle put through |
| `coreai_opt.coreai_utils.quantize_weights(dtype=DType.INT8)` — the Core AI *graph* compression |
| path, which rewrites an existing program's weight constants in place. There was no PyTorch |
| round-trip and no re-trace: the graph, the input signature and the tokenizer are the fp32 |
| bundle's, unchanged. |
|
|
| ## Requirements |
|
|
| - **Apple silicon Mac**, Core AI runtime, macOS 27+. |
| - **Static input signature:** `input_ids [1, 256] int32`, `attention_mask [1, 256] int32` → |
| `embedding [1, 768]` fp32, L2-normalised. **Sequence length 256 is baked into the graph** — |
| pad or truncate to it; there is no dynamic-length variant. |
| - Weights ≈ 1.24 GB resident (fp32) or ≈ 0.32 GB (int8). **No KV cache** — this is an encoder, |
| so there is no per-token memory growth and no context ladder, and none of the residency |
| cautions that apply to the generative bundles in this org apply here. |
| - **Minimum practical machine memory: 8 GB.** |
|
|
| Note the sequence-length difference from the Core ML artifact in this org |
| ([`visible-cx/embeddinggemma-300m-CoreML`](https://huggingface.co/visible-cx/embeddinggemma-300m-CoreML)), |
| which is **seq 128**. The two are not drop-in substitutes for each other. |
|
|
| ## Measurements |
|
|
| **None, from either bundle.** No embedding vector has been computed from either artifact on |
| Apple silicon: no throughput, no latency, no ANE-residency figure, no cross-runtime vector |
| parity. `export_embeddinggemma.py` exposes only `--dtype`, `--seq-len`, `--output-dir`, |
| `--overwrite`; there is no compute-unit flag to report either. |
|
|
| What *is* verified is the source pipeline, not the exported graph's execution: |
|
|
| - **Torch oracle agrees to ~6e-8.** `reference.json`'s pairwise cosines from an independent |
| rebuild match the published values to roughly seven decimal places (largest deviation ~6e-8). |
| That is CPU float nondeterminism across machines, not a different model. It validates |
| **checkpoint → prompts → pooling → dense → L2**, and explicitly does **not** validate the |
| exported graph. |
| - **The exporter's pre-export gate passed** — the wrapper against |
| `sentence_transformers.encode`, cosine > 0.999. |
|
|
| **The int8 caveat is not boilerplate.** fp16 embeddinggemma is already a known failure at vector |
| precision, because a full `.to(float16)` overflows Gemma 3 activations and emits NaN embeddings. |
| A 4× compression passing parity is a hypothesis, not an expectation, and the gate it needs is |
| **cross-runtime vector parity against the fp32 bundle on Apple silicon — cosine similarity per |
| vector *and* retrieval-rank agreement on a fixed corpus**, not a spot-check of output shapes. |
|
|
| ## Usage |
|
|
| Swift Package Manager, via [CoreAIKit](https://github.com/john-rocky/coreai-kit) — a community |
| package, not affiliated with Apple, requiring macOS 27 beta: |
|
|
| ```swift |
| .package(url: "https://github.com/john-rocky/coreai-kit", branch: "main") |
| // target dependency: .product(name: "CoreAIKitEmbeddings", package: "coreai-kit") |
| ``` |
|
|
| `TextEmbedder` takes a bundle directory holding one `*.aimodel` plus a `tokenizer/` folder, and |
| applies EmbeddingGemma's own query/document prompt prefixes: |
|
|
| ```swift |
| import CoreAIKitEmbeddings |
| |
| let url = try await ModelStore.default.download( |
| ModelID("visible-cx/embeddinggemma-300m-CoreAI", path: "model")) |
| |
| let embedder = try await TextEmbedder(bundleAt: url, prompts: .embeddingGemma) |
| |
| let q = try await embedder.embed(query: "how do I cancel my order") |
| let d = try await embedder.embed(document: "Orders can be cancelled within 24 hours…") |
| // both are L2-normalised [Float] of embedder.dimension (768); cosine == dot product |
| ``` |
|
|
| Two things to hold onto: |
|
|
| - **Use the matching prompts.** `.embeddingGemma` supplies the query and document prefixes this |
| checkpoint was trained with. Embedding a query with the document prefix quietly degrades |
| retrieval; `embed(text:)` applies no prefix at all and is for when you want that. |
| - **Both sides of a comparison must come from the same artifact.** Vectors from this bundle and |
| vectors from the Core ML sibling are not interchangeable — different sequence length, |
| different runtime, and no measured parity between them. |
|
|
| `embedder.sequenceLength` reads 256 off the graph; text longer than that is truncated. |
|
|
| ## Integrity |
|
|
| Core AI `.aimodel` bundles are **not byte-reproducible** — the exporter is not deterministic |
| even against itself (see Provenance), so integrity here rests on **per-file SHA-256 of the exact |
| published bytes**, not on a rebuild. Every bundle carries `main.hash`, the raw 32 bytes of |
| `sha256(main.mlirb)`; on the Hub the same value is recoverable from the LFS oid without fetching |
| the file. |
|
|
| The sibling [`visible-cx/embeddinggemma-300m-CoreML`](https://huggingface.co/visible-cx/embeddinggemma-300m-CoreML) |
| repo **is** bit-reproducible for its weights — a different supply-chain argument for the same |
| model, and the exception in this org. |
|
|
| ## Status |
|
|
| | Artifact | Status | |
| |---|---| |
| | `model/` (fp32, seq 256) | **UNQUALIFIED — built and fingerprinted, runtime-unverified.** The recipe is the zoo's `status = "verified"` entry, the producer fingerprint passes, and the exporter's pre-export gate passed. No vector has been computed from the *exported graph* on Apple silicon. | |
| | `int8/` (graph-compressed) | **EXPERIMENTAL.** Gate: cross-runtime vector parity against the fp32 bundle on Apple silicon — cosine per vector *and* retrieval-rank agreement on a fixed corpus. Do not use for production embeddings until that passes. | |
|
|
| ## License |
|
|
| EmbeddingGemma is Gemma-family and the upstream checkpoint is **gated** on Hugging Face. These |
| bundles are a derivative of `google/embeddinggemma-300m`, and use is subject to the |
| [Gemma Terms of Use](https://ai.google.dev/gemma/terms) and the |
| [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy). The fp32 |
| bundle's own asset metadata carries `license: "Gemma"` and `author: "Google DeepMind"`. Those |
| terms travel with the artifact and with any redistribution of it; the contribution here is the |
| conversion, not the weights. |
|
|
