enconvo's picture
EmbeddingGemma 2 for MLX: bf16 with an 8-bit token table, audio removed
d0090d2 verified
|
Raw History Blame Contribute Delete
1.44 kB
---
base_model: google/embeddinggemma-2
library_name: mlx
license: apache-2.0
pipeline_tag: feature-extraction
tags:
- mlx
- embedding
- feature-extraction
- sentence-similarity
- image-feature-extraction
---
# enconvo/embeddinggemma-2-mlx
[EmbeddingGemma 2](https://huggingface.co/google/embeddinggemma-2) by Google for MLX,
as Enconvo's Local Embeddings runs it on the Mac: text and image encoders,
768-dimensional normalized vectors.
Built from [`mlx-community/embeddinggemma-2-bf16`](https://huggingface.co/mlx-community/embeddinggemma-2-bf16/tree/1a4ffddb7905d3f63486748deabe091a01fb6201)
(revision `1a4ffddb`, converted from Google's revision
`914f7f89`):
- The audio encoder is removed (`audio_config: null`).
- The token embedding table (`language_model.embed_tokens`) is stored in 8 bits
(affine, group size 64): it is only looked up, so this costs no speed.
- Every other weight is bf16, unchanged. Keep activations in bf16.
`config.json` records the quantization of that one module, in mlx-lm's layout.
Checked against the source on a fixed probe set (16 texts, 8 images at
soft-token budgets 70, 280): minimum cosine 0.999761 for text
and 0.999757 for images.
Use the task prefixes in `config_sentence_transformers.json`
(`task: search result | query: ` for queries, `title: none | text: ` for
documents). An image is embedded as `<bos> <boi> image tokens <eoi> <eos>`.
Licence: Apache 2.0, see `LICENSE` and `NOTICE`.