Hugging Face's logo

fredrunit
/
embeddinggemma-2-text-8bit

Sentence Similarity
MLX
Safetensors
embedding_gemma2
embeddings
text-embeddings
8-bit precision
Model card Files
xet
 Community
embeddinggemma-2-text-8bit / README.md
fredrunit's picture
fredrunit
EmbeddingGemma 2 text-only 8-bit MLX (from google/embeddinggemma-2 914f7f89)
997ec04 verified
preview code
|
Raw History Blame Contribute Delete
1.34 kB
metadata 
license: apache-2.0
base_model: google/embeddinggemma-2
library_name: mlx
pipeline_tag: sentence-similarity
tags:
  - mlx
  - embeddings
  - text-embeddings

EmbeddingGemma 2, text only, 8-bit MLX

google/embeddinggemma-2 (revision 914f7f89142e33e77833254d9c9b90c3cef7303b) with its image and audio encoders removed, for text search on Apple silicon. 320 MB in all, against about 1.27 GB for the 8-bit conversion that keeps the encoders.

  • Text tower only (271M of the model's 744M parameters).
  • Matrices quantized affine, 8 bits, group 64; scales, biases, norms and scalars in bfloat16 (never float16: the model overflows in float16).
  • 768-dimensional output with Matryoshka prefixes 512, 256 and 128; mean pooling, L2 normalization. Use the source model's prompts, for example task: search result | query: … and title: none | text: ….
  • The tokenizer and the sentence-transformers configs are Google's files, byte for byte.

Converted by RuniT-ChatFramework's Scripts/convert-embeddinggemma2-text.py. Its vectors match the FP32 reference cosines of Google's checkpoint within 0.002 on multilingual query and document pairs, the same as the 8-bit build with encoders.

License: Apache 2.0, as the source model (https://ai.google.dev/gemma/docs/gemma_4_license).