embeddinggemma-2-oQ4e

This model was quantized using oQ (oMLX v0.7.1) mixed-precision quantization with Improved oQe Quantization (jundot/omlx#4385).

A mixed-precision MLX checkpoint of google/embeddinggemma-2, Google DeepMind's multimodal embedding model that maps text, images, video and audio into one 768-dimensional vector space. It includes the text encoder, the vision encoder and the audio encoder. oQ quantizes the text encoder; the vision and audio encoders stay in BF16.

oMLX serves text and image inputs through POST /v1/embeddings (jundot/omlx#4333) and does not load the audio encoder.

The effective average is 11.991 bits/weight for the complete checkpoint.

Quantization and Bit Distribution

oQ4e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (128 samples x 512 tokens) and a weighted least-squares refit of its scale and bias.

Storage format	Logical weights	Share of total	Tensor storage	Effective bits/weight
Affine 4-bit, group 64	0.211B	28.314%	0.110 GiB	4.50
Affine 5-bit, group 64	0.033B	4.367%	0.021 GiB	5.50
Affine 6-bit, group 64	0.002B	0.211%	0.001 GiB	6.50
Affine 8-bit, group 64	0.027B	3.663%	0.027 GiB	8.50
BF16	0.472B	63.445%	0.880 GiB	16.00
Total	0.744B	100%	1.039 GiB	11.991

Storage Breakdown

Component	Logical weights	Storage
Text encoder	0.272B	0.171 GB / 0.160 GiB
Vision encoder	0.167B	0.335 GB / 0.312 GiB
Audio encoder	0.305B	0.610 GB / 0.568 GiB
Total	0.744B	1.116 GB / 1.039 GiB

License

Apache-2.0, following the source model. See LICENSE.