vespa-engine/embeddinggemma-2-ONNX

EmbeddingGemma 2 — text-only ONNX (for Vespa)

Text-only ONNX export of google/embeddinggemma-2, derived from onnx-community/embeddinggemma-2-ONNX for use with Vespa's hugging-face-embedder.

The upstream graph requires image_features, video_features and audio_features inputs that are concatenated after the text tokens. These are replaced with empty [0, 512] constants, so the graph takes only input_ids and attention_mask. Weights are unchanged. See scripts/make_text_only.py.

File	Precision	Size
onnx/model.onnx (+ model.onnx_data)	fp32	1.08 GB
onnx/int8/model.onnx (+ model.onnx_data)	int8 (dynamic quantization)	314 MB

Outputs: last_hidden_state [batch, seq, 768] (use mean pooling + L2 normalize) and sentence_embedding [batch, 768].

Do not use fp16. Per the model card, activations overflow float16.

Verification

scripts/verify.py compares Vespa-style inference (tokenizer.json with special tokens, mean pooling over last_hidden_state, L2 normalization) against sentence-transformers 6.1 / transformers 5.19 in fp32, on 11 texts including multilingual, code, and a 2,718-token document (exercises sliding-window attention). Token IDs are identical to the reference.

Variant	Cosine vs. reference (min, 768d / 512d / 256d / 128d)	Same ranking
fp32	1.000000 / 1.000000 / 1.000000 / 1.000000 (max abs diff 6e-7)	yes
int8	0.999896 / 0.999901 / 0.999910 / 0.999940	yes

Vespa usage

<component id="embeddinggemma2" type="hugging-face-embedder">
    <transformer-model url="https://huggingface.co/vespa-engine/embeddinggemma-2-ONNX/resolve/main/onnx/model.onnx"/>
    <tokenizer-model url="https://huggingface.co/vespa-engine/embeddinggemma-2-ONNX/resolve/main/tokenizer.json"/>
    <max-tokens>8192</max-tokens>
    <pooling-strategy>mean</pooling-strategy>
    <normalize>true</normalize>
    <prepend>
        <query>task: search result | query: </query>
        <document>title: none | text: </document>
    </prepend>
</component>

Supports Matryoshka truncation to 512, 256 and 128 dimensions (re-normalize after truncating).