typesense

/

models-moved

Like 28

Follow

Typesense 12

Adds google/embeddinggemma-300m as a built-in model, usable as ts/embeddinggemma-300m.

Blocked on typesense/typesense#3038. This model needs the gemma tokenizer type added there; until that ships, model_type: "gemma" is rejected by validate_and_init_local_model. Opening this now so both halves can be reviewed together.

EmbeddingGemma is a 300M-parameter multilingual model trained on 100+ languages, at 768 dims — a third of the size of the multilingual-e5-large / bge-m3 class and the smallest capable multilingual option in the list.

Files

.gitattributes gains an LFS rule for embeddinggemma-300m/model.onnx_data, matching the existing multilingual-e5-large/model.onnx_data line.

About the export

EmbeddingGemma's sentence embedding is not the raw token output. Its pipeline is mean pooling → Dense(768→3072) → Dense(3072→768) → L2 normalise, and the Dense layers come after pooling, so they cannot be left to TextEmbedder::mean_pooling.

The graph therefore ends with the whole pipeline baked in, then broadcasts the finished 768-d vector across the sequence axis. Mean pooling over the attention mask returns that vector unchanged, so Typesense gets the real sentence embedding without any change to the pooling code. This is the same approach used for bge-m3 in #17.

One compatibility note for whoever regenerates this: torch.onnx.export produces IR version 10, and the pinned onnxruntime (rel-1.14.1) accepts at most IR 8. Setting ir_version = 8 before saving is enough — opset 18 itself is fine.

Prefixes

EmbeddingGemma ships asymmetric prompts. Mapped onto Typesense's prefixes, with the trailing space included since prefixes are concatenated raw:

• indexing_prefix: "title: none | text: " (the model's document prompt)
• query_prefix: "task: search result | query: " (the model's query prompt)

Testing

The ONNX was loaded into typesense/typesense:31.0.rc14: the session builds, num_dim resolves to 768, documents index, and nothing is logged as an error. That build has no gemma tokenizer yet, so the full path was verified by feeding the ids produced by the compiled GemmaTokenizer from #3038 through this ONNX and applying TextEmbedder::mean_pooling:

• cosine 1.000000 against sentence-transformers on 15 inputs — Latin, Chinese, Cyrillic, Arabic and Japanese text, emoji, mixed case, whitespace, numerics, and a 3603-token input exercising truncation.
• cosine 1.000000 for a padded batch of 8, exercising batch_encode.
• cosine 1.000000 on the query path, comparing query_prefix + q against prompt_name="query".

The document path lands at 0.991–0.997 rather than 1.000000. That is not the export: src/index.cpp:8348 appends " " after each field value, and removing that single trailing space gives exactly 1.000000. It applies to every model in this repo equally, so I have left it alone.

One note for maintainers

data_md5 is required for any model with external weights — without it download_public_model re-downloads the weights on every start — but the README's config table does not list it. Same note as on #17.

🤖 Generated with Claude Code

https://claude.ai/code/session_01A9ydwqcWZWFG2bLUnDLt8E