typesense

/

models-moved

Like 28

Follow

Typesense 12

Adds BAAI/bge-m3 as a built-in model, usable as ts/bge-m3.

bge-m3 is a multilingual retrieval model covering 100+ languages. It fills a
gap in the current list: the only multilingual options today are the
multilingual-e5-* family and paraphrase-multilingual-mpnet-base-v2.

Files

.gitattributes gains an LFS rule for bge-m3/model.onnx_data, matching the
existing multilingual-e5-large/model.onnx_data line.

Two changes to the upstream ONNX export

1. External data consolidated into one file. The upstream export splits its
weights across model.onnx_data and a second file, Constant_7_attr__value.
EmbedderManager::download_public_model only fetches model.onnx and
model.onnx_data, so the upstream file loads locally but would fail to
initialise when downloaded as a public model. Re-saved with
all_tensors_to_one_file=True.

2. CLS pooling baked into the graph. bge-m3 pools on the CLS token
(1_Pooling/config.json: pooling_mode_cls_token: true), but
TextEmbedder::embed_query always mean-pools. Feeding the raw export through
Typesense gives embeddings at cosine 0.82 to the reference — wrong vectors, not
just slightly worse ones.

Rather than change the C++, the graph now broadcasts the CLS vector across the
sequence axis, so mean pooling over the attention mask returns exactly that
vector. Three nodes, no change to the weights:

cls    = Slice(token_embeddings, starts=[0], ends=[1], axes=[1])
shape  = Shape(token_embeddings)
output = Expand(cls, shape)

Verified against onnxruntime running the upstream export: cosine 1.000000
on every test document.

Testing

Against typesense/typesense:31.0.rc14, with the exact files in this PR placed
in <data-dir>/models/bge-m3/:

• Collection creates, num_dim resolves to 1024.
• Embeddings match the reference CLS output at cosine 1.000000 (6 documents, mixed scripts).
• Cross-lingual retrieval works: "What is the capital of Japan?" ranks 东京是日本的首都。first (distance 0.279); "programozási nyelv" ranks the Python sentence first.

Two notes for maintainers

data_md5 is undocumented. Models with external data need a data_md5 key
in config.json or download_public_model re-downloads the weights on every
start. multilingual-e5-large/config.json has it, but the README's config table
does not list it. This PR sets it; the README could use a row.

bge-m3 is capped at 128 tokens. XLMRobertaTokenizer::Encode
(src/text_embedder_tokenizer.cpp:82) truncates to 128, so bge-m3's 8192-token
context is not reachable. That is a pre-existing limit shared with the
multilingual-e5-* models, and it does not make bge-m3 wrong — just bounded.

Separately, the truncation branch at src/text_embedder_tokenizer.cpp:86 reads
fairseq_tokens_to_ids_["<eos>"], but that map holds </s>, not <eos>.
operator[] inserts the missing key with value 0, so a truncated input ends in
token 0 (<s>) instead of 2 (</s>). Worth a separate fix in the server repo.