wolfram-pro/multilingual-e5-small-coreml

multilingual-e5-small · Core ML

intfloat/multilingual-e5-small converted to a Core ML ML Program (fp16) for on-device semantic search in the DND Master app (iPad / iPhone / Mac).

What is inside

File	Purpose
MultilingualE5Small.mlpackage	Core ML model. Inputs input_ids, attention_mask (int32, shape [1, L], L ∈ {16, 32, 64, 128, 256, 512}); output embedding [1, 384] — masked mean pooling and L2 normalisation are inside the graph.
tokenizer.json, tokenizer_config.json, special_tokens_map.json, config.json	XLM-R tokenizer files from the upstream repo, unchanged.

Pad token id is 1 (<pad>), attention mask is 1 for real tokens. Use the e5 prefixes: query: for queries, passage: for documents.

Conversion notes

• coremltools 9.0, torch 2.7.0, transformers 4.46.3, minimum_deployment_target = iOS18.
• Enumerated input shapes instead of a RangeDim: with a flexible range Core ML returned NaN (“Data-dependent shapes were disabled”).
• The additive attention mask was replaced by -1e4 before tracing: the default finfo(float32).min overflows to -inf in fp16 and the softmax produces NaN.
• Parity with the PyTorch reference: cosine 1.000000 on test sentences; parity with an MLX fp16 embedding of a 12 270-chunk Russian corpus: min cosine 0.9997.
• Recommended compute units: .cpuAndNeuralEngine. The GPU path is ~25× slower for this graph on Apple silicon.

The conversion script lives in the app repository (BestiaryEmbedder/convert-e5-coreml.py).

License

The weights are derived from intfloat/multilingual-e5-small (MIT).