multilingual-e5-small → Core ML (fp16, enumerated shapes, pooling inside)

1626a6b verified about 1 month ago

1.9 kB

license: mit
base_model: intfloat/multilingual-e5-small
language:
  - ru
  - en
  - multilingual
tags:
  - coreml
  - sentence-embeddings
  - e5
library_name: coreml
pipeline_tag: feature-extraction

multilingual-e5-small · Core ML

intfloat/multilingual-e5-small converted to a Core ML ML Program (fp16) for on-device semantic search in the DND Master app (iPad / iPhone / Mac).

What is inside

File	Purpose
`MultilingualE5Small.mlpackage`	Core ML model. Inputs `input_ids`, `attention_mask` (`int32`, shape `[1, L]`, L ∈ {16, 32, 64, 128, 256, 512}); output `embedding` `[1, 384]` — masked mean pooling and L2 normalisation are inside the graph.
`tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`, `config.json`	XLM-R tokenizer files from the upstream repo, unchanged.

Pad token id is 1 (<pad>), attention mask is 1 for real tokens. Use the e5 prefixes: query: for queries, passage: for documents.

Conversion notes

coremltools 9.0, torch 2.7.0, transformers 4.46.3, minimum_deployment_target = iOS18.
Enumerated input shapes instead of a RangeDim: with a flexible range Core ML returned NaN (“Data-dependent shapes were disabled”).
The additive attention mask was replaced by -1e4 before tracing: the default finfo(float32).min overflows to -inf in fp16 and the softmax produces NaN.
Parity with the PyTorch reference: cosine 1.000000 on test sentences; parity with an MLX fp16 embedding of a 12 270-chunk Russian corpus: min cosine 0.9997.
Recommended compute units: .cpuAndNeuralEngine. The GPU path is ~25× slower for this graph on Apple silicon.

The conversion script lives in the app repository (BestiaryEmbedder/convert-e5-coreml.py).

License

The weights are derived from intfloat/multilingual-e5-small (MIT).