multilingual-e5-small in ORT format (memory-mappable)
The weights of intfloat/multilingual-e5-small in the ONNX Runtime ORT format, for Memoa for iOS. Loaded with its initialisers memory-mapped, it embeds with a fraction of the memory the ONNX file needs, and gives bit-identical vectors.
| Source | intfloat/multilingual-e5-small, revision 614241f622f53c4eeff9890bdc4f31cfecc418b3, onnx/model.onnx (470,268,510 bytes, SHA-256 ca456c06b3a9505ddfd9131408916dd79290368331e7d76bb621f1cba6bc8665) |
| File | model.ort: 470,350,304 bytes, SHA-256 ba3519110cbb648984f04415ebc6e32946de1c55ac85afbe3719864a76b4ef29 |
| Converter | ONNX Runtime 1.31.0 (Python wheel, installed by hash), macOS arm64 |
| Session options | CPU execution provider only; intra-op and inter-op threads 2; GraphOptimizationLevel.ORT_ENABLE_ALL; optimized_model_filepath with session.save_model_format=ORT |
| Script | scripts/convert-e5-ort.sh in Memoa for iOS (mentisia-memoa-ios) |
What it is
It is the same graph ONNX Runtime 1.31.0 builds from the source model.onnx at every load with
these options, saved once. Nothing is retrained, quantised or pruned; the weights are unchanged
(float32).
To load it with the initialisers mapped from the file, use ONNX Runtime 1.31.0 (CPU) and set
session.load_model_format=ORT, session.use_memory_mapped_ort_model=1 and
session.use_ort_model_bytes_for_initializers=1.
ONNX Runtime warns that a model optimised at ORT_ENABLE_ALL may hold hardware-specific layout
transforms (NCHWc, an x86 convolution layout). e5 has no convolutions, and the vectors below were
checked on arm64 (an M1 Mac and an A16 iPad). Two conversions differ only in about 1 KB of
flatbuffer metadata, with the same vectors. This file is the one Memoa pins.
Vectors
The 33 reference texts of Memoa's e5 parity fixture (25 passages, 8 queries, made with memoryd's
fastembed / ONNX Runtime 1.31.0 stack) were embedded from this file, with mapped initialisers, and
from the source model.onnx:
- every vector is bit-identical to the source model's (M1 Mac: this file; iPad A16: a conversion of the same source by the same ORT version);
- max |difference| to the reference vectors 9.69e-08, int8 codes (Γ 640) identical.
On an iPad (A16), the load's peak memory footprint was 264 MB with this file, against 1,351 MB
for model.onnx read from its path. The load took 0.64 s instead of 1.03β1.23 s.
Attribution and citation
The model is by intfloat (Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei); see the original model card for training data, intended use, and limitations, which all apply here unchanged.
@article{wang2024multilingual,
title={Multilingual E5 Text Embeddings: A Technical Report},
author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
journal={arXiv preprint arXiv:2402.05672},
year={2024}
}
As with the original, prefix inputs with query: or passage: .
Licence: MIT, as the original (see LICENSE).
Model tree for mentisialog/multilingual-e5-small-ort
Base model
intfloat/multilingual-e5-small