mensialog's picture
Upload folder using huggingface_hub
25fa072 verified
|
Raw History Blame Contribute Delete
3.49 kB
---
license: mit
library_name: onnxruntime
base_model: intfloat/multilingual-e5-small
tags:
- onnxruntime
- ort-format
- sentence-embeddings
- e5
- multilingual
- ios
---
# multilingual-e5-small in ORT format (memory-mappable)
The weights of [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small)
in the [ONNX Runtime ORT format](https://onnxruntime.ai/docs/performance/model-optimizations/ort-format-models.html),
for **Memoa for iOS**. Loaded with its initialisers memory-mapped, it embeds with a fraction of the
memory the ONNX file needs, and gives **bit-identical** vectors.
| | |
|---|---|
| Source | `intfloat/multilingual-e5-small`, revision `614241f622f53c4eeff9890bdc4f31cfecc418b3`, `onnx/model.onnx` (470,268,510 bytes, SHA-256 `ca456c06b3a9505ddfd9131408916dd79290368331e7d76bb621f1cba6bc8665`) |
| File | `model.ort`: 470,350,304 bytes, SHA-256 `ba3519110cbb648984f04415ebc6e32946de1c55ac85afbe3719864a76b4ef29` |
| Converter | ONNX Runtime **1.31.0** (Python wheel, installed by hash), macOS arm64 |
| Session options | CPU execution provider only; intra-op and inter-op threads 2; `GraphOptimizationLevel.ORT_ENABLE_ALL`; `optimized_model_filepath` with `session.save_model_format=ORT` |
| Script | `scripts/convert-e5-ort.sh` in Memoa for iOS (`mentisia-memoa-ios`) |
## What it is
It is the same graph ONNX Runtime 1.31.0 builds from the source `model.onnx` at every load with
these options, saved once. Nothing is retrained, quantised or pruned; the weights are unchanged
(float32).
To load it with the initialisers mapped from the file, use ONNX Runtime 1.31.0 (CPU) and set
`session.load_model_format=ORT`, `session.use_memory_mapped_ort_model=1` and
`session.use_ort_model_bytes_for_initializers=1`.
ONNX Runtime warns that a model optimised at `ORT_ENABLE_ALL` may hold hardware-specific layout
transforms (NCHWc, an x86 convolution layout). e5 has no convolutions, and the vectors below were
checked on arm64 (an M1 Mac and an A16 iPad). Two conversions differ only in about 1 KB of
flatbuffer metadata, with the same vectors. This file is the one Memoa pins.
## Vectors
The 33 reference texts of Memoa's e5 parity fixture (25 passages, 8 queries, made with memoryd's
fastembed / ONNX Runtime 1.31.0 stack) were embedded from this file, with mapped initialisers, and
from the source `model.onnx`:
- every vector is **bit-identical** to the source model's (M1 Mac: this file; iPad A16: a conversion
of the same source by the same ORT version);
- max |difference| to the reference vectors 9.69e-08, int8 codes (× 640) identical.
On an iPad (A16), the load's peak memory footprint was **264 MB with this file**, against 1,351 MB
for `model.onnx` read from its path. The load took 0.64 s instead of 1.03–1.23 s.
## Attribution and citation
The model is by **intfloat** (Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder,
Furu Wei); see the [original model card](https://huggingface.co/intfloat/multilingual-e5-small)
for training data, intended use, and limitations, which all apply here unchanged.
```
@article{wang2024multilingual,
title={Multilingual E5 Text Embeddings: A Technical Report},
author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
journal={arXiv preprint arXiv:2402.05672},
year={2024}
}
```
As with the original, prefix inputs with `query: ` or `passage: `.
Licence: MIT, as the original (see `LICENSE`).