|
Download README.md from mentisialog/multilingual-e5-small-ort: direct link, hf CLI and curl.
- Browser
- Download file 3.49 kB
-
https://huggingface.co/mentisialog/multilingual-e5-small-ort/resolve/main/README.md
- Command line
-
hf download hf://mentisialog/multilingual-e5-small-ort/README.md
-
curl -L -o README.md https://huggingface.co/mentisialog/multilingual-e5-small-ort/resolve/main/README.md
3.49 kB
| license: mit | |
| library_name: onnxruntime | |
| base_model: intfloat/multilingual-e5-small | |
| tags: | |
| - onnxruntime | |
| - ort-format | |
| - sentence-embeddings | |
| - e5 | |
| - multilingual | |
| - ios | |
| # multilingual-e5-small in ORT format (memory-mappable) | |
| The weights of [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small) | |
| in the [ONNX Runtime ORT format](https://onnxruntime.ai/docs/performance/model-optimizations/ort-format-models.html), | |
| for **Memoa for iOS**. Loaded with its initialisers memory-mapped, it embeds with a fraction of the | |
| memory the ONNX file needs, and gives **bit-identical** vectors. | |
| | | | | |
| |---|---| | |
| | Source | `intfloat/multilingual-e5-small`, revision `614241f622f53c4eeff9890bdc4f31cfecc418b3`, `onnx/model.onnx` (470,268,510 bytes, SHA-256 `ca456c06b3a9505ddfd9131408916dd79290368331e7d76bb621f1cba6bc8665`) | | |
| | File | `model.ort`: 470,350,304 bytes, SHA-256 `ba3519110cbb648984f04415ebc6e32946de1c55ac85afbe3719864a76b4ef29` | | |
| | Converter | ONNX Runtime **1.31.0** (Python wheel, installed by hash), macOS arm64 | | |
| | Session options | CPU execution provider only; intra-op and inter-op threads 2; `GraphOptimizationLevel.ORT_ENABLE_ALL`; `optimized_model_filepath` with `session.save_model_format=ORT` | | |
| | Script | `scripts/convert-e5-ort.sh` in Memoa for iOS (`mentisia-memoa-ios`) | | |
| ## What it is | |
| It is the same graph ONNX Runtime 1.31.0 builds from the source `model.onnx` at every load with | |
| these options, saved once. Nothing is retrained, quantised or pruned; the weights are unchanged | |
| (float32). | |
| To load it with the initialisers mapped from the file, use ONNX Runtime 1.31.0 (CPU) and set | |
| `session.load_model_format=ORT`, `session.use_memory_mapped_ort_model=1` and | |
| `session.use_ort_model_bytes_for_initializers=1`. | |
| ONNX Runtime warns that a model optimised at `ORT_ENABLE_ALL` may hold hardware-specific layout | |
| transforms (NCHWc, an x86 convolution layout). e5 has no convolutions, and the vectors below were | |
| checked on arm64 (an M1 Mac and an A16 iPad). Two conversions differ only in about 1 KB of | |
| flatbuffer metadata, with the same vectors. This file is the one Memoa pins. | |
| ## Vectors | |
| The 33 reference texts of Memoa's e5 parity fixture (25 passages, 8 queries, made with memoryd's | |
| fastembed / ONNX Runtime 1.31.0 stack) were embedded from this file, with mapped initialisers, and | |
| from the source `model.onnx`: | |
| - every vector is **bit-identical** to the source model's (M1 Mac: this file; iPad A16: a conversion | |
| of the same source by the same ORT version); | |
| - max |difference| to the reference vectors 9.69e-08, int8 codes (× 640) identical. | |
| On an iPad (A16), the load's peak memory footprint was **264 MB with this file**, against 1,351 MB | |
| for `model.onnx` read from its path. The load took 0.64 s instead of 1.03–1.23 s. | |
| ## Attribution and citation | |
| The model is by **intfloat** (Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, | |
| Furu Wei); see the [original model card](https://huggingface.co/intfloat/multilingual-e5-small) | |
| for training data, intended use, and limitations, which all apply here unchanged. | |
| ``` | |
| @article{wang2024multilingual, | |
| title={Multilingual E5 Text Embeddings: A Technical Report}, | |
| author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu}, | |
| journal={arXiv preprint arXiv:2402.05672}, | |
| year={2024} | |
| } | |
| ``` | |
| As with the original, prefix inputs with `query: ` or `passage: `. | |
| Licence: MIT, as the original (see `LICENSE`). | |