mensialog commited on
Commit
25fa072
·
verified ·
1 Parent(s): 2e34c1b

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. .gitattributes +1 -35
  2. LICENSE +29 -0
  3. README.md +74 -0
  4. model.ort +3 -0
.gitattributes CHANGED
@@ -1,35 +1 @@
1
- *.7z filter=lfs diff=lfs merge=lfs -text
2
- *.arrow filter=lfs diff=lfs merge=lfs -text
3
- *.bin filter=lfs diff=lfs merge=lfs -text
4
- *.bz2 filter=lfs diff=lfs merge=lfs -text
5
- *.ckpt filter=lfs diff=lfs merge=lfs -text
6
- *.ftz filter=lfs diff=lfs merge=lfs -text
7
- *.gz filter=lfs diff=lfs merge=lfs -text
8
- *.h5 filter=lfs diff=lfs merge=lfs -text
9
- *.joblib filter=lfs diff=lfs merge=lfs -text
10
- *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
- *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
- *.model filter=lfs diff=lfs merge=lfs -text
13
- *.msgpack filter=lfs diff=lfs merge=lfs -text
14
- *.npy filter=lfs diff=lfs merge=lfs -text
15
- *.npz filter=lfs diff=lfs merge=lfs -text
16
- *.onnx filter=lfs diff=lfs merge=lfs -text
17
- *.ot filter=lfs diff=lfs merge=lfs -text
18
- *.parquet filter=lfs diff=lfs merge=lfs -text
19
- *.pb filter=lfs diff=lfs merge=lfs -text
20
- *.pickle filter=lfs diff=lfs merge=lfs -text
21
- *.pkl filter=lfs diff=lfs merge=lfs -text
22
- *.pt filter=lfs diff=lfs merge=lfs -text
23
- *.pth filter=lfs diff=lfs merge=lfs -text
24
- *.rar filter=lfs diff=lfs merge=lfs -text
25
- *.safetensors filter=lfs diff=lfs merge=lfs -text
26
- saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
- *.tar.* filter=lfs diff=lfs merge=lfs -text
28
- *.tar filter=lfs diff=lfs merge=lfs -text
29
- *.tflite filter=lfs diff=lfs merge=lfs -text
30
- *.tgz filter=lfs diff=lfs merge=lfs -text
31
- *.wasm filter=lfs diff=lfs merge=lfs -text
32
- *.xz filter=lfs diff=lfs merge=lfs -text
33
- *.zip filter=lfs diff=lfs merge=lfs -text
34
- *.zst filter=lfs diff=lfs merge=lfs -text
35
- *tfevents* filter=lfs diff=lfs merge=lfs -text
 
1
+ *.ort filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
LICENSE ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ The model weights in model.ort are those of intfloat/multilingual-e5-small
4
+ (https://huggingface.co/intfloat/multilingual-e5-small), revision
5
+ 614241f622f53c4eeff9890bdc4f31cfecc418b3, released under the MIT licence
6
+ (`license: mit` in that repository's model card; the repository carries no
7
+ separate LICENSE file at that revision, so the standard MIT licence text
8
+ follows). Converted to the ONNX Runtime (ORT) format by Mentisia.
9
+
10
+ Copyright (c) Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei
11
+ (intfloat, the authors of multilingual-e5-small)
12
+
13
+ Permission is hereby granted, free of charge, to any person obtaining a copy
14
+ of this software and associated documentation files (the "Software"), to deal
15
+ in the Software without restriction, including without limitation the rights
16
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
17
+ copies of the Software, and to permit persons to whom the Software is
18
+ furnished to do so, subject to the following conditions:
19
+
20
+ The above copyright notice and this permission notice shall be included in all
21
+ copies or substantial portions of the Software.
22
+
23
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
24
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
25
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
26
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
27
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
28
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
29
+ SOFTWARE.
README.md ADDED
@@ -0,0 +1,74 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: onnxruntime
4
+ base_model: intfloat/multilingual-e5-small
5
+ tags:
6
+ - onnxruntime
7
+ - ort-format
8
+ - sentence-embeddings
9
+ - e5
10
+ - multilingual
11
+ - ios
12
+ ---
13
+
14
+ # multilingual-e5-small in ORT format (memory-mappable)
15
+
16
+ The weights of [intfloat/multilingual-e5-small](https://huggingface.co/intfloat/multilingual-e5-small)
17
+ in the [ONNX Runtime ORT format](https://onnxruntime.ai/docs/performance/model-optimizations/ort-format-models.html),
18
+ for **Memoa for iOS**. Loaded with its initialisers memory-mapped, it embeds with a fraction of the
19
+ memory the ONNX file needs, and gives **bit-identical** vectors.
20
+
21
+ | | |
22
+ |---|---|
23
+ | Source | `intfloat/multilingual-e5-small`, revision `614241f622f53c4eeff9890bdc4f31cfecc418b3`, `onnx/model.onnx` (470,268,510 bytes, SHA-256 `ca456c06b3a9505ddfd9131408916dd79290368331e7d76bb621f1cba6bc8665`) |
24
+ | File | `model.ort`: 470,350,304 bytes, SHA-256 `ba3519110cbb648984f04415ebc6e32946de1c55ac85afbe3719864a76b4ef29` |
25
+ | Converter | ONNX Runtime **1.31.0** (Python wheel, installed by hash), macOS arm64 |
26
+ | Session options | CPU execution provider only; intra-op and inter-op threads 2; `GraphOptimizationLevel.ORT_ENABLE_ALL`; `optimized_model_filepath` with `session.save_model_format=ORT` |
27
+ | Script | `scripts/convert-e5-ort.sh` in Memoa for iOS (`mentisia-memoa-ios`) |
28
+
29
+ ## What it is
30
+
31
+ It is the same graph ONNX Runtime 1.31.0 builds from the source `model.onnx` at every load with
32
+ these options, saved once. Nothing is retrained, quantised or pruned; the weights are unchanged
33
+ (float32).
34
+
35
+ To load it with the initialisers mapped from the file, use ONNX Runtime 1.31.0 (CPU) and set
36
+ `session.load_model_format=ORT`, `session.use_memory_mapped_ort_model=1` and
37
+ `session.use_ort_model_bytes_for_initializers=1`.
38
+
39
+ ONNX Runtime warns that a model optimised at `ORT_ENABLE_ALL` may hold hardware-specific layout
40
+ transforms (NCHWc, an x86 convolution layout). e5 has no convolutions, and the vectors below were
41
+ checked on arm64 (an M1 Mac and an A16 iPad). Two conversions differ only in about 1 KB of
42
+ flatbuffer metadata, with the same vectors. This file is the one Memoa pins.
43
+
44
+ ## Vectors
45
+
46
+ The 33 reference texts of Memoa's e5 parity fixture (25 passages, 8 queries, made with memoryd's
47
+ fastembed / ONNX Runtime 1.31.0 stack) were embedded from this file, with mapped initialisers, and
48
+ from the source `model.onnx`:
49
+
50
+ - every vector is **bit-identical** to the source model's (M1 Mac: this file; iPad A16: a conversion
51
+ of the same source by the same ORT version);
52
+ - max |difference| to the reference vectors 9.69e-08, int8 codes (× 640) identical.
53
+
54
+ On an iPad (A16), the load's peak memory footprint was **264 MB with this file**, against 1,351 MB
55
+ for `model.onnx` read from its path. The load took 0.64 s instead of 1.03–1.23 s.
56
+
57
+ ## Attribution and citation
58
+
59
+ The model is by **intfloat** (Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder,
60
+ Furu Wei); see the [original model card](https://huggingface.co/intfloat/multilingual-e5-small)
61
+ for training data, intended use, and limitations, which all apply here unchanged.
62
+
63
+ ```
64
+ @article{wang2024multilingual,
65
+ title={Multilingual E5 Text Embeddings: A Technical Report},
66
+ author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
67
+ journal={arXiv preprint arXiv:2402.05672},
68
+ year={2024}
69
+ }
70
+ ```
71
+
72
+ As with the original, prefix inputs with `query: ` or `passage: `.
73
+
74
+ Licence: MIT, as the original (see `LICENSE`).
model.ort ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba3519110cbb648984f04415ebc6e32946de1c55ac85afbe3719864a76b4ef29
3
+ size 470350304