Pringled commited on
Commit
0691906
·
verified ·
1 Parent(s): 641201e

Update model card

Browse files
Files changed (1) hide show
  1. README.md +26 -5
README.md CHANGED
@@ -49,6 +49,24 @@ Model2vec creates a small, static model that outperforms other static embedding
49
 
50
  The results for this model can be found on the [Model2Vec results page](https://github.com/MinishLab/model2vec/blob/main/results/README.md).
51
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
  ## Additional Resources
53
 
54
  - [All Model2Vec models on the hub](https://huggingface.co/models?library=model2vec)
@@ -64,11 +82,14 @@ Model2Vec was developed by the [Minish Lab](https://github.com/MinishLab) team c
64
  ## Citation
65
 
66
  Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
67
- ```
68
  @software{minishlab2024model2vec,
69
- authors = {Stephan Tulkens, Thomas van Dongen},
70
- title = {Model2Vec: Turn any Sentence Transformer into a Small Fast Model},
71
- year = {2024},
72
- url = {https://github.com/MinishLab/model2vec},
 
 
 
73
  }
74
  ```
 
49
 
50
  The results for this model can be found on the [Model2Vec results page](https://github.com/MinishLab/model2vec/blob/main/results/README.md).
51
 
52
+
53
+ ## Model Performance
54
+
55
+ | Model | Avg (All) | Avg (MTEB) | Class | Clust | PairClass | Rank | Ret | STS | Sum | Pearl | WordSim |
56
+ |:---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
57
+ | [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 55.80 | 55.93 | 69.25 | 44.90 | 82.37 | 47.14 | 42.92 | 78.95 | 25.96 | 60.83 | 49.91 |
58
+ | [potion-base-32M](https://huggingface.co/minishlab/potion-base-32M) | 52.83 | 52.13 | 71.70 | 41.25 | 78.17 | 42.45 | 32.67 | 73.93 | 24.74 | 55.37 | 55.15 |
59
+ | **[potion-base-8M](https://huggingface.co/minishlab/potion-base-8M)** | **51.32** | **51.08** | **70.34** | **39.74** | **76.62** | **41.79** | **31.11** | **72.91** | **25.06** | **53.54** | **50.75** |
60
+ | [potion-base-4M](https://huggingface.co/minishlab/potion-base-4M) | 50.01 | 49.77 | 68.00 | 39.47 | 75.37 | 41.41 | 28.43 | 71.87 | 23.82 | 52.55 | 49.21 |
61
+ | [potion-base-2M](https://huggingface.co/minishlab/potion-base-2M) | 47.55 | 47.49 | 64.13 | 37.53 | 73.72 | 40.46 | 22.99 | 69.77 | 23.80 | 50.82 | 44.72 |
62
+ | [M2V_base_output](https://huggingface.co/minishlab/M2V_base_output) | 48.77 | 47.96 | 66.84 | 33.96 | 74.90 | 39.31 | 25.36 | 68.76 | 26.61 | 54.02 | 49.18 |
63
+ | [GloVe_300d](https://huggingface.co/sentence-transformers/average_word_embeddings_glove.6B.300d) | 45.49 | 45.82 | 62.73 | 37.10 | 72.48 | 38.28 | 21.80 | 61.52 | 26.81 | 45.65 | 43.05 |
64
+ | [BPEmb_50k_300d](https://github.com/bheinzerling/bpemb) | 42.33 | 41.74 | 61.72 | 35.17 | 57.86 | 37.26 | 15.36 | 55.30 | 29.49 | 47.56 | 41.28 |
65
+
66
+ The results show that **potion-base-8M** reaches 91.96% of the performance of [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) with an average score of 51.32 while being orders of magnitude faster.
67
+
68
+ For full results, see the [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard).
69
+
70
  ## Additional Resources
71
 
72
  - [All Model2Vec models on the hub](https://huggingface.co/models?library=model2vec)
 
82
  ## Citation
83
 
84
  Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
85
+ ```bibtex
86
  @software{minishlab2024model2vec,
87
+ author = {Stephan Tulkens and {van Dongen}, Thomas},
88
+ title = {Model2Vec: Fast State-of-the-Art Static Embeddings},
89
+ year = {2024},
90
+ publisher = {Zenodo},
91
+ doi = {10.5281/zenodo.17270888},
92
+ url = {https://github.com/MinishLab/model2vec},
93
+ license = {MIT}
94
  }
95
  ```