Pringled commited on
Commit
76c720a
·
verified ·
1 Parent(s): 41a72fb

Update model card

Browse files
Files changed (1) hide show
  1. README.md +26 -5
README.md CHANGED
@@ -48,6 +48,24 @@ Model2vec creates a small, static model that outperforms other static embedding
48
 
49
  The results for this model can be found on the [Model2Vec results page](https://github.com/MinishLab/model2vec/blob/main/results/README.md).
50
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
  ## Additional Resources
52
 
53
  - [All Model2Vec models on the hub](https://huggingface.co/models?library=model2vec)
@@ -63,11 +81,14 @@ Model2Vec was developed by the [Minish Lab](https://github.com/MinishLab) team c
63
  ## Citation
64
 
65
  Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
66
- ```
67
  @software{minishlab2024model2vec,
68
- authors = {Stephan Tulkens, Thomas van Dongen},
69
- title = {Model2Vec: Turn any Sentence Transformer into a Small Fast Model},
70
- year = {2024},
71
- url = {https://github.com/MinishLab/model2vec},
 
 
 
72
  }
73
  ```
 
48
 
49
  The results for this model can be found on the [Model2Vec results page](https://github.com/MinishLab/model2vec/blob/main/results/README.md).
50
 
51
+
52
+ ## Model Performance
53
+
54
+ | Model | Avg (All) | Avg (MTEB) | Class | Clust | PairClass | Rank | Ret | STS | Sum | Pearl | WordSim |
55
+ |:---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
56
+ | [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 55.80 | 55.93 | 69.25 | 44.90 | 82.37 | 47.14 | 42.92 | 78.95 | 25.96 | 60.83 | 49.91 |
57
+ | [potion-base-32M](https://huggingface.co/minishlab/potion-base-32M) | 52.83 | 52.13 | 71.70 | 41.25 | 78.17 | 42.45 | 32.67 | 73.93 | 24.74 | 55.37 | 55.15 |
58
+ | [potion-base-8M](https://huggingface.co/minishlab/potion-base-8M) | 51.32 | 51.08 | 70.34 | 39.74 | 76.62 | 41.79 | 31.11 | 72.91 | 25.06 | 53.54 | 50.75 |
59
+ | **[potion-base-4M](https://huggingface.co/minishlab/potion-base-4M)** | **50.01** | **49.77** | **68.00** | **39.47** | **75.37** | **41.41** | **28.43** | **71.87** | **23.82** | **52.55** | **49.21** |
60
+ | [potion-base-2M](https://huggingface.co/minishlab/potion-base-2M) | 47.55 | 47.49 | 64.13 | 37.53 | 73.72 | 40.46 | 22.99 | 69.77 | 23.80 | 50.82 | 44.72 |
61
+ | [M2V_base_output](https://huggingface.co/minishlab/M2V_base_output) | 48.77 | 47.96 | 66.84 | 33.96 | 74.90 | 39.31 | 25.36 | 68.76 | 26.61 | 54.02 | 49.18 |
62
+ | [GloVe_300d](https://huggingface.co/sentence-transformers/average_word_embeddings_glove.6B.300d) | 45.49 | 45.82 | 62.73 | 37.10 | 72.48 | 38.28 | 21.80 | 61.52 | 26.81 | 45.65 | 43.05 |
63
+ | [BPEmb_50k_300d](https://github.com/bheinzerling/bpemb) | 42.33 | 41.74 | 61.72 | 35.17 | 57.86 | 37.26 | 15.36 | 55.30 | 29.49 | 47.56 | 41.28 |
64
+
65
+ The results show that **potion-base-4M** reaches 89.63% of the performance of [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) with an average score of 50.01 while being orders of magnitude faster.
66
+
67
+ For full results, see the [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard).
68
+
69
  ## Additional Resources
70
 
71
  - [All Model2Vec models on the hub](https://huggingface.co/models?library=model2vec)
 
81
  ## Citation
82
 
83
  Please cite the [Model2Vec repository](https://github.com/MinishLab/model2vec) if you use this model in your work.
84
+ ```bibtex
85
  @software{minishlab2024model2vec,
86
+ author = {Stephan Tulkens and {van Dongen}, Thomas},
87
+ title = {Model2Vec: Fast State-of-the-Art Static Embeddings},
88
+ year = {2024},
89
+ publisher = {Zenodo},
90
+ doi = {10.5281/zenodo.17270888},
91
+ url = {https://github.com/MinishLab/model2vec},
92
+ license = {MIT}
93
  }
94
  ```