Feature Extraction
sentence-transformers
Safetensors
Chinese
embedding_gemma2
embeddinggemma
embedding
chinese
c-mteb
mteb
Instructions to use Weidows/embeddinggemma-2-zh with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use Weidows/embeddinggemma-2-zh with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("Weidows/embeddinggemma-2-zh") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
|
Download eval/comparison.md from Weidows/embeddinggemma-2-zh: direct link, hf CLI and curl.
- Browser
- Download file 5.69 kB
-
https://huggingface.co/Weidows/embeddinggemma-2-zh/resolve/main/eval/comparison.md
- Command line
-
hf download hf://Weidows/embeddinggemma-2-zh/eval/comparison.md
-
curl -L -o comparison.md https://huggingface.co/Weidows/embeddinggemma-2-zh/resolve/main/eval/comparison.md
5.69 kB
| # C-MTEB (MTEB(cmn, v1)) — embeddinggemma-2 vs bge-m3 vs Qwen3-Embedding-0.6B | |
| Every column uses **the same harness** (mteb 2.24.0, `MTEB(cmn, v1)`, 31 tasks). | |
| `local run` = executed on this machine (RTX 5070 Ti, bf16, truncate 2048, batch 16); | |
| other columns are the published `mteb/results` entries for the same task names. | |
| | column | tasks covered | source | | |
| |---|---|---| | |
| | embeddinggemma-2 (local run) | 31/31 | local run | | |
| | bge-m3 (local run) | 31/31 | local run | | |
| | embeddinggemma-2-zh (LoRA r32, local run) | 31/31 | local run | | |
| | finetuned_weak (local run) | 8/31 | local run | | |
| | BAAI__bge-m3 | 5/31 | mteb/results (public) | | |
| | google__embeddinggemma-300m | 3/31 | mteb/results (public) | | |
| | Qwen__Qwen3-Embedding-0.6B | 31/31 | mteb/results (public) | | |
| ## Full 31-task table | |
| | task | type | embeddinggemma-2 (local run) | bge-m3 (local run) | embeddinggemma-2-zh (LoRA r32, local run) | Qwen__Qwen3-Embedding-0.6B | | |
| |---|---|---|---|---|---| | |
| | T2Retrieval | Retrieval | 0.7618 | 0.8149 | 0.7292 | 0.8368 | | |
| | MMarcoRetrieval | Retrieval | 0.7825 | 0.7726 | 0.6997 | 0.7985 | | |
| | DuRetrieval | Retrieval | 0.7637 | 0.8399 | 0.7243 | 0.8410 | | |
| | CovidRetrieval | Retrieval | 0.7721 | 0.7751 | 0.6433 | 0.8476 | | |
| | CmedqaRetrieval | Retrieval | 0.2451 | 0.3374 | 0.3361 | 0.4109 | | |
| | EcomRetrieval | Retrieval | 0.6040 | 0.5845 | 0.5532 | 0.6432 | | |
| | MedicalRetrieval | Retrieval | 0.4335 | 0.5427 | 0.4726 | 0.5604 | | |
| | VideoRetrieval | Retrieval | 0.5808 | 0.5699 | 0.5557 | 0.7436 | | |
| | T2Reranking | Reranking | 0.6613 | 0.6688 | 0.6587 | 0.6715 | | |
| | MMarcoReranking | Reranking | 0.2511 | 0.2754 | 0.2088 | 0.2175 | | |
| | CMedQAv1-reranking | Reranking | 0.6650 | 0.7769 | 0.8003 | 0.8006 | | |
| | CMedQAv2-reranking | Reranking | 0.6882 | 0.7909 | 0.8200 | 0.8135 | | |
| | Ocnli | PairClassification | 0.6297 | 0.6508 | 0.5826 | 0.7298 | | |
| | Cmnli | PairClassification | 0.7167 | 0.7183 | 0.6672 | 0.7986 | | |
| | CLSClusteringS2S | Clustering | 0.3743 | 0.3827 | 0.3767 | 0.5828 | | |
| | CLSClusteringP2P | Clustering | 0.3844 | 0.3899 | 0.4068 | 0.6062 | | |
| | ThuNewsClusteringS2S | Clustering | 0.5803 | 0.5107 | 0.5628 | 0.7457 | | |
| | ThuNewsClusteringP2P | Clustering | 0.6311 | 0.5627 | 0.6383 | 0.8150 | | |
| | LCQMC | STS | 0.7253 | 0.7618 | 0.7251 | 0.7646 | | |
| | PAWSX | STS | 0.1497 | 0.1566 | 0.1423 | 0.2763 | | |
| | AFQMC | STS | 0.2783 | 0.4000 | 0.3685 | 0.4323 | | |
| | QBQTC | STS | 0.3372 | 0.3331 | 0.3006 | 0.3707 | | |
| | ATEC | STS | 0.3447 | 0.4647 | 0.4344 | 0.4858 | | |
| | BQ | STS | 0.4731 | 0.5739 | 0.6396 | 0.6466 | | |
| | STSB | STS | 0.8002 | 0.8060 | 0.7866 | 0.8462 | | |
| | TNews | Classification | 0.4673 | 0.4961 | 0.4903 | 0.5296 | | |
| | IFlyTek | Classification | 0.4302 | 0.4768 | 0.4948 | 0.5167 | | |
| | Waimai | Classification | 0.8499 | 0.8665 | 0.8481 | 0.8129 | | |
| | OnlineShopping | Classification | 0.8991 | 0.9206 | 0.8811 | 0.8360 | | |
| | JDReview | Classification | 0.7964 | 0.8008 | 0.7587 | 0.8041 | | |
| | MultilingualSentiment | Classification | 0.7255 | 0.7168 | 0.6868 | 0.7897 | | |
| ## Aggregate | |
| | | embeddinggemma-2 (local run) | bge-m3 (local run) | embeddinggemma-2-zh (LoRA r32, local run) | Qwen__Qwen3-Embedding-0.6B | | |
| |---|---|---|---|---| | |
| | Retrieval | 0.6179 | 0.6546 | 0.5893 | 0.7103 | | |
| | Reranking | 0.5664 | 0.6280 | 0.6219 | 0.6258 | | |
| | PairClassification | 0.6732 | 0.6846 | 0.6249 | 0.7642 | | |
| | Clustering | 0.4925 | 0.4615 | 0.4962 | 0.6874 | | |
| | STS | 0.4441 | 0.4994 | 0.4853 | 0.5461 | | |
| | Classification | 0.6948 | 0.7129 | 0.6933 | 0.7149 | | |
| | **Overall (mean of type means)** | **0.5815** | **0.6068** | **0.5851** | **0.6748** | | |
| ## Delta vs `embeddinggemma-2 (local run)` (positive = ours better) | |
| | task | type | bge-m3 (local run) | embeddinggemma-2-zh (LoRA r32, local run) | Qwen__Qwen3-Embedding-0.6B | | |
| |---|---|---|---|---| | |
| | T2Retrieval | Retrieval | -0.0532 | +0.0326 | -0.0750 | | |
| | MMarcoRetrieval | Retrieval | +0.0099 | +0.0829 | -0.0160 | | |
| | DuRetrieval | Retrieval | -0.0762 | +0.0393 | -0.0773 | | |
| | CovidRetrieval | Retrieval | -0.0029 | +0.1288 | -0.0755 | | |
| | CmedqaRetrieval | Retrieval | -0.0923 | -0.0910 | -0.1658 | | |
| | EcomRetrieval | Retrieval | +0.0196 | +0.0508 | -0.0391 | | |
| | MedicalRetrieval | Retrieval | -0.1092 | -0.0391 | -0.1269 | | |
| | VideoRetrieval | Retrieval | +0.0108 | +0.0251 | -0.1629 | | |
| | T2Reranking | Reranking | -0.0075 | +0.0026 | -0.0102 | | |
| | MMarcoReranking | Reranking | -0.0243 | +0.0423 | +0.0336 | | |
| | CMedQAv1-reranking | Reranking | -0.1120 | -0.1353 | -0.1356 | | |
| | CMedQAv2-reranking | Reranking | -0.1026 | -0.1317 | -0.1253 | | |
| | Ocnli | PairClassification | -0.0211 | +0.0471 | -0.1002 | | |
| | Cmnli | PairClassification | -0.0017 | +0.0494 | -0.0819 | | |
| | CLSClusteringS2S | Clustering | -0.0084 | -0.0024 | -0.2086 | | |
| | CLSClusteringP2P | Clustering | -0.0055 | -0.0224 | -0.2218 | | |
| | ThuNewsClusteringS2S | Clustering | +0.0695 | +0.0174 | -0.1654 | | |
| | ThuNewsClusteringP2P | Clustering | +0.0684 | -0.0072 | -0.1839 | | |
| | LCQMC | STS | -0.0365 | +0.0002 | -0.0393 | | |
| | PAWSX | STS | -0.0069 | +0.0074 | -0.1266 | | |
| | AFQMC | STS | -0.1218 | -0.0902 | -0.1540 | | |
| | QBQTC | STS | +0.0042 | +0.0366 | -0.0335 | | |
| | ATEC | STS | -0.1200 | -0.0897 | -0.1411 | | |
| | BQ | STS | -0.1008 | -0.1665 | -0.1735 | | |
| | STSB | STS | -0.0059 | +0.0136 | -0.0460 | | |
| | TNews | Classification | -0.0288 | -0.0230 | -0.0623 | | |
| | IFlyTek | Classification | -0.0466 | -0.0646 | -0.0865 | | |
| | Waimai | Classification | -0.0166 | +0.0018 | +0.0370 | | |
| | OnlineShopping | Classification | -0.0215 | +0.0180 | +0.0631 | | |
| | JDReview | Classification | -0.0043 | +0.0377 | -0.0077 | | |
| | MultilingualSentiment | Classification | +0.0088 | +0.0387 | -0.0642 | | |
| | **Overall** | | **-0.0254** | **-0.0037** | **-0.0933** | | |