---
license: apache-2.0
library_name: sentence-transformers
pipeline_tag: sentence-similarity
language:
  - multilingual
tags:
  - sentence-transformers
  - sentence-similarity
  - static-embeddings
  - multilingual
---

# Loss-Guided Static Multilingual Sentence Embeddings

## Overview

A static multilingual sentence encoder: one embedding table, mean pooling, no Transformer at inference. Its vocabulary was extended with **loss-guided vocabulary discovery**: during contrastive training, adjacent token pairs are scored by how much the sentence objective would suffer without them, and the useful pairs receive their own embedding rows. The base tokenizer stays unchanged. The model is trained on 176 languages and is intended for CPU-efficient semantic similarity, retrieval, clustering, and classification.

## Model Specifications

- **Embedding dimension**: 1,024
- **Base vocabulary**: 257,914 rows (mmBERT tokenizer + 1,914 language markers)
- **Learned pair rows**: 56,071
- **Parameters**: 321.7M (embedding table only), FP32 safetensors, 1.29 GB
- **Training languages**: 176
- **Inference**: table lookup and mean pooling; requires a language marker at the start of each text

## Performance

MTEB(Multilingual, v2), 131 tasks, primary score (Mean over tasks) **50.13**, Mean over task types 43.45:

| Task type | Count | Mean score |
|-----------|-------|-----------|
| Bitext mining | 13 | 53.90 |
| Classification | 43 | 52.87 |
| Clustering | 16 | 38.91 |
| Instruction reranking | 3 | -1.04 |
| Multilabel classification | 5 | 17.64 |
| Pair classification | 11 | 72.69 |
| Reranking | 6 | 49.42 |
| Retrieval | 18 | 41.64 |
| STS | 16 | 64.99 |

Cross-lingual retrieval (macro F1 over language pairs): Tatoeba 57.26 (106 pairs), BUCC 98.12 (4 pairs), Flores 63.70 (193 pairs).

## Installation & Usage

Every input must start with a language marker `__<iso639-3>_<script>__`, for example `__eng_Latn__`, `__rus_Cyrl__`, `__arb_Arab__`, `__cmn_Hani__`. A text without a marker raises an error. The repository ships the custom embedding module, so `trust_remote_code=True` is required.

```python
pip install -U sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BorisTM/loss-guided-static-multi", trust_remote_code=True)

sentences = [
    "__eng_Latn__ The cat sleeps on the sofa.",
    "__rus_Cyrl__ Кошка спит на диване.",
    "__deu_Latn__ Die Katze schläft auf dem Sofa.",
    "__eng_Latn__ The stock market fell sharply today.",
]

embeddings = model.encode(sentences, normalize_embeddings=True)
similarities = model.similarity(embeddings, embeddings)
# [[1.00, 0.44, 0.80, 0.05],
#  [0.44, 1.00, 0.41, 0.06],
#  [0.80, 0.41, 1.00, 0.08],
#  [0.05, 0.06, 0.08, 1.00]]
```

**License**: Apache 2.0