---
license: apache-2.0
base_model: google/embeddinggemma-2
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags: [sentence-transformers, embedding, multilingual, turkish, retrieval, audio, image, norovox]
---

# Nythos Embed Zero

![banner](assets/banner.webp)

Nythos Embed Zero is a multilingual embedding model from Norovox Labs, from the Nythos open-base line (its own-architecture counterpart is Norovox Embed Zero). It covers text, Turkish, retrieval, classification, clustering, audio and images, and is built on EmbeddingGemma 2. The text path has 271M parameters.

> Every number below was measured by us under one protocol: fp32, the same prompts, and the same test sets for both models. Each test set was run once, after model selection on separate development sets.

## Results vs. EmbeddingGemma 2: ahead on 20 of 22 measures, tied on 1, behind on 1

| Test | EmbeddingGemma 2 | Nythos Embed Zero |
|---|---|---|
| STS-B (English similarity, Spearman) | .8749 | **.8865** |
| STS-TR (Turkish similarity, Spearman) | .7863 | **.8091** |
| WikiRAG-TR (Turkish retrieval, nDCG@10) | .4726 | **.4837** |
| SciFact (scientific retrieval, nDCG@10) | .8645 | **.8714** |
| NFCorpus (medical retrieval, nDCG@10) | .3608 | **.3913** |
| Flickr30k text→image R@1 / R@5 / R@10 | .8676 / .9788 / .9882 | **.8714 / .9818 / .9896** |
| Flickr30k image→text R@1 | .954 | **.956** |
| Flickr30k image→text R@5 / R@10 | 1.000 / 1.000 | .999 / 1.000 |
| AudioCaps text→audio R@1 / R@5 / R@10 | .068 / .250 / .362 | **.121 / .413 / .580** |

Paired bootstrap per query, 95% interval of the difference:
- WikiRAG-TR: +.0111 [+.0072, +.0150] (n = 4,999)
- NFCorpus: +.0305 [+.0203, +.0414] (n = 323)
- SciFact: +.0070 [−.0033, +.0176] (n = 300; not significant)

### Classification (logistic regression on embeddings) and clustering (V-measure): ahead on 8 of 8

| Task | EmbeddingGemma 2 | Nythos Embed Zero |
|---|---|---|
| Banking77 | .9293 | **.9357** |
| Emotion | .7009 | **.7190** |
| Amazon counterfactual (en) | .9194 | **.9224** |
| Tweet sentiment | .7230 | **.7253** |
| IMDB | .9230 | **.9240** |
| TwentyNewsgroups clustering | .4710 | **.4850** |
| Reddit clustering | .5579 | **.5874** |
| StackExchange clustering | .6732 | **.6772** |

The only measure where it is behind: image→text R@5 is .999 vs 1.000, i.e. one image out of 1,000.

### Smaller vectors (Matryoshka truncation; measured on the previous text mixture, α = 0.6)
| Test | EG2 768-d | Zero 512-d | EG2 256-d | Zero 256-d |
|---|---|---|---|---|
| STS-B | .8749 | **.8873** | .8738 | **.8844** |
| STS-TR | .7863 | **.8066** | .7909 | **.8083** |
| WikiRAG-TR | .4726 | .4708 | .4271 | **.4404** |
| SciFact | .8645 | **.8701** | .8418 | **.8548** |
| NFCorpus | .3608 | **.3878** | .3432 | **.3793** |

### Against larger open embedding models (same five text tests, each with its own prompts; previous text mixture, α = 0.6)
| Test | Nythos Embed Zero (271M) | Qwen3-Embedding-0.6B (596M) | multilingual-e5-large-instruct (560M) | BGE-M3 (568M) |
|---|---|---|---|---|
| STS-B | .8873 | **.9113** | .8838 | .8485 |
| STS-TR | .8067 | .7501 | **.8296** | .7978 |
| WikiRAG-TR | **.4804** | .3551 | .3659 | .4260 |
| SciFact | **.8729** | .7023 | .7118 | .6437 |
| NFCorpus | **.3913** | .3638 | .3589 | .3157 |

Note: our training data includes scientific and medical retrieval pairs, which helps on SciFact and NFCorpus.

## Fast indexing with the Kapsül engine
Kapsül is our inference layer on top of vLLM (pooling runner, bf16, length-aware batching). We measured it on 20,000 web documents on one H100, with the GPU shared with another training job; both paths ran under the same conditions.

| Path | Documents / second | Same vectors? |
|---|---|---|
| SentenceTransformers (fp32, default) | 69 | reference |
| **Kapsül** | **554 (8.0×)** | cosine to reference: mean .99998, min .99992 |

```python
from vllm import LLM
llm = LLM(model="emrevrg/Nythos-Embed-Zero", runner="pooling", dtype="bfloat16")
docs = ["title: none | text: " + d for d in documents]   # same prompt as prompt_name="Document"
vectors = [o.outputs.embedding for o in llm.embed(docs)]
```

What we tried and dropped: a 12-layer shallow document encoder distilled from the full model was 1.84× faster but lost too much retrieval quality (WikiRAG-TR dev .258 vs .516), so it is not part of this release.

## How it was made
- **Verified distillation:** Two teachers, EmbeddingGemma 2 and Qwen3-Embedding-8B. For each example, only a teacher that ranks the gold answer correctly is allowed to teach.
- **Our own synthetic data:**
  - Turkish search queries for Wikipedia passages (40,000 pairs).
  - StackExchange-style question titles (for clustering).
  - Both were generated with our Kapsül speed engine and kept only if the query retrieves its own passage first. Passages overlapping the test sets were removed with 10-gram filtering.
- **Weight soup + mixing:** Two training runs were averaged and mixed with the base model (α = 0.7). Alpha was chosen only on 8 development slices: similarity, retrieval, Turkish, clustering and classification.
- **Audio and image:** The text path is frozen; only the modality layers are trained. The image layers use relational distillation from EmbeddingGemma 2 on Flickr30k images that do not overlap the test set.

## Usage
```python
from sentence_transformers import SentenceTransformer
m = SentenceTransformer("emrevrg/Nythos-Embed-Zero")
q = m.encode(["What is the longest river in Turkey?"], prompt_name="SearchQuery")
d = m.encode(["The Kızılırmak, at 1,355 km, is the longest river in Turkey."], prompt_name="Document")
print(q @ d.T)
```

## Support, acknowledgements and the goal behind this work
This model was trained and measured on one week of NVIDIA H100 time provided by **portvMind (vMind) Public Cloud**. Thank you, vMind: without that week this release would not exist.

I'm an independent developer in Türkiye and I build the Norovox ecosystem alone, without a company. Most compute and hardware programs require a registered company or a minimum age. I have sent close to 600 applications; vMind is the only company that said yes, and the others are still pending or declined on company or age grounds. Day to day I work on a laptop whose disk keeps filling up and that cannot run large models locally, and I regularly hit the usage limits of the AI tools I depend on.

The goal is an open ecosystem: by the end of the year, my own programming language, operating system, office apps, and agent and coding tools, built with stronger AI and released openly. The techniques behind these models (IRIS lossless model compression, Aktar encrypted transfer, Sıkıştır) are part of the same effort.

### How you can help
- **Companies:** if you can support open models with compute, hardware or credits, write to **emre.games72@gmail.com**.
- **AI companies** such as Anthropic, OpenAI and Google: access to your models and API credits would let me keep building and releasing these tools.
- **Everyone else:** if you work at one of these companies, or know someone who does, please share this page with them or introduce me. One introduction can make the difference for an independent project like this.

Every supporter is credited in the model and technical cards of the projects they help.

## License and attribution
Apache-2.0. This is a derivative of google/embeddinggemma-2 (Apache-2.0); see the NOTICE file. The Qwen3-Embedding-8B teacher is also Apache-2.0.