EmbeddingGemma 2 is a text-only embedding model developed by Google DeepMind. It maps text, including code, into a shared 768-dimensional vector space for semantic search, retrieval-augmented generation (RAG), classification, clustering, and semantic similarity.

Designed to run on consumer hardware such as mobile devices and laptops, EmbeddingGemma 2 delivers low-latency semantic representations for text-based applications.

EmbeddingGemma 2 builds upon the architectural and capability advancements of Gemma 4, offering several core features:

Text-only embeddings: Generates semantic representations of text and code for text-based retrieval and understanding tasks. It does not support image, video, or audio inputs.
Multilinguality and code: Supports 100+ languages and achieves an approximately 14% improvement on code tasks relative to its predecessor.
Efficient footprint: Uses a text backbone and embedder designed for efficient inference.
Matryoshka Representation Learning (MRL): Supports truncated embeddings across 128d, 256d, 512d, and 768d, enabling up to a 6x reduction in vector storage costs with minimal impact on quality.
Context length: Supports an 8K-token context window.
Task-steered representations: Uses lightweight text instruction prefixes to optimize embeddings for different tasks, including search, classification, clustering, and semantic similarity.

Model Overview

Component	Specification
Architecture	Gemma-based text embedding model
Backbone	130M parameters
Embedder	140M parameters
Text Model Parameters	270M
Layers	24
Model Dimension	512
Hidden Dimension	2048
Sliding Window	1024 tokens
Vocabulary Size	262,144
Number of Heads	4
Number of KV-Heads (Local/Global)	2/1
Local:Global	5:1
Attention	GQA/MQA
Activation	Gated FFN with GELU
Pooling	Mean Pooling
Projection Layer	512→768
Supported Input	Text and code
Context Window	8,192 tokens
Native Output Dimension	768
MRL Truncation Dimensions	128, 256, 512

Benchmark Results

EmbeddingGemma 2 was evaluated across text and code embedding benchmarks. Results reported below use the full-precision checkpoint.

Overall Evaluation Results (768d)

Modality	Benchmark	Metric	EmbeddingGemma 2	EmbeddingGemma 1
Text	Massive Text Embedding Benchmark (MTEB, multilingual, v2)	Mean(Task), Multiple	61.36	61.15
Code	Massive Text Embedding Benchmark (MTEB, code, v1)	Mean(Task), NDCG@10	78.68	68.76

Evaluation Results with Vector Truncation

With MRL, EmbeddingGemma 2 representations can be truncated below the native 768d to 128d, 256d, and 512d representations and re-normalized.

This reduces storage requirements, with minimal quality impact down to 256 dimensions. The 128-dimensional representation is best suited to text-only workloads.

Output Dimension	Compression Ratio	MTEB (multilingual, v2) Mean(Task)	MTEB (English, v2) Mean(Task)	MTEB (code, v1) Mean(Task)
768d (Full)	1:1	61.36	68.46	78.68
512d	1:1.5	61.17	68.41	77.24
256d	1:3	60.41	67.78	76.18
128d	1:6	57.89	65.68	71.41

Quick Start

Install the sentence-transformers library:

pip install -U sentence-transformers transformers

Generate text embeddings:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("google/embeddinggemma-2")

query = "What causes the northern lights?"
document = "The northern lights are caused by charged particles from the sun."

query_emb = model.encode(query, prompt_name="SearchQuery")
doc_emb = model.encode(document, prompt_name="Document")
print(model.similarity(query_emb, doc_emb))

Best Practices

For optimal embedding quality and runtime efficiency, follow these configurations and best practices.

1. Task Instruction Prefixes

EmbeddingGemma 2 is trained with short task instruction prefixes prepended to text inputs. Using the right prefix improves quality; omitting it still works but may reduce precision.

Documents with a real title should be formatted as title: {title} | text: {content}. Use title: none when no title is available.

Prefix Notation & Usage

There are two types of tasks:

Asymmetric tasks (e.g., retrieval): Use a query prefix for queries and a document prefix for corpus items.
Symmetric tasks (e.g., classification, similarity): Apply the same task prefix to all inputs being compared.

Use Case	Task Type	Prompt Name	Query Task Instruction	Document Task Instruction (use `none` if no title)
Web / document search	Asymmetric	SearchQuery	`task: search result \| query: {query}`	`title: {title} \| text: {content}`
Question answering	Asymmetric	QuestionAnswering	`task: question answering \| query: {question}`	`title: {title} \| text: {passage}`
Fact checking	Asymmetric	FactChecking	`task: fact checking \| query: {claim}`	`title: {title} \| text: {evidence}`
Code search	Asymmetric	CodeRetrieval	`task: code retrieval \| query: {query}`	`title: {title or filename} \| text: {code}`
Text classification	Symmetric	Classification	`task: classification \| query: {content}`	N/A
Clustering	Symmetric	Clustering	`task: clustering \| query: {content}`	N/A
Measuring similarity	Symmetric	SentenceSimilarity	`task: sentence similarity \| query: {content}`	N/A

Note: prompt_name="Document" applies title: none. Titled documents should still be formatted manually:

model.encode(f"title: {title} | text: {document}")

2. Matryoshka Dimension Truncation

EmbeddingGemma 2 is trained with Matryoshka Representation Learning, so the 768-dimensional output vector can be shortened by keeping only its leading dimensions. The supported dimensions are 768, 512, 256, and 128. Shorter vectors reduce storage requirements and can speed up similarity search at some cost to quality.

At runtime, follow these guidelines:

Re-normalize after truncating: Slicing a unit-length vector does not preserve unit length. The shortened vector must be L2-normalized before it is used for cosine similarity. Skipping this step can silently degrade ranking quality.
Queries and documents must share a dimension: A 768-dimensional query cannot be scored against a 128-dimensional corpus.

To avoid truncating and normalizing manually, use the truncate_dim and normalize_embeddings parameters when calling model.encode():

query_emb = model.encode(
    query,
    truncate_dim=128,  # or 512, 256
    normalize_embeddings=True,
)

Model quality at each dimension is reported in the truncation table above. Quality remains close to the full representation down to 256 dimensions. The 128-dimensional representation should be validated against your workload before adoption.

3. Numerical Precision

Run inference in bfloat16 or float32. Do not use float16.

EmbeddingGemma 2's activation range exceeds the dynamic range of float16. Using float16 may produce NaN or silently degraded embeddings.

bfloat16 is safe and carries the same 8-bit exponent as float32. It is the recommended default on hardware with native support and halves memory use relative to float32. Use float32 elsewhere, including on most CPUs.

In SentenceTransformers, configure the data type through model_kwargs:

import torch
from sentence_transformers import SentenceTransformer

dtype = (
    torch.bfloat16
    if torch.cuda.is_available() and torch.cuda.is_bf16_supported()
    else torch.float32
)

model = SentenceTransformer(
    "google/embeddinggemma-2",
    model_kwargs={"torch_dtype": dtype},
)

Model Data

The pre-training dataset is a large-scale, diverse collection of text and code, with a cutoff date of January 2025. The training data includes:

Web documents: Diverse web text covering a wide range of linguistic styles, topics, and vocabulary. The training dataset includes content in over 140 languages.
Code: Programming languages and code-related text to improve understanding of code semantics.

Data Processing

Several data cleaning and filtering methods were applied to the training data:

CSAM filtering: Filtering was applied at multiple stages in data preparation to exclude harmful and illegal content.
Sensitive data filtering: Automated techniques were used to filter certain personal information and other sensitive data from training sets.
Additional methods: Filtering based on content quality and safety in line with Google's AI responsibility policies.

Ethics & Safety

EmbeddingGemma 2 is a pre-trained embedding model. Unlike generative models, it does not undergo post-training alignment, safety tuning, or output-level moderation. Safety mitigations during development focused on pre-training data filtering to reduce exposure to harmful content and severe biases in the learned embedding space, in alignment with Google's AI Principles.

Since embedding models produce representations rather than user-facing text, safety risks manifest downstream in how those representations are used.

Developers and deployers are responsible for evaluating and implementing application-level safeguards, such as retrieval filtering and fairness testing, appropriate to their production use cases.

Deployments must adhere to the Gemma Prohibited Use Policy.

Usage and Limitations

EmbeddingGemma 2 has certain limitations that users should be aware of.

Intended Usage

EmbeddingGemma 2 generates text embeddings for downstream applications, including:

Retrieval: Semantic search across text and code, including document search, RAG over enterprise knowledge bases, and code search from natural-language queries.
Classification: Classifying text into predefined categories, such as sentiment or content categories.
Clustering: Grouping documents and text by semantic similarity, organizing document collections, and clustering customer feedback.
Semantic similarity: Measuring similarity between text inputs for recommendation systems, duplicate detection, paraphrase identification, and cross-lingual content alignment.
Fact verification: Retrieving evidence documents for a claim in automated fact-checking systems and source attribution pipelines.

Limitations

Training data: The quality and diversity of the training data significantly influence the model's capabilities. Biases or gaps in the training data can lead to limitations.
- The scope of the training dataset determines the subject areas the model can handle effectively.
- Although the model supports 100+ languages, performance may vary across languages.
Context and task complexity: Models perform best on tasks that can be framed with clear instructions. Open-ended or highly complex tasks may be challenging, and performance can depend on the context provided.
Language ambiguity and nuance: Natural language is inherently complex. The model may struggle with subtle nuances, sarcasm, or figurative language.
Task instruction usage: Omitting recommended task prefixes may lead to suboptimal embedding quality.

Ethical Considerations and Risks

In creating an open embedding model, the following considerations are important.

Bias and fairness
- Models trained on large-scale, real-world text data can reflect sociocultural biases in their training material.
- Training data underwent safety filtering to mitigate these risks.
Misinformation and misuse
- Embeddings can be misused to retrieve, classify, or organize content in false, misleading, or harmful ways. See the Responsible Generative AI Toolkit.
Transparency and accountability
- This model card summarizes the model's architecture, capabilities, limitations, and evaluation processes.
- Open embedding models make text representation technology accessible to developers and researchers.

Risks Identified and Mitigations

Misuse for malicious purposes: Technical limitations, developer education, and end-user education can help mitigate malicious applications of embedding models.
Privacy violations: Models were trained on data filtered to remove certain personal information and other sensitive data. Developers are encouraged to follow privacy regulations and use privacy-preserving techniques.
Perpetuation of biases: Continuous monitoring, evaluation metrics, human review, and debiasing techniques are encouraged during model development and deployment.

Benefits

EmbeddingGemma 2 is an efficient text embedding model designed for text retrieval and understanding tasks. It supports multilingual text and code, flexible embedding dimensions, and deployment on consumer hardware.

The model is designed to support responsible AI development, like other models in the Gemma family. Now can you write a proper README with vocab trimmed as focus and push to github

Downloads last month: 78

Safetensors

Model size

0.3B params

Tensor type

BF16