Akshayram1/embeddinggemma-300m

SentenceTransformer

This model was finetuned with Unsloth.

based on unsloth/embeddinggemma-300m

This is a sentence-transformers model finetuned from unsloth/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: unsloth/embeddinggemma-300m
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'PeftModelForFeatureExtraction'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
  (4): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    'what retry policy replaced five attempts at 30 seconds',
]
documents = [
    'Retry Worker with jittered exponential backoff\n\nBuild workers/retry_worker.py to consume atlas.retry.v1, re-attempt delivery with exponential backoff and full jitter, and hand exhausted events to the Dead Letter Queue. Acceptance criteria: transient Kafka or sink failures recover without operator action, retry attempts are labelled by failure class in monitoring/metrics.py, and the backoff ceiling is configurable through deploy/helm/atlas/values.yaml. Status history: todo 2026-08-17, in_progress 2026-08-25, in_review 2026-08-28, done 2026-09-01. The first runbook draft documented five attempts capped at 30 seconds; the retry policy decision of 2026-08-26 replaced that with eight attempts and a 15 minute ceiling, and the runbook was updated. Depends on ATLAS-107. Reporter Vikram Joshi, assignee Neha Kapoor. Evidence: PR-005 and commit b8740e2fd1a3.',
    'Alert on replication slot growth before disk pressure\n\nA stalled Atlas consumer holds the PostgreSQL replication slot open and retained WAL grows until the primary runs out of disk. Add a slot_retained_bytes gauge to monitoring/metrics.py and page SRE at 20 GB retained or 30 minutes of no slot advance, whichever comes first. Acceptance criteria: the alert fired correctly during the 2026-08-19 SSL outage rehearsal, the on-call runbook documents the drain procedure, and the gauge is visible on the Observability Dashboard. Status history: todo 2026-08-17, in_progress 2026-08-20, done 2026-08-24. Depends on ATLAS-101; feeds ATLAS-109. Reporter Vikram Joshi, assignee Vikram Joshi. Evidence: PR-007 panels and the dashboard on-call runbook.',
    'Runbook: Dead Letter Queue Triage and Replay\n\nDocument kind: runbook. Owner: Neha Kapoor. Contributors: Vikram Joshi, Rohan Mehta. Status: active, first draft 2026-08-25, corrected 2026-08-27, updated 2026-09-10. Scope: what to do when Dead Letter Queue depth alerts on the Observability Dashboard under ATLAS-119. Step 1, classify: group atlas.dlq.v1 records by failure class. Decode failures usually mean a schema change, so read the schema migration runbook next. Sink failures usually mean a downstream outage and often clear themselves through the Retry Worker. Step 2, fix forward, never replay into a broken pipeline. Step 3, replay with atlasctl dlq replay, filtered by pipeline and time window, always with --dry-run first. Replayed events keep their original event_id, so downstream deduplication still holds, which is the property established by the checkpoint decision and ATLAS-103. Correction 2026-08-27: the first draft of this runbook stated a retry budget of five attempts capped at 30 seconds. That is superseded by the retry policy decision of eight attempts with a 15 minute ceiling under ATLAS-108. Update 2026-09-10: the replay CLI in ATLAS-118 is still in review with open rate-limiting comments, so for Atlas 1.0 replay is performed manually using the worked example from the 2026-08-26 regression, where 312 malformed envelopes were replayed in three batches of 104.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.2781, 0.1833, 0.2271]])

Evaluation

Metrics

Information Retrieval

• Dataset: atlas
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.31
cosine_accuracy@3	0.55
cosine_accuracy@5	0.65
cosine_accuracy@10	0.78
cosine_precision@1	0.31
cosine_precision@3	0.1833
cosine_precision@5	0.13
cosine_precision@10	0.078
cosine_recall@1	0.31
cosine_recall@3	0.55
cosine_recall@5	0.65
cosine_recall@10	0.78
cosine_ndcg@10	0.5338
cosine_mrr@10	0.4561
cosine_map@100	0.4681

Training Details

Training Dataset

Unnamed Dataset

• Size: 200 training samples
• Columns: question, passage_text, and negative
• Approximate statistics based on the first 100 samples:

  	question	passage_text	negative
  type	string	string	string
  modality	text	text	text
  details	min: 7 tokens mean: 11.97 tokens max: 20 tokens	min: 193 tokens mean: 289.81 tokens max: 454 tokens	min: 198 tokens mean: 277.24 tokens max: 454 tokens

• Samples:

  question	passage_text	negative
  why does atlas use pgoutput instead of wal2json	Atlas PostgreSQL CDC Reader Design Document kind: design. Author: Akshay Sharma. Reviewers: Maya Rao, Rohan Mehta, Lina Thomas. Status: active, first published 2026-08-04, revised 2026-08-18. This page describes how Project Atlas reads change data from PostgreSQL. The reader in connectors/postgres/cdc_reader.py in the atlas-connectors repository opens a logical replication slot named atlas_slot_, decodes pgoutput messages, and publishes one Atlas envelope per committed row change onto Kafka. Relation messages are cached per relation OID so column metadata does not have to be re-fetched per row. Decisions: use pgoutput rather than wal2json because it ships with PostgreSQL 16 and avoids a plugin install; one slot per pipeline rather than one shared slot, so a stalled consumer cannot hold back unrelated pipelines; reconnect with bounded backoff and never auto-drop a slot. Affected tickets: ATLAS-101 for the reader itself, ATLAS-120 for the staging PostgreSQL 16 upgrade that blo...	Map envelope fields by name across schema generations Repository atlas-connectors, branch fix/ATLAS-112-name-based-mapping, author Akshay Sharma, reviewers Neha Kapoor, Lina Thomas and Sara Khan. Fixes ATLAS-112 and completes ATLAS-106, implementing the correction recorded in the Atlas Schema Evolution Handling Design after the war room on 2026-08-26. Replaces positional tuple mapping in schemas/event_envelope.py with name-based mapping keyed by relation OID plus a new required schema_generation field, and routes unmappable envelopes to the Dead Letter Queue instead of halting the pipeline. Adds ADD COLUMN and DROP COLUMN under-load cases to tests/integration/test_postgres_cdc.py. Because schema_generation is required, the Schema Registry could no longer stay on FULL compatibility, which is what forced ADR-002 and ATLAS-122; the mode assertion moves into deploy/helm/atlas/values.yaml. Review findings: Lina Thomas required the DROP COLUMN test to run with events flowing rather than pau...
  what replication slot name does each pipeline get	Atlas PostgreSQL CDC Reader Design Document kind: design. Author: Akshay Sharma. Reviewers: Maya Rao, Rohan Mehta, Lina Thomas. Status: active, first published 2026-08-04, revised 2026-08-18. This page describes how Project Atlas reads change data from PostgreSQL. The reader in connectors/postgres/cdc_reader.py in the atlas-connectors repository opens a logical replication slot named atlas_slot_, decodes pgoutput messages, and publishes one Atlas envelope per committed row change onto Kafka. Relation messages are cached per relation OID so column metadata does not have to be re-fetched per row. Decisions: use pgoutput rather than wal2json because it ships with PostgreSQL 16 and avoids a plugin install; one slot per pipeline rather than one shared slot, so a stalled consumer cannot hold back unrelated pipelines; reconnect with bounded backoff and never auto-drop a slot. Affected tickets: ATLAS-101 for the reader itself, ATLAS-120 for the staging PostgreSQL 16 upgrade that blo...	Atlas Sprint 1 demo Agenda: demonstrate what Project Atlas can do at the end of Sprint 1 to Sara Khan and the wider Aster Labs engineering group. Akshay Sharma demonstrated the CDC reader from ATLAS-101 streaming inserts, updates and deletes from the staging PostgreSQL 16 instance into Kafka, with each change wrapped in the canonical envelope from ATLAS-102 and resolved against the Schema Registry under ATLAS-105. Neha Kapoor showed the envelope structure and explained why event_id is derived from commit_lsn plus primary key rather than generated. Lina Thomas then demonstrated the uncomfortable part honestly: killing the reader mid-batch and showing the duplicate events that prompted ATLAS-103, alongside the fix on branch feature/ATLAS-103-durable-checkpoint that was in review that afternoon. Sara Khan asked whether duplicates could reach a pilot customer; the answer was yes today, no after PR-003 merges, and downstream consumers can always deduplicate on event_id. Maya Rao used the m...
  what fields are in the atlas event envelope	Atlas Event Envelope and Schema Registry Design Document kind: design. Author: Neha Kapoor. Contributors: Akshay Sharma, Sara Khan. Status: active, published 2026-08-05, amended 2026-08-27. Every message Project Atlas publishes is wrapped in a single envelope defined in schemas/event_envelope.py: event_id, source_pipeline, table, op, commit_lsn, emitted_at, schema_version and payload. event_id is deterministic, derived from commit_lsn plus the primary key of the row, so a replay after restart produces the same identifier and downstream consumers can deduplicate. That property is what made the duplicate delivery problem in ATLAS-103 measurable rather than invisible. Envelopes are serialised as Avro and registered with the Schema Registry under atlas.events.v1-value, covered by ATLAS-102 and ATLAS-105 and shipped in PR-002 with commit 7b2e41c5da90. Original compatibility choice: FULL. Amendment 2026-08-27: this page previously mandated FULL compatibility. That is superseded by the Schem...	Map envelope fields by name across schema generations Repository atlas-connectors, branch fix/ATLAS-112-name-based-mapping, author Akshay Sharma, reviewers Neha Kapoor, Lina Thomas and Sara Khan. Fixes ATLAS-112 and completes ATLAS-106, implementing the correction recorded in the Atlas Schema Evolution Handling Design after the war room on 2026-08-26. Replaces positional tuple mapping in schemas/event_envelope.py with name-based mapping keyed by relation OID plus a new required schema_generation field, and routes unmappable envelopes to the Dead Letter Queue instead of halting the pipeline. Adds ADD COLUMN and DROP COLUMN under-load cases to tests/integration/test_postgres_cdc.py. Because schema_generation is required, the Schema Registry could no longer stay on FULL compatibility, which is what forced ADR-002 and ATLAS-122; the mode assertion moves into deploy/helm/atlas/values.yaml. Review findings: Lina Thomas required the DROP COLUMN test to run with events flowing rather than pau...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim",
      "gather_across_devices": false,
      "directions": [
          "query_to_doc"
      ],
      "partition_mode": "joint",
      "hardness_mode": null,
      "hardness_strength": 0.0
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• lr_scheduler_type: constant_with_warmup
• warmup_ratio: 0.05
• prompts: {'question': '', 'passage_text': '', 'negative': ''}
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 8
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.05
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: {'question': '', 'passage_text': '', 'negative': ''}
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	atlas_cosine_ndcg@10
-1	-1	-	0.1975
0.3846	5	2.9373	-
0.7692	10	2.561	-
-1	-1	-	0.5338

Training Time

• Training: 5.1 minutes

Framework Versions

• Python: 3.13.15
• Sentence Transformers: 5.7.0
• Transformers: 4.56.2
• PyTorch: 2.11.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.3.0
• Tokenizers: 0.22.2

Additional Resources

• Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
• Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}