Blame for README.md

---
language:
- lug
library_name: transformers
pipeline_tag: sentence-similarity
tags:
- luganda
- embeddings
- sentence-embeddings
- semantic-search
- research
model-index:
- name: SAGE POND Luganda Sentence Embedding Model
  results:
  - task:
      type: sentence-similarity
    dataset:
      name: SAGE POND STS
      type: sagepond/sts
    metrics:
    - name: Pearson correlation
      type: pearson_correlation
      value: 34.05598714230456
    - name: Spearman correlation
      type: spearman_correlation
      value: 47.91287610143128
    - name: STS score
      type: sts_score
      value: 40.98443162186792
license: apache-2.0
datasets:
- sagepond/luganda
- sagepond/sts
metrics:
- spearmanr
- pearsonr
---

# SAGE POND Luganda Sentence Embedding Model

This is a sentence-embedding model developed by SAGE POND for experimentation with semantic representations in Luganda. It maps an input sentence to a **3,072-dimensional, L2-normalized embedding** that can be compared with other sentence embeddings using cosine similarity or used as a representation in downstream semantic applications.

## Intended use

The model was trained on **sentences**, not question-and-answer pairs, instruction-response examples, or conversational QA data. It should therefore be treated as a sentence-level embedding model.

The release is intended as a research example and an accessible starting point for people experimenting with Luganda language representations. It can also be used to build prototypes and semantic applications, including:

- Sentence similarity and semantic matching.
- Semantic search over Luganda text.
- Sentence clustering and grouping.
- Duplicate or near-duplicate sentence exploration.
- Embedding-based retrieval and other downstream experiments.

These are potential application areas, not claims that the model has been evaluated or optimized for every task listed above. Retrieval quality will depend on the corpus, preprocessing, similarity strategy, and evaluation data used by the application.

## STS benchmark results

The model was evaluated on the SAGE POND Semantic Textual Similarity benchmark, [`sagepond/sts`](https://huggingface.co/datasets/sagepond/sts). The evaluation set contains approximately **282 samples**.

| Metric | Score |
|---|---:|
| Pearson correlation score | 34.06 |
| Spearman correlation score | 47.91 |
| STS score | **40.98** |

The reported STS score is the arithmetic mean of the reported Pearson and Spearman scores. The values above are shown on the same 0–100 scale as the evaluation output.

**Benchmark characteristics:** although the evaluation set is compact (approximately 282 examples), it contains notably challenging sentence pairs. Its size should not be mistaken for an indication that the examples are easy or that the score is low simply because the benchmark is small. The results reflect the model's ability to capture semantic similarity across the particular difficult examples included in `sagepond/sts`; a similarly sized but easier test set could produce substantially different scores.

These results are an initial evaluation on the SAGE POND benchmark, not evidence of broad performance across every Luganda domain. They should not be directly compared with scores from other STS datasets or evaluation protocols without checking differences in data selection, difficulty, scoring, and preprocessing. A larger and independently curated evaluation would provide a stronger estimate of generalization.

## Model details

| Property | Value |
|---|---|
| Model type | Transformer sentence encoder |
| Primary language | Luganda |
| Training data format | Sentences (not QA-style examples) |
| Tokenizer | SentencePiece |
| Vocabulary size | 40,000 tokens |
| Transformer layers | 12 |
| Hidden size | 768 |
| Attention heads | 16 |
| Feed-forward dimension | 3,072 |
| Maximum sequence length | 600 tokens |
| Sentence embedding dimension | 3,072 |

The model adds learned positional embeddings to token embeddings, encodes the sequence with a Transformer encoder, applies masked mean pooling, and projects the pooled representation to the final 3,072-dimensional embedding space. The output embeddings are L2-normalized.

## Usage

The repository uses custom Transformers code. Load it with `trust_remote_code=True`:

```python
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

model_id = "sagepond/embedding-small"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model.eval()

sentences = [
    "Ekigambo eky'okulabirako eky'Oluganda.",
    "Sentensi endala ey'okugezesa.",
]

inputs = tokenizer(
    sentences,
    padding=True,
    truncation=True,
    max_length=600,
    return_tensors="pt",
)

with torch.inference_mode():
    embeddings = model(**inputs).pooler_output

print(embeddings.shape)  # torch.Size([2, 3072])
```

Because the model returns normalized embeddings, cosine similarity can be calculated with a dot product:

```python
similarity = embeddings[0] @ embeddings[1]
print(similarity.item())
```

For a larger collection of sentences, encode texts in batches and store the resulting vectors in a vector index or another retrieval system. Use the same model and tokenizer version for both indexed documents and incoming queries.

## Limitations

- The benchmark contains approximately 282 samples, so its scores have limited statistical coverage.
- The model was trained on sentence-level data, not question-answer pairs. Do not assume that it is optimized for question-to-passage retrieval or QA matching without task-specific evaluation.
- The 600-token maximum sequence length is a model input limit; long documents should be split into suitable chunks and embedded separately.
- Similarity scores are relative signals for ranking or comparison. They are not calibrated probabilities or factual confidence scores.

## Intended audience

Researchers, developers, students, and language-technology practitioners interested in experimenting with Luganda sentence embeddings, semantic similarity, and semantic retrieval.

## Dataset

The STS evaluation dataset is available at [sagepond/sts](https://huggingface.co/datasets/sagepond/sts).

## Citation and attribution

This model was developed by **SAGE POND**. When using it in research or derivative work, cite the model repository and the [`sagepond/sts`](https://huggingface.co/datasets/sagepond/sts) dataset, and report the evaluation setup used in your experiments.