Sentence Similarity
sentence-transformers
Safetensors
English
embedding_gemma2
feature-extraction
embeddings
retrieval
semantic-search
matryoshka
nasa
combustion
microgravity
spacecraft-fire-safety
Instructions to use rarfileexe/FlameLedger-Encoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use rarfileexe/FlameLedger-Encoder with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("rarfileexe/FlameLedger-Encoder") sentences = [ "That is a happy person", "That is a happy dog", "That is a very happy person", "Today is a sunny day" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
|
Download scorecard.md from rarfileexe/FlameLedger-Encoder: direct link, hf CLI and curl.
- Browser
- Download file 3.72 kB
-
https://huggingface.co/rarfileexe/FlameLedger-Encoder/resolve/main/scorecard.md
- Command line
-
hf download hf://rarfileexe/FlameLedger-Encoder/scorecard.md
-
curl -L -o scorecard.md https://huggingface.co/rarfileexe/FlameLedger-Encoder/resolve/main/scorecard.md
3.72 kB
synthetic TEST, held-out papers only (n=150 queries)
| method | recall@1 | recall@5 | recall@10 | MRR@10 |
|---|---|---|---|---|
| BM25 | 0.800 | 0.967 | 1.000 | 0.874 |
| base dense 768 | 0.713 | 0.967 | 0.993 | 0.811 |
| base dense 512 | 0.693 | 0.967 | 0.993 | 0.807 |
| base dense 256 | 0.720 | 0.947 | 0.987 | 0.814 |
| base dense 128 | 0.687 | 0.900 | 0.987 | 0.790 |
| base + BM25 (RRF) | 0.787 | 0.987 | 1.000 | 0.873 |
| fine-tuned dense 768 | 0.793 | 0.993 | 1.000 | 0.871 |
| fine-tuned dense 512 | 0.807 | 0.987 | 1.000 | 0.880 |
| fine-tuned dense 256 | 0.800 | 0.973 | 0.993 | 0.871 |
| fine-tuned dense 128 | 0.800 | 0.967 | 0.993 | 0.865 |
| fine-tuned + BM25 (RRF) | 0.813 | 1.000 | 1.000 | 0.891 |
synthetic TEST, full archive (n=150 queries)
| method | recall@1 | recall@5 | recall@10 | MRR@10 |
|---|---|---|---|---|
| BM25 | 0.647 | 0.900 | 0.940 | 0.752 |
| base dense 768 | 0.527 | 0.840 | 0.893 | 0.652 |
| base dense 512 | 0.513 | 0.840 | 0.900 | 0.646 |
| base dense 256 | 0.527 | 0.793 | 0.907 | 0.641 |
| base dense 128 | 0.453 | 0.767 | 0.847 | 0.589 |
| base + BM25 (RRF) | 0.673 | 0.913 | 0.967 | 0.776 |
| fine-tuned dense 768 | 0.647 | 0.920 | 0.953 | 0.747 |
| fine-tuned dense 512 | 0.653 | 0.907 | 0.967 | 0.752 |
| fine-tuned dense 256 | 0.627 | 0.893 | 0.960 | 0.734 |
| fine-tuned dense 128 | 0.627 | 0.833 | 0.920 | 0.720 |
| fine-tuned + BM25 (RRF) | 0.713 | 0.940 | 0.987 | 0.807 |
GOLD (teacher-drafted, UNREVIEWED) (n=23 queries)
| method | recall@1 | recall@5 | recall@10 | MRR@10 |
|---|---|---|---|---|
| BM25 | 0.304 | 0.609 | 0.739 | 0.436 |
| base dense 768 | 0.391 | 0.609 | 0.739 | 0.506 |
| base dense 512 | 0.391 | 0.696 | 0.696 | 0.504 |
| base dense 256 | 0.348 | 0.609 | 0.696 | 0.476 |
| base dense 128 | 0.391 | 0.565 | 0.565 | 0.454 |
| base + BM25 (RRF) | 0.261 | 0.565 | 0.696 | 0.422 |
| fine-tuned dense 768 | 0.478 | 0.609 | 0.696 | 0.537 |
| fine-tuned dense 512 | 0.435 | 0.652 | 0.696 | 0.523 |
| fine-tuned dense 256 | 0.435 | 0.565 | 0.739 | 0.509 |
| fine-tuned dense 128 | 0.391 | 0.522 | 0.609 | 0.454 |
| fine-tuned + BM25 (RRF) | 0.391 | 0.696 | 0.696 | 0.505 |
Fine-tuned minus base (paired bootstrap, 95% CI)
| query set | comparison | metric | delta | 95% CI |
|---|---|---|---|---|
| synthetic TEST, held-out papers only | base dense 768 -> fine-tuned dense 768 | recall@5 | +0.027 | [+0.007, +0.060] |
| synthetic TEST, held-out papers only | base dense 768 -> fine-tuned dense 768 | mrr@10 | +0.060 | [+0.031, +0.092] |
| synthetic TEST, held-out papers only | base + BM25 (RRF) -> fine-tuned + BM25 (RRF) | recall@5 | +0.013 | [+0.000, +0.033] |
| synthetic TEST, held-out papers only | base + BM25 (RRF) -> fine-tuned + BM25 (RRF) | mrr@10 | +0.018 | [-0.002, +0.037] |
| synthetic TEST, full archive | base dense 768 -> fine-tuned dense 768 | recall@5 | +0.080 | [+0.033, +0.127] |
| synthetic TEST, full archive | base dense 768 -> fine-tuned dense 768 | mrr@10 | +0.095 | [+0.053, +0.141] |
| synthetic TEST, full archive | base + BM25 (RRF) -> fine-tuned + BM25 (RRF) | recall@5 | +0.027 | [+0.000, +0.060] |
| synthetic TEST, full archive | base + BM25 (RRF) -> fine-tuned + BM25 (RRF) | mrr@10 | +0.031 | [-0.001, +0.065] |
| GOLD (teacher-drafted, UNREVIEWED) | base dense 768 -> fine-tuned dense 768 | recall@5 | +0.000 | [-0.130, +0.130] |
| GOLD (teacher-drafted, UNREVIEWED) | base dense 768 -> fine-tuned dense 768 | mrr@10 | +0.031 | [-0.021, +0.099] |
| GOLD (teacher-drafted, UNREVIEWED) | base + BM25 (RRF) -> fine-tuned + BM25 (RRF) | recall@5 | +0.130 | [+0.000, +0.261] |
| GOLD (teacher-drafted, UNREVIEWED) | base + BM25 (RRF) -> fine-tuned + BM25 (RRF) | mrr@10 | +0.083 | [+0.014, +0.164] |