docs: keep benchmark reporting focused on established results
Browse files
README.md
CHANGED
|
@@ -45,9 +45,9 @@ The name Constella combines "constellation" and "Stella." The document embedding
|
|
| 45 |
stars, and the query encoder navigates their shared vector space.
|
| 46 |
|
| 47 |
Nano and Zero are swappable at query time. Both can search the same document index, so you can
|
| 48 |
-
choose between them without re-encoding documents or rebuilding the collection.
|
| 49 |
-
|
| 50 |
-
|
| 51 |
|
| 52 |
## Installation
|
| 53 |
|
|
@@ -145,9 +145,8 @@ Nano passed the predefined superiority test against bge-small-en-v1.5 on both th
|
|
| 145 |
without disclosed Stella contact and all six datasets. The mean nDCG@10 differences were +0.017648
|
| 146 |
and +0.027449, respectively.
|
| 147 |
|
| 148 |
-
Against LEAF asym, Nano passed the test across all six datasets with a
|
| 149 |
-
|
| 150 |
-
was -0.001063. That result does not establish equality between the models.
|
| 151 |
|
| 152 |
## Query encoding cost
|
| 153 |
|
|
@@ -180,8 +179,6 @@ CC BY-SA sources retain the required attribution. PAQ is licensed under CC BY-SA
|
|
| 180 |
|
| 181 |
- The model is English-only and truncates inputs after 512 tokens.
|
| 182 |
- Document indexing still requires the 400M-parameter Stella document encoder.
|
| 183 |
-
- The reported retrieval evaluation covers six datasets and does not establish performance in
|
| 184 |
-
other domains or applications.
|
| 185 |
- Stella discloses training or evaluation contact with ArguAna and FiQA, so the scores on those
|
| 186 |
datasets are not treated as independent evidence.
|
| 187 |
- Nano is slower than constella-zero at query time.
|
|
|
|
| 45 |
stars, and the query encoder navigates their shared vector space.
|
| 46 |
|
| 47 |
Nano and Zero are swappable at query time. Both can search the same document index, so you can
|
| 48 |
+
choose between them without re-encoding documents or rebuilding the collection. Their rankings
|
| 49 |
+
differ: Nano has higher retrieval scores on the six reported datasets, while Zero is the faster
|
| 50 |
+
option.
|
| 51 |
|
| 52 |
## Installation
|
| 53 |
|
|
|
|
| 145 |
without disclosed Stella contact and all six datasets. The mean nDCG@10 differences were +0.017648
|
| 146 |
and +0.027449, respectively.
|
| 147 |
|
| 148 |
+
Against LEAF asym, Nano passed the predefined superiority test across all six datasets with a
|
| 149 |
+
+0.016181 mean nDCG@10 difference.
|
|
|
|
| 150 |
|
| 151 |
## Query encoding cost
|
| 152 |
|
|
|
|
| 179 |
|
| 180 |
- The model is English-only and truncates inputs after 512 tokens.
|
| 181 |
- Document indexing still requires the 400M-parameter Stella document encoder.
|
|
|
|
|
|
|
| 182 |
- Stella discloses training or evaluation contact with ArguAna and FiQA, so the scores on those
|
| 183 |
datasets are not treated as independent evidence.
|
| 184 |
- Nano is slower than constella-zero at query time.
|