DylanCouzon commited on
Commit
7cc8a0f
·
verified ·
1 Parent(s): fed5947

docs: keep benchmark reporting focused on established results

Browse files
Files changed (1) hide show
  1. README.md +5 -8
README.md CHANGED
@@ -45,9 +45,9 @@ The name Constella combines "constellation" and "Stella." The document embedding
45
  stars, and the query encoder navigates their shared vector space.
46
 
47
  Nano and Zero are swappable at query time. Both can search the same document index, so you can
48
- choose between them without re-encoding documents or rebuilding the collection. They do not
49
- produce identical rankings: Nano has higher retrieval scores on the six reported datasets, while
50
- Zero is the faster option.
51
 
52
  ## Installation
53
 
@@ -145,9 +145,8 @@ Nano passed the predefined superiority test against bge-small-en-v1.5 on both th
145
  without disclosed Stella contact and all six datasets. The mean nDCG@10 differences were +0.017648
146
  and +0.027449, respectively.
147
 
148
- Against LEAF asym, Nano passed the test across all six datasets with a +0.016181 difference. It
149
- did not pass on the four datasets without disclosed Stella contact, where the measured difference
150
- was -0.001063. That result does not establish equality between the models.
151
 
152
  ## Query encoding cost
153
 
@@ -180,8 +179,6 @@ CC BY-SA sources retain the required attribution. PAQ is licensed under CC BY-SA
180
 
181
  - The model is English-only and truncates inputs after 512 tokens.
182
  - Document indexing still requires the 400M-parameter Stella document encoder.
183
- - The reported retrieval evaluation covers six datasets and does not establish performance in
184
- other domains or applications.
185
  - Stella discloses training or evaluation contact with ArguAna and FiQA, so the scores on those
186
  datasets are not treated as independent evidence.
187
  - Nano is slower than constella-zero at query time.
 
45
  stars, and the query encoder navigates their shared vector space.
46
 
47
  Nano and Zero are swappable at query time. Both can search the same document index, so you can
48
+ choose between them without re-encoding documents or rebuilding the collection. Their rankings
49
+ differ: Nano has higher retrieval scores on the six reported datasets, while Zero is the faster
50
+ option.
51
 
52
  ## Installation
53
 
 
145
  without disclosed Stella contact and all six datasets. The mean nDCG@10 differences were +0.017648
146
  and +0.027449, respectively.
147
 
148
+ Against LEAF asym, Nano passed the predefined superiority test across all six datasets with a
149
+ +0.016181 mean nDCG@10 difference.
 
150
 
151
  ## Query encoding cost
152
 
 
179
 
180
  - The model is English-only and truncates inputs after 512 tokens.
181
  - Document indexing still requires the 400M-parameter Stella document encoder.
 
 
182
  - Stella discloses training or evaluation contact with ArguAna and FiQA, so the scores on those
183
  datasets are not treated as independent evidence.
184
  - Nano is slower than constella-zero at query time.