DylanCouzon commited on
Commit
4380add
·
verified ·
1 Parent(s): d9d575a

constella-zero — FastEmbed-first card, run p35w-2m-s2500

Browse files
Files changed (1) hide show
  1. README.md +9 -5
README.md CHANGED
@@ -19,8 +19,9 @@ The query side of an **asymmetric dual encoder**: documents are indexed once, in
19
  large frozen encoder; queries are encoded on the device by **a lookup table**.
20
 
21
  There is no transformer here. The model is 30,522 × 1024 int8 rows and one pooling rule —
22
- encoding a query is a gather and a weighted sum. The query asset is **31.8 MB** and a query costs
23
- **~0.05 ms** on one CPU core.
 
24
 
25
  It was distilled from [`stella_en_400M_v5`](https://huggingface.co/NovaSearch/stella_en_400M_v5)
26
  so that its output lands in that model's document space. The matching document encoder is
@@ -177,12 +178,15 @@ scores — not reciprocal rank fusion, so Qdrant's `Fusion.RRF` will not reprodu
177
  | | |
178
  |---|---|
179
  | query asset (int8 rows + scales + tokenizer) | 31.8 MB |
180
- | `model.onnx`, batch 1, one thread, 8-token query | 0.047 ms |
181
- | `model.onnx`, batch 1, one thread, 512-token query | 1.22 ms |
182
- | `zero_encoder.py`, batch 1, one CPU core | 0.38 ms |
183
  | hydration (cold load to first query) | 0.22 s |
184
  | document vectors, 1024-d fp16 / int8 | 2.05 / 1.02 GB per 1M — raw payload, before index overhead |
185
 
 
 
 
186
  The graph derives token counts from an all-pairs comparison, so cost grows with the **square** of
187
  sequence length — 26x from an 8-token query to a 512-token one. Real queries sit at the short end
188
  (median 13 wordpieces).
 
19
  large frozen encoder; queries are encoded on the device by **a lookup table**.
20
 
21
  There is no transformer here. The model is 30,522 × 1024 int8 rows and one pooling rule —
22
+ encoding a query is a gather and a weighted sum. The query asset is **31.8 MB**, and the reference
23
+ implementation encodes a query end to end, tokenization included, in **0.38 ms** on one CPU core
24
+ (the ONNX graph alone runs an 8-token query in 0.047 ms — see [Costs](#costs)).
25
 
26
  It was distilled from [`stella_en_400M_v5`](https://huggingface.co/NovaSearch/stella_en_400M_v5)
27
  so that its output lands in that model's document space. The matching document encoder is
 
178
  | | |
179
  |---|---|
180
  | query asset (int8 rows + scales + tokenizer) | 31.8 MB |
181
+ | `model.onnx` graph execution, batch 1, one thread, 8-token query | 0.047 ms |
182
+ | `model.onnx` graph execution, batch 1, one thread, 512-token query | 1.22 ms |
183
+ | `zero_encoder.py` end to end, batch 1, one CPU core, incl. tokenization | 0.38 ms |
184
  | hydration (cold load to first query) | 0.22 s |
185
  | document vectors, 1024-d fp16 / int8 | 2.05 / 1.02 GB per 1M — raw payload, before index overhead |
186
 
187
+ The graph rows exclude tokenization; `zero_encoder.py`'s 0.38 ms is the end-to-end figure and the
188
+ honest one to compare against another encoder. No end-to-end FastEmbed timing is published here.
189
+
190
  The graph derives token counts from an all-pairs comparison, so cost grows with the **square** of
191
  sequence length — 26x from an 8-token query to a 512-token one. Real queries sit at the short end
192
  (median 13 wordpieces).