constella-zero — FastEmbed-first card, run p35w-2m-s2500
Browse files
README.md
CHANGED
|
@@ -19,8 +19,9 @@ The query side of an **asymmetric dual encoder**: documents are indexed once, in
|
|
| 19 |
large frozen encoder; queries are encoded on the device by **a lookup table**.
|
| 20 |
|
| 21 |
There is no transformer here. The model is 30,522 × 1024 int8 rows and one pooling rule —
|
| 22 |
-
encoding a query is a gather and a weighted sum. The query asset is **31.8 MB** and
|
| 23 |
-
**
|
|
|
|
| 24 |
|
| 25 |
It was distilled from [`stella_en_400M_v5`](https://huggingface.co/NovaSearch/stella_en_400M_v5)
|
| 26 |
so that its output lands in that model's document space. The matching document encoder is
|
|
@@ -177,12 +178,15 @@ scores — not reciprocal rank fusion, so Qdrant's `Fusion.RRF` will not reprodu
|
|
| 177 |
| | |
|
| 178 |
|---|---|
|
| 179 |
| query asset (int8 rows + scales + tokenizer) | 31.8 MB |
|
| 180 |
-
| `model.onnx`, batch 1, one thread, 8-token query | 0.047 ms |
|
| 181 |
-
| `model.onnx`, batch 1, one thread, 512-token query | 1.22 ms |
|
| 182 |
-
| `zero_encoder.py`, batch 1, one CPU core | 0.38 ms |
|
| 183 |
| hydration (cold load to first query) | 0.22 s |
|
| 184 |
| document vectors, 1024-d fp16 / int8 | 2.05 / 1.02 GB per 1M — raw payload, before index overhead |
|
| 185 |
|
|
|
|
|
|
|
|
|
|
| 186 |
The graph derives token counts from an all-pairs comparison, so cost grows with the **square** of
|
| 187 |
sequence length — 26x from an 8-token query to a 512-token one. Real queries sit at the short end
|
| 188 |
(median 13 wordpieces).
|
|
|
|
| 19 |
large frozen encoder; queries are encoded on the device by **a lookup table**.
|
| 20 |
|
| 21 |
There is no transformer here. The model is 30,522 × 1024 int8 rows and one pooling rule —
|
| 22 |
+
encoding a query is a gather and a weighted sum. The query asset is **31.8 MB**, and the reference
|
| 23 |
+
implementation encodes a query end to end, tokenization included, in **0.38 ms** on one CPU core
|
| 24 |
+
(the ONNX graph alone runs an 8-token query in 0.047 ms — see [Costs](#costs)).
|
| 25 |
|
| 26 |
It was distilled from [`stella_en_400M_v5`](https://huggingface.co/NovaSearch/stella_en_400M_v5)
|
| 27 |
so that its output lands in that model's document space. The matching document encoder is
|
|
|
|
| 178 |
| | |
|
| 179 |
|---|---|
|
| 180 |
| query asset (int8 rows + scales + tokenizer) | 31.8 MB |
|
| 181 |
+
| `model.onnx` graph execution, batch 1, one thread, 8-token query | 0.047 ms |
|
| 182 |
+
| `model.onnx` graph execution, batch 1, one thread, 512-token query | 1.22 ms |
|
| 183 |
+
| `zero_encoder.py` end to end, batch 1, one CPU core, incl. tokenization | 0.38 ms |
|
| 184 |
| hydration (cold load to first query) | 0.22 s |
|
| 185 |
| document vectors, 1024-d fp16 / int8 | 2.05 / 1.02 GB per 1M — raw payload, before index overhead |
|
| 186 |
|
| 187 |
+
The graph rows exclude tokenization; `zero_encoder.py`'s 0.38 ms is the end-to-end figure and the
|
| 188 |
+
honest one to compare against another encoder. No end-to-end FastEmbed timing is published here.
|
| 189 |
+
|
| 190 |
The graph derives token counts from an all-pairs comparison, so cost grows with the **square** of
|
| 191 |
sequence length — 26x from an 8-token query to a 512-token one. Real queries sit at the short end
|
| 192 |
(median 13 wordpieces).
|