card: quality numbers re-measured on the released weights
Browse files
README.md
CHANGED
|
@@ -51,22 +51,20 @@ Laptop RTX 5060, 2026-09-11.
|
|
| 51 |
|
| 52 |
| model | layers | texts/s | cosqa | stack-qa | code→code | our queries | Russian hit@1 |
|
| 53 |
|---|--:|--:|--:|--:|--:|--:|--:|
|
| 54 |
-
| **code-daemon-embed-v1** | **4** | **3 457** |
|
| 55 |
| multilingual-e5-base | 12 | 854 | 29.8 | 79.4 | 57.7 | 60.7 | **0.24** |
|
| 56 |
| CodeRankEmbed | 12 | 723 | 35.5 | 73.3 | — | 64.6 | 0.00 |
|
| 57 |
| gte-modernbert-base *(its teacher)* | 22 | 606 | **36.6** | **81.4** | **82.5** | **70.5** | 0.15 |
|
| 58 |
|
| 59 |
-
- **4.0–5.7× the throughput for
|
| 60 |
constraint and a lexical channel runs beside it; pick `gte-modernbert-base` when vector-only
|
| 61 |
accuracy is.
|
| 62 |
- **Inside the hybrid search it was built for** (80 real agent queries, file level, runtime
|
| 63 |
-
reranker off): the right file is first **
|
| 64 |
-
- **Its own depth trade:** against its 6-layer parent it gives up ~4 % relative hit@1 for ~1.5×
|
| 65 |
-
the engine speed.
|
| 66 |
- **Not multilingual.** The vocabulary prune kept 942 of XLM-R's 31 671 Cyrillic pieces; for
|
| 67 |
non-English queries use `multilingual-e5-base`.
|
| 68 |
-
- **Not for long text
|
| 69 |
-
|
| 70 |
|
| 71 |
### Throughput in a real index
|
| 72 |
|
|
|
|
| 51 |
|
| 52 |
| model | layers | texts/s | cosqa | stack-qa | code→code | our queries | Russian hit@1 |
|
| 53 |
|---|--:|--:|--:|--:|--:|--:|--:|
|
| 54 |
+
| **code-daemon-embed-v1** | **4** | **3 457** | 28.5 | 49.9 | 42.9 | 60.1 | 0.06 |
|
| 55 |
| multilingual-e5-base | 12 | 854 | 29.8 | 79.4 | 57.7 | 60.7 | **0.24** |
|
| 56 |
| CodeRankEmbed | 12 | 723 | 35.5 | 73.3 | — | 64.6 | 0.00 |
|
| 57 |
| gte-modernbert-base *(its teacher)* | 22 | 606 | **36.6** | **81.4** | **82.5** | **70.5** | 0.15 |
|
| 58 |
|
| 59 |
+
- **4.0–5.7× the throughput for 1–10 points of nDCG on our queries.** Pick it when indexing speed is the
|
| 60 |
constraint and a lexical channel runs beside it; pick `gte-modernbert-base` when vector-only
|
| 61 |
accuracy is.
|
| 62 |
- **Inside the hybrid search it was built for** (80 real agent queries, file level, runtime
|
| 63 |
+
reranker off): the right file is first **44 %** of the time and in the top 5 **69 %**.
|
|
|
|
|
|
|
| 64 |
- **Not multilingual.** The vocabulary prune kept 942 of XLM-R's 31 671 Cyrillic pieces; for
|
| 65 |
non-English queries use `multilingual-e5-base`.
|
| 66 |
+
- **Not for long text or general prose.** On four CoIR tasks it averages 52.8 nDCG@10 — 89.8
|
| 67 |
+
on Python docstring→code, 28.5 on cosqa.
|
| 68 |
|
| 69 |
### Throughput in a real index
|
| 70 |
|