faxenoff commited on
Commit
d774b09
·
verified ·
1 Parent(s): fe5f5e9

card: quality numbers re-measured on the released weights

Browse files
Files changed (1) hide show
  1. README.md +5 -7
README.md CHANGED
@@ -51,22 +51,20 @@ Laptop RTX 5060, 2026-09-11.
51
 
52
  | model | layers | texts/s | cosqa | stack-qa | code→code | our queries | Russian hit@1 |
53
  |---|--:|--:|--:|--:|--:|--:|--:|
54
- | **code-daemon-embed-v1** | **4** | **3 457** | 18.5 | 41.7 | 30.7 | 42.7 | 0.00 |
55
  | multilingual-e5-base | 12 | 854 | 29.8 | 79.4 | 57.7 | 60.7 | **0.24** |
56
  | CodeRankEmbed | 12 | 723 | 35.5 | 73.3 | — | 64.6 | 0.00 |
57
  | gte-modernbert-base *(its teacher)* | 22 | 606 | **36.6** | **81.4** | **82.5** | **70.5** | 0.15 |
58
 
59
- - **4.0–5.7× the throughput for 18–28 points of nDCG.** Pick it when indexing speed is the
60
  constraint and a lexical channel runs beside it; pick `gte-modernbert-base` when vector-only
61
  accuracy is.
62
  - **Inside the hybrid search it was built for** (80 real agent queries, file level, runtime
63
- reranker off): the right file is first **46 %** of the time and in the top 5 **78 %**.
64
- - **Its own depth trade:** against its 6-layer parent it gives up ~4 % relative hit@1 for ~1.5×
65
- the engine speed.
66
  - **Not multilingual.** The vocabulary prune kept 942 of XLM-R's 31 671 Cyrillic pieces; for
67
  non-English queries use `multilingual-e5-base`.
68
- - **Not for long text, docstring paraphrase or general prose.** On the full CoIR suite (six
69
- in-scope tasks) it averages 37.3 nDCG@10 — 68.3 on Python docstring→code, 17.5 on cosqa.
70
 
71
  ### Throughput in a real index
72
 
 
51
 
52
  | model | layers | texts/s | cosqa | stack-qa | code→code | our queries | Russian hit@1 |
53
  |---|--:|--:|--:|--:|--:|--:|--:|
54
+ | **code-daemon-embed-v1** | **4** | **3 457** | 28.5 | 49.9 | 42.9 | 60.1 | 0.06 |
55
  | multilingual-e5-base | 12 | 854 | 29.8 | 79.4 | 57.7 | 60.7 | **0.24** |
56
  | CodeRankEmbed | 12 | 723 | 35.5 | 73.3 | — | 64.6 | 0.00 |
57
  | gte-modernbert-base *(its teacher)* | 22 | 606 | **36.6** | **81.4** | **82.5** | **70.5** | 0.15 |
58
 
59
+ - **4.0–5.7× the throughput for 1–10 points of nDCG on our queries.** Pick it when indexing speed is the
60
  constraint and a lexical channel runs beside it; pick `gte-modernbert-base` when vector-only
61
  accuracy is.
62
  - **Inside the hybrid search it was built for** (80 real agent queries, file level, runtime
63
+ reranker off): the right file is first **44 %** of the time and in the top 5 **69 %**.
 
 
64
  - **Not multilingual.** The vocabulary prune kept 942 of XLM-R's 31 671 Cyrillic pieces; for
65
  non-English queries use `multilingual-e5-base`.
66
+ - **Not for long text or general prose.** On four CoIR tasks it averages 52.8 nDCG@10 — 89.8
67
+ on Python docstring→code, 28.5 on cosqa.
68
 
69
  ### Throughput in a real index
70