Summarise benchmark wins at the top of the results
Browse files
README.md
CHANGED
|
@@ -63,6 +63,11 @@ Training and evaluation code, the benchmarks and an MCP server for coding agents
|
|
| 63 |
|
| 64 |
## Results
|
| 65 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
All numbers come from held-out test data that no training or model-selection step used. Each model was scored once,
|
| 67 |
with the same protocol. The comparison models are open models of similar size, evaluated zero-shot.
|
| 68 |
|
|
|
|
| 63 |
|
| 64 |
## Results
|
| 65 |
|
| 66 |
+
CodeIntel-150M is the best open model of its size on **7 of 15 benchmarks**, including all three issue-localization
|
| 67 |
+
benchmarks, test-to-implementation retrieval, TypeScript call-site navigation, CoIR StackOverflow QA and
|
| 68 |
+
CodeSearchNet Python. The full per-model table is in
|
| 69 |
+
[docs/results/benchmarks.md](https://github.com/vedevpatel/codeintel-150m/blob/main/docs/results/benchmarks.md).
|
| 70 |
+
|
| 71 |
All numbers come from held-out test data that no training or model-selection step used. Each model was scored once,
|
| 72 |
with the same protocol. The comparison models are open models of similar size, evaluated zero-shot.
|
| 73 |
|