Blame for README.md

---
license: apache-2.0
language:
  - en
  - code
library_name: sentence-transformers
pipeline_tag: sentence-similarity
base_model: ibm-granite/granite-embedding-english-r2
tags:
  - sentence-transformers
  - feature-extraction
  - code-search
  - code-retrieval
  - bug-localization
  - modernbert
  - onnx
model-index:
  - name: CodeIntel-150M
    results:
      - task: {type: retrieval, name: Issue -> code localization (descriptive issues, Python)}
        dataset: {type: custom, name: codeintel issue localization (35 repositories)}
        metrics: [{type: ndcg_at_10, value: 0.268, name: symbol nDCG@10 (hybrid)}]
      - task: {type: retrieval, name: Issue -> code localization (descriptive issues, TypeScript)}
        dataset: {type: custom, name: codeintel issue localization (35 repositories)}
        metrics: [{type: ndcg_at_10, value: 0.274, name: symbol nDCG@10 (hybrid)}]
      - task: {type: retrieval, name: Test -> implementation (Python)}
        dataset: {type: custom, name: codeintel code-to-code}
        metrics: [{type: ndcg_at_10, value: 0.640}]
      - task: {type: retrieval, name: CoSQA}
        dataset: {type: CoIR-Retrieval/cosqa, name: CoSQA}
        metrics: [{type: ndcg_at_10, value: 0.651}]
      - task: {type: retrieval, name: CoIR StackOverflow QA}
        dataset: {type: CoIR-Retrieval/stackoverflow-qa, name: StackOverflow QA}
        metrics: [{type: ndcg_at_10, value: 0.922}]
      - task: {type: retrieval, name: CoIR CodeSearchNet (Python)}
        dataset: {type: CoIR-Retrieval/CodeSearchNet, name: CodeSearchNet Python}
        metrics: [{type: ndcg_at_10, value: 0.963}]
---

# CodeIntel-150M

CodeIntel-150M is a 149M-parameter embedding model for **issue-to-code localization**: given a bug report or feature
request, it retrieves the functions and classes in a Python or TypeScript repository that a fix is likely to
modify. On issues that describe only symptoms, with no reference to the relevant code, it outperforms every other
open embedding model of comparable size by **roughly 55%**. It is also **27% better** on issues that reference code directly
and **23% better** at test-to-implementation retrieval.

Beyond localization, it supports call-site-to-definition navigation and code search, where it leads its size class on CoIR StackOverflow QA
and CodeSearchNet Python. It is a fine-tune of
[granite-embedding-english-r2](https://huggingface.co/ibm-granite/granite-embedding-english-r2) (ModernBERT,
Apache-2.0). It runs locally, including as a 325 MB INT8 ONNX model, and needs no remote code.

Training and evaluation code, the benchmarks and an MCP server for coding agents are at
[github.com/vedevpatel/codeintel-150m](https://github.com/vedevpatel/codeintel-150m).

| | |
|---|---|
| Parameters | 149M |
| Embedding | 768-d, CLS pooling, L2-normalised (use cosine / dot product) |
| Max length | trained at 512 tokens per document, evaluated at 1,024 (base supports 8,192) |
| Languages | English text; Python and TypeScript code (other languages untested) |
| Licence | Apache-2.0 |

## Results

CodeIntel-150M is the best open model of its size on **7 of 15 benchmarks**, including all three issue-localization
benchmarks, test-to-implementation retrieval, TypeScript call-site navigation, CoIR StackOverflow QA and
CodeSearchNet Python. The full per-model table is in
[docs/results/benchmarks.md](https://github.com/vedevpatel/codeintel-150m/blob/main/docs/results/benchmarks.md).

All numbers come from held-out test data that no training or model-selection step used. Each model was scored once,
with the same protocol. The comparison models are open models of similar size, evaluated zero-shot.

**Issue -> code localization.** The test set has 3,804 GitHub issues from 35 Python and TypeScript repositories, none
of which were used in training. A hit means retrieving a function or class that the issue's fix edited, among all
definitions in the repository at the issue's commit. "Descriptive" issues name no identifier from the fix (for example
"server ignores the port when debug is on"). "Lexical" issues do name one. The metric is symbol-level nDCG@10.

| model | descriptive, Python | descriptive, TypeScript | lexical |
|---|---|---|---|
| **CodeIntel-150M**, dense | **0.242** | **0.260** | **0.482** |
| **CodeIntel-150M**, + BM25 (RRF, dense weight 2) | **0.268** | **0.274** | **0.516** |
| granite-embedding-english-r2, dense | 0.166 | 0.161 | 0.346 |
| bge-base-en-v1.5, dense | 0.163 | 0.166 | 0.356 |
| gte-modernbert-base, dense | 0.125 | 0.112 | 0.308 |
| CodeRankEmbed, dense | 0.094 | 0.095 | 0.292 |
| BM25 | 0.120 | 0.133 | 0.371 |

The best zero-shot model, each with BM25 fusion at its own dev-chosen weight, scores 0.173 / 0.178 descriptive. The
gain over it is +0.095 in both languages, with repository-clustered 95% intervals excluding zero.

**Code-to-code and general code search.** Dense nDCG@10. For each benchmark, the right-hand column gives the best of
granite-embedding-english-r2, bge-base-en-v1.5, modernbert-embed-base, gte-modernbert-base, CodeRankEmbed,
jina-embeddings-v2-base-code and codesage-small-v2.

| benchmark | CodeIntel-150M | best other open ~150M |
|---|---|---|
| test -> implementation (Python) | **0.640** | 0.522 (CodeRankEmbed) |
| call site -> definition, Python / TypeScript | 0.219 / **0.278** | 0.244 (bge-base) / 0.264 (granite-r2) |
| docstring -> function, Python / TypeScript | 0.717 / 0.672 | 0.863 / 0.762 (CodeRankEmbed) |
| CoSQA | 0.651 | 0.737 (jina-v2-code) |
| CoIR StackOverflow QA | **0.922** | 0.917 (granite-r2) |
| CoIR CodeSearchNet, Python / JavaScript | **0.963** / 0.634 | 0.960 (codesage) / 0.764 (jina-v2-code) |
| CoIR CodeSearchNet-CCR, Python / JavaScript | 0.703 / 0.688 | 0.940 / 0.943 (gte-modernbert) |
| CoIR APPS | 0.174 | 0.575 (gte-modernbert) |

**Choose CodeIntel-150M** for repository-scale retrieval driven by issues, bug reports or tests, for example as the
retriever in a coding agent. **Choose a general code-search model instead** (gte-modernbert-base, CodeRankEmbed,
jina-embeddings-v2-base-code) for docstring-style or web-query code search and code completion-style retrieval (CCR),
where CodeIntel-150M trails.

## Usage

Documents are code definitions prefixed with a one-line `path::QualifiedName` header in the language's comment
syntax, as in training. Queries are raw issue text; for best results remove template boilerplate, logs and long code
blocks first. No query or document prefix is needed.

```python
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("vedevpatel/CodeIntel-150M")
code = [
    "# src/flask/app.py::Flask.run\ndef run(self, host=None, port=None, debug=None):\n    ...",
    "# src/utils/date.ts::parseDate\nexport function parseDate(s: string): Date { return new Date(s); }",
]
query = "Server ignores the port argument when debug is enabled"
scores = model.encode([query]) @ model.encode(code).T   # cosine similarities (embeddings are normalised)
```

On a GPU, load in half precision (bf16 on Ampere or newer, fp16 otherwise); the default is float32 for CPU. With
`flash-attn` installed, `attn_implementation="flash_attention_2"` speeds up long inputs further.

```python
import torch

model = SentenceTransformer("vedevpatel/CodeIntel-150M", device="cuda", model_kwargs={"dtype": torch.bfloat16})
```

With plain transformers:

```python
import torch
from transformers import AutoModel, AutoTokenizer

tok = AutoTokenizer.from_pretrained("vedevpatel/CodeIntel-150M")
model = AutoModel.from_pretrained("vedevpatel/CodeIntel-150M").eval()
batch = tok(texts, padding=True, truncation=True, max_length=1024, return_tensors="pt")
with torch.no_grad():
    emb = torch.nn.functional.normalize(model(**batch).last_hidden_state[:, 0], dim=-1)
```

**ONNX / INT8.** `onnx/model.onnx` (fp32) and `onnx/model_int8.onnx` (dynamic INT8; the GeGLU down-projections stay
fp32). Outputs are the last hidden states; take the CLS vector and L2-normalise it. The INT8 model's minimum cosine
to PyTorch is 0.995. On dev retrieval its nDCG@10 matches fp32 within noise: CoSQA +0.003 [-0.005, +0.011],
docstrings-TypeScript -0.002 [-0.010, +0.006]. It is about 1.7x faster on CPU.

**Hybrid retrieval.** Fusing dense scores with BM25 by reciprocal-rank fusion (dense weight 2, k = 60) adds about
+0.03 on issue localization.

## Use with coding agents (MCP)

`codeintel mcp` gives Claude Code, Codex, Cursor and other MCP clients semantic search over the repository they are
working in: the agent passes an issue or a plain-language description and gets back ranked functions and classes with
file paths and line ranges. It runs locally over stdio, indexes the repository in the background and re-embeds only
changed code after edits.

```bash
# Claude Code (run in the repository)
claude mcp add codeintel -- uvx --from "codeintel[mcp] @ git+https://github.com/vedevpatel/codeintel-150m" codeintel mcp .
```

```toml
# Codex: ~/.codex/config.toml
[mcp_servers.codeintel]
command = "uvx"
args = ["--from", "codeintel[mcp] @ git+https://github.com/vedevpatel/codeintel-150m", "codeintel", "mcp", "."]
```

Setup for Cursor is in the [GitHub README](https://github.com/vedevpatel/codeintel-150m#use-with-coding-agents-mcp).

## Training

- **Objective:** multi-positive InfoNCE (symmetric, temperature 0.05) with BM25-mined hard negatives and GradCache
  (batch 128). Plus distillation (weight 0.5) toward the frozen base model's similarity structure, which preserves
  general search ability. 3,000 steps, learning rate 5e-6, bf16, maximum document length 512.
- **Data** (422k query groups; every source permissively licensed):
  - 82 GitHub repositories (MIT, Apache-2.0, BSD, ISC and similar; each licence verified at every revision code was
    taken from; none is an evaluation or development repository): issue -> edited function and commit -> edited
    function (issue text as originally opened where recoverable), docstring -> function, test -> implementation, call
    site -> definition, code context -> continuation.
  - [The Vault](https://huggingface.co/datasets/Fsoft-AIC/the-vault-function): docstring -> function and code-context
    pairs, only items whose declared licences are all permissive.
  - [CodeSearchNet](https://huggingface.co/datasets/code-search-net/code_search_net) train split (Python,
    JavaScript): code -> docstring, code -> continuation and docstring -> code. Rows kept only if the licence GitHub
    detects at the row's own commit is permissive (3,213 of the 4,000 largest repositories). CoIR's CodeSearchNet
    tasks are built from CodeSearchNet's test split, whose repositories are disjoint from the train split.

  All data was decontaminated against every evaluation and development text, including the CoIR corpora (exact and
  MinHash near-duplicates).
- **Sources and licences:** every repository and dataset used is listed in [TRAINING_DATA.md](TRAINING_DATA.md).
- **Selection:** the final checkpoint, chosen on a separate development set of 19 repositories by a fixed
  rule: maximise descriptive localization, subject to no loss on lexical localization, general search and
  code-to-code retrieval.
- **Compute:** one NVIDIA H100 80GB, about 1.6 hours (embedding cache 10 minutes, training 87 minutes).

## Limitations

- Issue localization remains hard. Even CodeIntel-150M puts an edited function in the top 10 for well under half of
  descriptive issues.
- Fixes that only add new code cannot be retrieved, because the target does not exist yet. The benchmark reports this
  as a recall ceiling.
- Evaluated on Python and TypeScript only. Behaviour on other languages is unknown.
- Docstring-style search and CoSQA are at the base model's level, below CodeRankEmbed and jina-embeddings-v2-base-code.
- General natural-language code search and code-continuation retrieval (CoIR CCR, APPS) remain below the best open
  models of this size.

## Citation

```bibtex
@misc{<CITATION_KEY>,
  title  = {CodeIntel-150M: an embedding model for issue-to-code localization},
  author = {<AUTHORS>},
  year   = {2026},
  url    = {https://huggingface.co/vedevpatel/CodeIntel-150M}
}
```