---
datasets:
- minishlab/tokenlearn-cornstack-docs-coderankembed-v2
- minishlab/tokenlearn-cornstack-queries-coderankembed-v2
language:
- code
library_name: model2vec
license: mit
pipeline_tag: feature-extraction
tags:
- embeddings
- static-embeddings
- code-retrieval
- tokenlearn
- coir
model-index:
- name: miru-codev3-tokenlearn
  results:
  - task:
      type: Retrieval
      name: CoIR (dense)
    dataset:
      name: CoIR average (6 tasks) [dense]
      type: coir-dense/average
    metrics:
    - type: ndcg_at_10
      value: 38.26
      name: NDCG@10
    source:
      url: https://github.com/CoIR-team/coir
      name: CoIR / MTEB (self-reported)
  - task:
      type: Retrieval
      name: CoIR (dense)
    dataset:
      name: AppsRetrieval [dense]
      type: coir-dense/AppsRetrieval
    metrics:
    - type: ndcg_at_10
      value: 4.23
      name: NDCG@10
    source:
      url: https://github.com/CoIR-team/coir
      name: CoIR / MTEB (self-reported)
  - task:
      type: Retrieval
      name: CoIR (dense)
    dataset:
      name: COIRCodeSearchNet [dense]
      type: coir-dense/COIRCodeSearchNetRetrieval
    metrics:
    - type: ndcg_at_10
      value: 44.93
      name: NDCG@10
    source:
      url: https://github.com/CoIR-team/coir
      name: CoIR / MTEB (self-reported)
  - task:
      type: Retrieval
      name: CoIR (dense)
    dataset:
      name: CodeFeedbackST [dense]
      type: coir-dense/CodeFeedbackST
    metrics:
    - type: ndcg_at_10
      value: 53.04
      name: NDCG@10
    source:
      url: https://github.com/CoIR-team/coir
      name: CoIR / MTEB (self-reported)
  - task:
      type: Retrieval
      name: CoIR (dense)
    dataset:
      name: CodeSearchNetCC [dense]
      type: coir-dense/CodeSearchNetCCRetrieval
    metrics:
    - type: ndcg_at_10
      value: 42.56
      name: NDCG@10
    source:
      url: https://github.com/CoIR-team/coir
      name: CoIR / MTEB (self-reported)
  - task:
      type: Retrieval
      name: CoIR (dense)
    dataset:
      name: CosQA [dense]
      type: coir-dense/CosQA
    metrics:
    - type: ndcg_at_10
      value: 25.76
      name: NDCG@10
    source:
      url: https://github.com/CoIR-team/coir
      name: CoIR / MTEB (self-reported)
  - task:
      type: Retrieval
      name: CoIR (dense)
    dataset:
      name: StackOverflowQA [dense]
      type: coir-dense/StackOverflowQA
    metrics:
    - type: ndcg_at_10
      value: 59.04
      name: NDCG@10
    source:
      url: https://github.com/CoIR-team/coir
      name: CoIR / MTEB (self-reported)
---

# miru-codev3-tokenlearn

Owned **Tokenlearn** static code embedding bag (256-d, Model2Vec) — branch **A** of [miru-codev3-dual](https://huggingface.co/Takara-DS1/miru-codev3-dual).

Trained with Zipf-SIF Tokenlearn on CornStack coderankembed-v2 features (`code_static.train_miru_codev3`).

## Evaluation

CoIR / MTEB **NDCG@10** (×100), dense retrieval only. Scores are self-reported from the training repo eval harness.

## Usage

```python
from model2vec import StaticModel

model = StaticModel.from_pretrained("Takara-DS1/miru-codev3-tokenlearn")
emb = model.encode(["def add(a, b): return a + b"])
```

For the 512-d dual (this bag ⊕ distill_fuse), use [Takara-DS1/miru-codev3-dual](https://huggingface.co/Takara-DS1/miru-codev3-dual).

## Recreate

See the dual repo `reproduce/REPRODUCE.md` (step 1: `train_miru_codev3`).

## License

MIT