| --- |
| datasets: |
| - minishlab/tokenlearn-cornstack-docs-coderankembed-v2 |
| - minishlab/tokenlearn-cornstack-queries-coderankembed-v2 |
| language: |
| - code |
| library_name: model2vec |
| license: mit |
| pipeline_tag: feature-extraction |
| tags: |
| - embeddings |
| - static-embeddings |
| - code-retrieval |
| - tokenlearn |
| - coir |
| model-index: |
| - name: miru-codev3-tokenlearn |
| results: |
| - task: |
| type: Retrieval |
| name: CoIR (dense) |
| dataset: |
| name: CoIR average (6 tasks) [dense] |
| type: coir-dense/average |
| metrics: |
| - type: ndcg_at_10 |
| value: 38.26 |
| name: NDCG@10 |
| source: |
| url: https://github.com/CoIR-team/coir |
| name: CoIR / MTEB (self-reported) |
| - task: |
| type: Retrieval |
| name: CoIR (dense) |
| dataset: |
| name: AppsRetrieval [dense] |
| type: coir-dense/AppsRetrieval |
| metrics: |
| - type: ndcg_at_10 |
| value: 4.23 |
| name: NDCG@10 |
| source: |
| url: https://github.com/CoIR-team/coir |
| name: CoIR / MTEB (self-reported) |
| - task: |
| type: Retrieval |
| name: CoIR (dense) |
| dataset: |
| name: COIRCodeSearchNet [dense] |
| type: coir-dense/COIRCodeSearchNetRetrieval |
| metrics: |
| - type: ndcg_at_10 |
| value: 44.93 |
| name: NDCG@10 |
| source: |
| url: https://github.com/CoIR-team/coir |
| name: CoIR / MTEB (self-reported) |
| - task: |
| type: Retrieval |
| name: CoIR (dense) |
| dataset: |
| name: CodeFeedbackST [dense] |
| type: coir-dense/CodeFeedbackST |
| metrics: |
| - type: ndcg_at_10 |
| value: 53.04 |
| name: NDCG@10 |
| source: |
| url: https://github.com/CoIR-team/coir |
| name: CoIR / MTEB (self-reported) |
| - task: |
| type: Retrieval |
| name: CoIR (dense) |
| dataset: |
| name: CodeSearchNetCC [dense] |
| type: coir-dense/CodeSearchNetCCRetrieval |
| metrics: |
| - type: ndcg_at_10 |
| value: 42.56 |
| name: NDCG@10 |
| source: |
| url: https://github.com/CoIR-team/coir |
| name: CoIR / MTEB (self-reported) |
| - task: |
| type: Retrieval |
| name: CoIR (dense) |
| dataset: |
| name: CosQA [dense] |
| type: coir-dense/CosQA |
| metrics: |
| - type: ndcg_at_10 |
| value: 25.76 |
| name: NDCG@10 |
| source: |
| url: https://github.com/CoIR-team/coir |
| name: CoIR / MTEB (self-reported) |
| - task: |
| type: Retrieval |
| name: CoIR (dense) |
| dataset: |
| name: StackOverflowQA [dense] |
| type: coir-dense/StackOverflowQA |
| metrics: |
| - type: ndcg_at_10 |
| value: 59.04 |
| name: NDCG@10 |
| source: |
| url: https://github.com/CoIR-team/coir |
| name: CoIR / MTEB (self-reported) |
| --- |
| |
| # miru-codev3-tokenlearn |
|
|
| Owned **Tokenlearn** static code embedding bag (256-d, Model2Vec) — branch **A** of [miru-codev3-dual](https://huggingface.co/Takara-DS1/miru-codev3-dual). |
|
|
| Trained with Zipf-SIF Tokenlearn on CornStack coderankembed-v2 features (`code_static.train_miru_codev3`). |
|
|
| ## Evaluation |
|
|
| CoIR / MTEB **NDCG@10** (×100), dense retrieval only. Scores are self-reported from the training repo eval harness. |
|
|
| ## Usage |
|
|
| ```python |
| from model2vec import StaticModel |
| |
| model = StaticModel.from_pretrained("Takara-DS1/miru-codev3-tokenlearn") |
| emb = model.encode(["def add(a, b): return a + b"]) |
| ``` |
|
|
| For the 512-d dual (this bag ⊕ distill_fuse), use [Takara-DS1/miru-codev3-dual](https://huggingface.co/Takara-DS1/miru-codev3-dual). |
| |
| ## Recreate |
| |
| See the dual repo `reproduce/REPRODUCE.md` (step 1: `train_miru_codev3`). |
| |
| ## License |
| |
| MIT |
| |
| |
