metadata
datasets:
- minishlab/tokenlearn-cornstack-docs-coderankembed-v2
- minishlab/tokenlearn-cornstack-queries-coderankembed-v2
language:
- code
library_name: model2vec
license: mit
pipeline_tag: feature-extraction
tags:
- embeddings
- static-embeddings
- code-retrieval
- tokenlearn
- coir
model-index:
- name: miru-codev3-tokenlearn
results:
- task:
type: Retrieval
name: CoIR (dense)
dataset:
name: CoIR average (6 tasks) [dense]
type: coir-dense/average
metrics:
- type: ndcg_at_10
value: 38.26
name: NDCG@10
source:
url: https://github.com/CoIR-team/coir
name: CoIR / MTEB (self-reported)
- task:
type: Retrieval
name: CoIR (dense)
dataset:
name: AppsRetrieval [dense]
type: coir-dense/AppsRetrieval
metrics:
- type: ndcg_at_10
value: 4.23
name: NDCG@10
source:
url: https://github.com/CoIR-team/coir
name: CoIR / MTEB (self-reported)
- task:
type: Retrieval
name: CoIR (dense)
dataset:
name: COIRCodeSearchNet [dense]
type: coir-dense/COIRCodeSearchNetRetrieval
metrics:
- type: ndcg_at_10
value: 44.93
name: NDCG@10
source:
url: https://github.com/CoIR-team/coir
name: CoIR / MTEB (self-reported)
- task:
type: Retrieval
name: CoIR (dense)
dataset:
name: CodeFeedbackST [dense]
type: coir-dense/CodeFeedbackST
metrics:
- type: ndcg_at_10
value: 53.04
name: NDCG@10
source:
url: https://github.com/CoIR-team/coir
name: CoIR / MTEB (self-reported)
- task:
type: Retrieval
name: CoIR (dense)
dataset:
name: CodeSearchNetCC [dense]
type: coir-dense/CodeSearchNetCCRetrieval
metrics:
- type: ndcg_at_10
value: 42.56
name: NDCG@10
source:
url: https://github.com/CoIR-team/coir
name: CoIR / MTEB (self-reported)
- task:
type: Retrieval
name: CoIR (dense)
dataset:
name: CosQA [dense]
type: coir-dense/CosQA
metrics:
- type: ndcg_at_10
value: 25.76
name: NDCG@10
source:
url: https://github.com/CoIR-team/coir
name: CoIR / MTEB (self-reported)
- task:
type: Retrieval
name: CoIR (dense)
dataset:
name: StackOverflowQA [dense]
type: coir-dense/StackOverflowQA
metrics:
- type: ndcg_at_10
value: 59.04
name: NDCG@10
source:
url: https://github.com/CoIR-team/coir
name: CoIR / MTEB (self-reported)
miru-codev3-tokenlearn
Owned Tokenlearn static code embedding bag (256-d, Model2Vec) — branch A of miru-codev3-dual.
Trained with Zipf-SIF Tokenlearn on CornStack coderankembed-v2 features (code_static.train_miru_codev3).
Evaluation
CoIR / MTEB NDCG@10 (×100), dense retrieval only. Scores are self-reported from the training repo eval harness.
Usage
from model2vec import StaticModel
model = StaticModel.from_pretrained("Takara-DS1/miru-codev3-tokenlearn")
emb = model.encode(["def add(a, b): return a + b"])
For the 512-d dual (this bag ⊕ distill_fuse), use Takara-DS1/miru-codev3-dual.
Recreate
See the dual repo reproduce/REPRODUCE.md (step 1: train_miru_codev3).
License
MIT