| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3-Embedding-0.6B |
| library_name: sentence-transformers |
| pipeline_tag: feature-extraction |
| tags: |
| - transformers |
| - sentence-similarity |
| - retrieval |
| - agent-skills |
| - skill-routing |
| - skillcorpus |
| - contrastive-learning |
| - text-embeddings-inference |
| language: |
| - en |
| --- |
| |
| # skillrep-embedding-0.6b |
|
|
| A bi-encoder for **agent-skill retrieval**: given a task description, embed it and |
| the skill documents into one space so the relevant skills come back by nearest |
| neighbour. Fine-tuned from |
| [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B). |
|
|
| It indexes a skill by its **body** only, unlike |
| [skillcorpus-embedding-0.6b](https://huggingface.co/EverMind-AI/skillcorpus-embedding-0.6b), |
| which indexes `name | description | body`. Skill documents follow the schema of |
| [skillcorpus-demo-1k](https://huggingface.co/datasets/EverMind-AI/skillcorpus-demo-1k). |
|
|
| | Property | Value | |
| |---|---| |
| | Parameters | 596M | |
| | Hidden size | 1024 (embedding dimension) | |
| | Layers | 28 | |
| | Precision | bfloat16 | |
| | Pooling | last token | |
| | Normalization | L2 | |
|
|
| Requires `transformers>=4.56` (the `dtype=` argument was named `torch_dtype=` |
| before that) or `sentence-transformers>=3.0`. |
|
|
| ## Usage |
|
|
| The two sides are encoded asymmetrically: a task description carries an |
| instruction prefix followed by one space, and a skill is its bare body. Encode |
| them this way or retrieval quality drops. Embeddings come back L2-normalized, so |
| cosine similarity is a plain dot product. |
|
|
| ```python |
| import torch |
| import torch.nn.functional as F |
| from transformers import AutoModel, AutoTokenizer |
| |
| MODEL = "EverMind-AI/skillrep-embedding-0.6b" |
| tok = AutoTokenizer.from_pretrained(MODEL, padding_side="left") |
| model = AutoModel.from_pretrained(MODEL, dtype=torch.bfloat16).cuda().eval() |
| |
| QUERY_INSTRUCTION = ( |
| "Instruct: Given a coding task description, retrieve the most relevant " |
| "skill document that would help an agent complete the task\nQuery: " |
| ) |
| |
| |
| def last_token_pool(hidden, attention_mask): |
| if attention_mask[:, -1].sum() == attention_mask.shape[0]: # left padding |
| return hidden[:, -1] |
| idx = attention_mask.sum(dim=1) - 1 |
| return hidden[torch.arange(hidden.shape[0], device=hidden.device), idx] |
| |
| |
| def embed(texts, max_length=4096): |
| enc = tok(texts, padding=True, truncation=True, |
| max_length=max_length, return_tensors="pt").to(model.device) |
| with torch.no_grad(): |
| out = model(**enc).last_hidden_state |
| return F.normalize(last_token_pool(out, enc["attention_mask"]), p=2, dim=1) |
| |
| |
| query = embed([QUERY_INSTRUCTION + "resolve conflicts after a git merge"]) |
| docs = embed(["# Resolving merge conflicts\n1. Run `git status` to list conflicted files ...", |
| "# Sourdough\nMix flour, water and starter ..."]) |
| print((query @ docs.T).tolist()) |
| ``` |
|
|
| Exact scores shift in the last decimal with dtype, hardware and batch |
| composition; the ordering is what matters. |
|
|
| ### With sentence-transformers |
|
|
| The repo ships a sentence-transformers configuration (last-token pooling + L2 |
| normalization, and the query instruction registered as the `query` prompt), so |
| this path is equivalent to the code above, up to bf16 noise: |
|
|
| ```python |
| from sentence_transformers import SentenceTransformer |
| |
| st = SentenceTransformer("EverMind-AI/skillrep-embedding-0.6b") |
| q = st.encode(["resolve conflicts after a git merge"], prompt_name="query") |
| d = st.encode(["# Resolving merge conflicts\n1. Run `git status` to list conflicted files ...", |
| "# Sourdough\nMix flour, water and starter ..."]) |
| print(st.similarity(q, d)) |
| ``` |
|
|
| Pass `prompt_name="query"` for tasks and nothing for skill documents. |
|
|
| ### Input lengths |
|
|
| | input | limit | |
| |---|---| |
| | task description (after the instruction prefix) | not cut by characters; 4,096 tokens | |
| | skill `body` | first 8,000 characters; 4,096 tokens | |
|
|
| ## Intended use |
|
|
| First-stage retrieval over a large skill registry: encode the registry offline, |
| encode each incoming task online and take the top K by cosine similarity. Not a |
| generative model: it produces embeddings, not answers. |
|
|
| ## License |
|
|
| Apache-2.0, inherited from the base model. Skills keep their own upstream |
| licenses. |
|
|
