skillrep-embedding-0.6b

A bi-encoder for agent-skill retrieval: given a task description, embed it and the skill documents into one space so the relevant skills come back by nearest neighbour. Fine-tuned from Qwen/Qwen3-Embedding-0.6B.

It indexes a skill by its body only, unlike skillcorpus-embedding-0.6b, which indexes name | description | body. Skill documents follow the schema of skillcorpus-demo-1k.

Property	Value
Parameters	596M
Hidden size	1024 (embedding dimension)
Layers	28
Precision	bfloat16
Pooling	last token
Normalization	L2

Requires transformers>=4.56 (the dtype= argument was named torch_dtype= before that) or sentence-transformers>=3.0.

Usage

The two sides are encoded asymmetrically: a task description carries an instruction prefix followed by one space, and a skill is its bare body. Encode them this way or retrieval quality drops. Embeddings come back L2-normalized, so cosine similarity is a plain dot product.

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

MODEL = "EverMind-AI/skillrep-embedding-0.6b"
tok = AutoTokenizer.from_pretrained(MODEL, padding_side="left")
model = AutoModel.from_pretrained(MODEL, dtype=torch.bfloat16).cuda().eval()

QUERY_INSTRUCTION = (
    "Instruct: Given a coding task description, retrieve the most relevant "
    "skill document that would help an agent complete the task\nQuery: "
)


def last_token_pool(hidden, attention_mask):
    if attention_mask[:, -1].sum() == attention_mask.shape[0]:   # left padding
        return hidden[:, -1]
    idx = attention_mask.sum(dim=1) - 1
    return hidden[torch.arange(hidden.shape[0], device=hidden.device), idx]


def embed(texts, max_length=4096):
    enc = tok(texts, padding=True, truncation=True,
              max_length=max_length, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model(**enc).last_hidden_state
    return F.normalize(last_token_pool(out, enc["attention_mask"]), p=2, dim=1)


query = embed([QUERY_INSTRUCTION + "resolve conflicts after a git merge"])
docs = embed(["# Resolving merge conflicts\n1. Run `git status` to list conflicted files ...",
              "# Sourdough\nMix flour, water and starter ..."])
print((query @ docs.T).tolist())

Exact scores shift in the last decimal with dtype, hardware and batch composition; the ordering is what matters.

With sentence-transformers

The repo ships a sentence-transformers configuration (last-token pooling + L2 normalization, and the query instruction registered as the query prompt), so this path is equivalent to the code above, up to bf16 noise:

from sentence_transformers import SentenceTransformer

st = SentenceTransformer("EverMind-AI/skillrep-embedding-0.6b")
q = st.encode(["resolve conflicts after a git merge"], prompt_name="query")
d = st.encode(["# Resolving merge conflicts\n1. Run `git status` to list conflicted files ...",
               "# Sourdough\nMix flour, water and starter ..."])
print(st.similarity(q, d))

Pass prompt_name="query" for tasks and nothing for skill documents.

Input lengths

input	limit
task description (after the instruction prefix)	not cut by characters; 4,096 tokens
skill `body`	first 8,000 characters; 4,096 tokens

Intended use

First-stage retrieval over a large skill registry: encode the registry offline, encode each incoming task online and take the top K by cosine similarity. Not a generative model: it produces embeddings, not answers.