EverMind-AI/skillcorpus-embedding-0.6b
skillcorpus-embedding-0.6b
A bi-encoder for agent-skill retrieval: given a task description, embed it and the skill documents into one space so the relevant skills come back by nearest neighbour. Fine-tuned from Qwen/Qwen3-Embedding-0.6B.
Paired with skillcorpus-reranker-0.6b, which reranks this model's top candidates. The skill documents it was built to index have the schema of skillcorpus-demo-1k.
Requires transformers>=4.56 (the dtype= argument was named torch_dtype= before that) or sentence-transformers>=3.0.
Usage
The two sides are encoded asymmetrically — a task description carries an instruction prefix, a skill is the bare name | description | body concatenation. Encode them the way the model was trained or retrieval quality drops. Embeddings come back L2-normalized, so cosine similarity is a plain dot product.
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer
MODEL = "EverMind-AI/skillcorpus-embedding-0.6b"
tok = AutoTokenizer.from_pretrained(MODEL, padding_side="left")
model = AutoModel.from_pretrained(MODEL, dtype=torch.bfloat16).cuda().eval()
QUERY_INSTRUCTION = (
"Instruct: Given a task description, retrieve the most relevant "
"skill document that would help an agent complete the task\nQuery:"
)
def doc(name, description, body):
return f"{name} | {description} | {body}"
def last_token_pool(hidden, attention_mask):
if attention_mask[:, -1].sum() == attention_mask.shape[0]: # left padding
return hidden[:, -1]
idx = attention_mask.sum(dim=1) - 1
return hidden[torch.arange(hidden.shape[0], device=hidden.device), idx]
def embed(texts, max_length=2048):
enc = tok(texts, padding=True, truncation=True,
max_length=max_length, return_tensors="pt").to(model.device)
with torch.no_grad():
out = model(**enc).last_hidden_state
return F.normalize(last_token_pool(out, enc["attention_mask"]), p=2, dim=1)
query = embed([QUERY_INSTRUCTION + "resolve conflicts after a git merge"])
docs = embed([doc("resolve-conflicts", "Resolve git merge conflicts.", "..."),
doc("sourdough", "Bake sourdough bread.", "...")])
print((query @ docs.T).tolist())
# -> [[0.75, 0.07]]Exact scores shift in the last decimal with dtype and hardware; the ordering is what matters.
With sentence-transformers
The repo ships a sentence-transformers configuration (last-token pooling + L2 normalization, and the query instruction registered as the query prompt), so this path is equivalent to the code above, up to bf16 noise:
from sentence_transformers import SentenceTransformer
st = SentenceTransformer("EverMind-AI/skillcorpus-embedding-0.6b")
q = st.encode(["resolve conflicts after a git merge"], prompt_name="query")
d = st.encode(["resolve-conflicts | Resolve git merge conflicts. | ...",
"sourdough | Bake sourdough bread. | ..."])
print(st.similarity(q, d))Pass prompt_name="query" for tasks and nothing for skill documents — that is the asymmetry above, applied for you.
Truncation used in training
Beyond the token-level max_length, each field was cut to a fixed number of characters before the strings were assembled. Matching this keeps inference inputs on the same distribution as training:
Intended use
First-stage retrieval over a large skill registry: encode the registry offline, encode each incoming task online, take the top K by cosine similarity (K in the 20–50 range is typical), then rerank that shortlist with skillcorpus-reranker-0.6b. Not a generative model — it produces embeddings, not answers.
Citation
@article{wang2026skillcorpus,
title = {SkillCorpus: Consolidating and Evaluating the Open Skill Ecosystem for Real-World LLM Agents},
author = {Wang, Yanze and Yao, Pengfei and Sun, Tianyi and Hu, Chuanrui and Xiao, Yan and Luo, Xiaotian and Han, Yunyun and Chen, Yifan and Sun, Jun and Deng, Yafeng},
year = {2026},
eprint = {2607.15557},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.15557}
}License
Apache-2.0, inherited from the base model. Skills in the corpus keep their own upstream licenses.