talentguide/talentclef-biencoder-v1
SkillScout Large — Job-to-Skill Dense Retriever
SkillScout Large is a dense bi-encoder for retrieving relevant skills from a job title. Given a job title (e.g., "Data Scientist"), it encodes it into a 1024-dimensional embedding and retrieves the most semantically relevant skills from the ESCO skill gazetteer (9,052 skills) using cosine similarity.
This is Stage 1 of the TalentGuide two-stage job-skill matching pipeline, trained for TalentCLEF 2026 Task B.
Best pipeline result (TalentCLEF 2026 validation set): nDCG@10 graded = 0.6896 · nDCG@10 binary = 0.7330 when combined with a fine-tuned cross-encoder re-ranker at blend α = 0.7. Bi-encoder alone: nDCG@10 graded = 0.3621 · MAP = 0.4545
Model Summary
What is TalentCLEF Task B?
TalentCLEF 2026 Task B is a graded information-retrieval shared task:
- Query: a job title (e.g., "Electrician")
- Corpus: 9,052 ESCO skills (e.g., "install electric switches", "comply with electrical safety regulations")
- Relevance levels:
2— Core skill (essential regardless of context)1— Contextual skill (depends on employer / industry)0— Non-relevant
Primary metric: nDCG with graded relevance (core=2, contextual=1)
Usage
Installation
pip install sentence-transformers faiss-cpu # or faiss-gpuEncode & Compare
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("talentguide/skillscout-large")
job = "Data Scientist"
skills = ["data science", "machine learning", "install electric switches"]
embs = model.encode([job] + skills, normalize_embeddings=True)
scores = embs[0] @ embs[1:].T
for skill, score in zip(skills, scores):
print(f"{score:.3f} {skill}")
# 0.872 data science
# 0.731 machine learning
# 0.112 install electric switchesFull Retrieval with FAISS (Recommended)
from sentence_transformers import SentenceTransformer
import faiss, numpy as np
model = SentenceTransformer("talentguide/skillscout-large")
# --- Build index once over your skill corpus ---
skill_texts = [...] # list of skill names / descriptions
embs = model.encode(skill_texts, batch_size=128,
normalize_embeddings=True,
show_progress_bar=True).astype(np.float32)
index = faiss.IndexFlatIP(embs.shape[1]) # inner product on L2-normed = cosine
index.add(embs)
# --- Query at inference time ---
job_title = "Software Engineer"
q = model.encode([job_title], normalize_embeddings=True).astype(np.float32)
scores, idxs = index.search(q, k=50)
for rank, (idx, score) in enumerate(zip(idxs[0], scores[0]), 1):
print(f"{rank:3d}. [{score:.4f}] {skill_texts[idx]}")Demo Output
Software Engineer
1. [0.942] define software architecture
2. [0.938] software frameworks
3. [0.935] create software design
Data Scientist
1. [0.951] data science
2. [0.921] establish data processes
3. [0.919] create data models
Electrician
1. [0.944] install electric switches
2. [0.938] install electricity sockets
3. [0.930] use electrical wire toolsTwo-Stage Pipeline Integration
SkillScout Large is designed as Stage 1 — fast ANN retrieval. For maximum ranking quality, pair it with a cross-encoder re-ranker:
Job title
│
▼
[SkillScout Large] ← this model
│ top-200 candidates (FAISS ANN, ~40ms)
▼
[Cross-encoder re-ranker]
│ fine-grained re-scoring of top-200
▼
Final ranked list (graded: core > contextual > irrelevant)Score blending (best result at α = 0.7):
final_score = alpha * biencoder_score + (1 - alpha) * crossencoder_scoreTraining Details
Data
Source: ESCO occupational ontology, TalentCLEF 2026 training split.
Essential pairs are included in full; optional skill pairs are downsampled to 50% of the essential count to maintain class balance.
Hyperparameters
Loss : MultipleNegativesRankingLoss (scale=20, cos_sim)
Batch size : 64 → 63 in-batch negatives per anchor
Epochs : 3
Warmup : 10% of total steps (~440 steps)
Optimizer : AdamW (fused), lr=5e-5, linear decay
Precision : fp16 (AMP)
Max seq length : 64 tokens
Best model saved : by cosine-nDCG@10 on validation (eval every 500 steps)
Seed : 42Training Curve
Best checkpoint saved at step 3500.
Validation Metrics (best checkpoint, binary relevance)
Evaluated with `sentence_transformers.evaluation.InformationRetrievalEvaluator` (binary: any qrel > 0 = relevant).
Pipeline Results (graded nDCG, full 9052-skill ranking, server-side)
Competitive Context (TalentCLEF 2025 Task B)
Limitations
- English only — trained on ESCO EN labels.
- ESCO-domain — optimised for the ESCO skill taxonomy; performance on other taxonomies (O*NET, custom) may vary without fine-tuning.
- 64-token cap — long job descriptions should be reduced to a concise title before encoding.
- Graded distinction — the bi-encoder alone does not reliably separate core (2) from contextual (1) skills; a cross-encoder re-ranker is needed for strong graded nDCG.
Citation
@misc{talentguide-skillscout-2026,
title = {SkillScout Large: Dense Job-to-Skill Retrieval for TalentCLEF 2026},
author = {TalentGuide},
year = {2026},
url = {https://huggingface.co/talentguide/skillscout-large}
}
@misc{talentclef2026taskb,
title = {TalentCLEF 2026 Task B: Job-Skill Matching},
author = {TalentCLEF Organizers},
year = {2026},
url = {https://talentclef.github.io/}
}