shikharshahi/sifter-redrob-reranker

Sifter Redrob Reranker

This is the first trained reranker for Sifter, an AI hiring-ranking system built for the Redrob challenge.

The model reads a job description and one candidate profile together, then predicts a 0-1 fit score. In Sifter, it is used as a learned second opinion on the finalist pool after the full 100,000-candidate explainable ranker has already run.

Project repo: Sifter_Redrob_Hackathon Live app: https://sifter1011.web.app

What This Model Does

Sifter already has a deterministic evidence ranker that can process the full Redrob candidate pool locally. This model adds a trainable layer on top:

1. 1.Sifter ranks the full candidate pool using explainable evidence.
2. 2.The backend sends only the finalist pool to this Hugging Face model.
3. 3.The model returns a learned fit score.
4. 4.Sifter blends the scores and keeps the explanation/bias guardrails visible.

Current blend in the Sifter backend:

70% explainable Sifter evidence score
30% learned reranker score

Default rerank scope:

top 25 finalist candidates

Training Data

This revised public model was trained on Redrob-derived Sifter preference data with human-reviewed recruiter-style labels, not on a generic public ranking benchmark.

Training run:

Each training example is shaped like this:

Job description + candidate profile -> fit score

The candidate profile text includes title, summary/headline, years of experience, location, career history, skills, certifications, assessments, and Redrob behavioral/logistics signals.

Label Scale

The revised run uses human-reviewed labels so the model learns from actual recruiter-style judgment instead of only bootstrapped scores.

Recruiter labels are supported by the training script and override weak labels when present:

Important: these labels are stronger than weak supervision, but they are still a compact review set. The next stronger version should add more reviewers and a separate held-out recruiter panel.

Metrics

Validation results from the human-reviewed revised run:

What Spearman means in plain language: when the human-reviewed labels say candidate A should usually rank above candidate B, the model's scores mostly move in the same direction. 0.7526 is a strong sign that the learned reranker is now aligned with the reviewed candidate judgments.

Training Procedure

Base model:

distilbert-base-uncased

Fine-tuning method:

Supervised reward-model regression fine-tuning

Training setup:

The model head is a single regression output (num_labels=1) trained with mean squared error loss.

Why This Is Still Human-In-The-Loop

This model is not treated as an automatic hiring decision system. The reviewed-label run improves the learned ranking signal, but Sifter still keeps human-facing checks:

• —every rank still shows evidence and concern text,
• —the bias guardrail stays visible,
• —reviewer-agent questions challenge the result,
• —recruiters can add more labels for future retraining.

How It Is Integrated Into Sifter

The model is wired into the Sifter backend:

The model is not allowed to become an unchecked black box. The deterministic Sifter reason, score breakdown, bias guardrail, and reviewer-agent questions remain visible after reranking.

Limitations

• —The model is trained for the Redrob/Sifter Senior AI Engineer ranking setup, not general hiring across every role.
• —The revised run uses 180 human-reviewed examples, so it is stronger than weak supervision but still small.
• —The validation metric is measured on a 14-row reviewed validation split, not a large independent recruiter panel.
• —The model can learn patterns present in the review labels, so Sifter keeps deterministic explanations and bias guardrails in the final product.
• —The Redrob dataset does not include protected demographic labels, so this model card does not claim protected-class fairness parity.

Responsible Use

Use this model as a recruiter-assist reranker, not as an automatic hiring decision system. It should support human review by providing an additional fit signal while Sifter continues to show evidence, concerns, and bias checks.

Recommended use:

• —rerank finalist pools,
• —compare candidate-job fit,
• —support interview shortlist review,
• —collect recruiter labels for a better second version.

Not recommended:

• —automatic rejection without human review,
• —ranking based on identity or protected traits,
• —claiming fairness parity without a protected-label audit,
• —using the score without reading the explanation and evidence.