AdarshSingh7647/Eklav-14B-Reranker
Eklav-14B-Reranker
Eklav trains a model to pick up a teacher's reasoning mid thought rather than imitate it end to end. The student sees a partial reasoning trace from the teacher, with the answer revealing tail removed, and learns to continue reasoning and produce the answer on its own. The model's own reasoning is conditioned on the teacher's partial trace during training rather than trained to reproduce it word for word. Same base model, same training data as standard full trace CoT distillation, only the training objective changes.
Highlights
- +7% on BRIGHT (nDCG@10, 12 domain average) vs. standard full trace CoT SFT, same base model and training data
- -24% training FLOPs vs. standard full trace CoT SFT
Model details
Results
nDCG@10 on BRIGHT, single evaluation run per domain.
Use as a reranker
This is a pointwise reranker, the same style as Rank1 (jhu-clsp/rank1-7b): the model generates its own reasoning trace ending in </think> true or </think> false, and relevance is scored from the logits at that final token rather than by parsing generated text. Unlike training, no hint is available at inference (a real query has no teacher trace to condition on), so the model reasons on its own from a bare prompt, the exact setup used to produce the results on this page.
from vllm import LLM, SamplingParams
import math
model_id = "AdarshSingh7647/Eklav-14B-Reranker"
model = LLM(model=model_id, max_model_len=20000)
tokenizer = model.get_tokenizer()
SYSTEM_PROMPT = "You are a careful retrieval assistant that judges whether a passage is relevant to a user's query."
TASK_INSTRUCTION = "Determine if the following passage is relevant to the query. Answer only with 'true' or 'false'."
def create_prompt(query: str, passage: str) -> str:
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"{TASK_INSTRUCTION}\n\nQuery: {query}\nPassage: {passage}"},
]
return tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True, enable_thinking=True
)
sampling_params = SamplingParams(
temperature=0,
max_tokens=4096,
logprobs=20,
stop=["</think> true", "</think> false", "</think>\ntrue", "</think>\nfalse",
"</think>\n\ntrue", "</think>\n\nfalse"],
)
def score(query: str, passage: str) -> float:
prompt = create_prompt(query, passage)
output = model.generate([prompt], sampling_params)[0].outputs[0]
# the answer token is usually the second to last logprob step (vLLM's stop
# string match can consume one extra token, e.g. <|im_end|>, after it), but
# scan from the end so this is robust to that off by one
for step in reversed(output.logprobs or []):
true_lp = next((v.logprob for k, v in step.items() if tokenizer.decode([k]).strip().lower() == "true"), None)
false_lp = next((v.logprob for k, v in step.items() if tokenizer.decode([k]).strip().lower() == "false"), None)
if true_lp is not None and false_lp is not None:
true_score, false_score = math.exp(true_lp), math.exp(false_lp)
return true_score / (true_score + false_score)
return 0.5
query = "What causes seasons on Earth?"
passages = [
"Seasons are caused by the tilt of Earth's axis relative to its orbit around the Sun.",
"The Great Wall of China is visible from space, according to popular belief.",
]
ranked = sorted(passages, key=lambda p: score(query, p), reverse=True)
for p in ranked:
print(p)This is the same Rank1StyleReranker generate-then-score recipe used for every number on this page, applied without a teacher hint since none exists at inference time.