Seraphic663/lrat-qwen3-0.6b-1epoch-20260716

LRAT Qwen3-Embedding-0.6B — 1 Epoch

This is a full-parameter retriever checkpoint based on Qwen/Qwen3-Embedding-0.6B, prepared for the CCIR LRAT competition.

Training

• —Data: official LRAT training pairs only; no external training data
• —Records: 96,504
• —Epochs: 1
• —Optimizer steps: 12,063
• —Hardware: 2 × NVIDIA A40
• —Precision: bfloat16
• —Train group: 1 positive and 5 negatives
• —Global effective query batch: 8
• —Query / passage maximum length: 128 / 512
• —Learning rate: 1e-6
• —Pooling: last token
• —Normalized embeddings: yes
• —Temperature: 0.02
• —Query instruction: Given a web search query, retrieve relevant passages that answer the query

The training objective uses LRAT reweight_rate values with cross-device negatives. The checkpoint contains merged full model parameters and does not require a LoRA or adapter at inference time.

Loading

import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

model_id = "Seraphic663/lrat-qwen3-0.6b-1epoch-20260716"
tokenizer = AutoTokenizer.from_pretrained(model_id, padding_side="left")
model = AutoModel.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
).cuda().eval()

instruction = "Given a web search query, retrieve relevant passages that answer the query"
texts = [f"Instruct: {instruction}\nQuery:example query", "example passage"]
batch = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt").to("cuda")

with torch.inference_mode():
    output = model(**batch).last_hidden_state
    embeddings = F.normalize(output[:, -1].float(), p=2, dim=1)

Validation Note

On a fixed 500-row candidate-set development diagnostic, this checkpoint obtained Recall@1/5/10 of 0.672/0.934/0.984 and MRR of 0.78037. These are local candidate-set diagnostics, not official BC-Plus leaderboard scores.

Integrity

model.safetensors
2,383,139,480 bytes
SHA-256 b25b3b08a3199a788ea5e8bc005ee20ab831ada36cb4eccd7ba915e0d6e02501