hhhhwhit/lrat-qwen3-0.6b-retriever-utility-v2-20260803

LRAT Qwen3-Embedding-0.6B — Retriever-Observable Utility V2

This retriever is trained from the released LRAT trajectory positives using supervision aligned with the information available to the retriever:

1. 1.Retain the original LRAT reasoning-aware binary positive filter.
2. 2.Score each unique (web search query, document) pair from 0 through 4 for standalone retrieval utility using Qwen3.5-4B.
3. 3.Give identical query-document pairs one shared score.
4. 4.Remove an explicit negative when that document is a retained positive for the identical search query.

The utility judge does not use the agent's prior knowledge, visited-document history, post-browse reasoning, next action, or distance to the final task answer. In particular, a document is not downweighted merely because an agent had already observed the information.

Utility scores map to raw weights {0: 0.50, 1: 0.75, 2: 1.00, 3: 1.50, 4: 2.00}. Raw weights are divided by their global dataset mean. Training uses weighted InfoNCE, sum_i(w_i * loss_i) / sum_i(w_i), with no pairwise loss or reasoning-length weight.

Training recipe

• —Base model: Qwen/Qwen3-Embedding-0.6B
• —Dataset: organizer-provided Yuqi-Zhou/LRAT-Train
• —Epochs: 2
• —Per-device batch size: 32 on 4 GPUs
• —Group size: 10 (one positive and nine trajectory negatives)
• —Learning rate: 1e-6
• —Warmup ratio: 0.1
• —Query and passage maximum length: 512
• —Temperature: 0.02
• —Pooling: last token
• —Normalized embeddings: yes
• —Cross-device negatives: yes
• —Precision: BF16
• —Seed: 42

Query instruction:

Instruct: Given a web search query, retrieve relevant passages that answer the query
Query:{query}

The repository contains a full Transformers checkpoint and tokenizer files.