CoolFace
Datasetpublic

rl-rag/open-scholar-gpt-oss-120b

open-scholar-gpt-oss-120b Deep research agent evaluation on data/drtulu_open_scholar.jsonl (normal split). Results Metric Value pass@1 0.0% avg@1 0.0% Trajectory accuracy 0.0% (0/11854) Questions 11854 Trajectories 11854 (1 per question) Avg tool calls 17.3 Full conversations ✅ Model & Setup Model gpt-oss-120b Judge gpt-4o Max tool calls 50 Temperature 0.7 Blocked domains None Tool… See the full description on the dataset page: https://huggingface.co/datasets/rl-rag/open-scholar-gpt-oss-120b.

sourceHugging Faceupdated 7mo agoView on Hugging Face
0likes46downloads
Dataset Card

open-scholar-gpt-oss-120b

Deep research agent evaluation on data/drtulu_open_scholar.jsonl (normal split).

Results

MetricValue
pass@10.0%
avg@10.0%
Trajectory accuracy0.0% (0/11854)
Questions11854
Trajectories11854 (1 per question)
Avg tool calls17.3
Full conversations✅

Model & Setup

Modelgpt-oss-120b
Judgegpt-4o
Max tool calls50
Temperature0.7
Blocked domainsNone

Tool Usage

ToolCalls%
browser.open109,54055%
browser.search50,46525%
functions.paper_search21,50011%
browser.find15,3258%
functions.pubmed_search2,1831%

Total: 199,013 tool calls (17.3 per trajectory)

⚠️ Contamination: 56/11854 trajectories flagged (search results contained evaluation dataset pages). Filter with ds.filter(lambda x: not x['contaminated']).

Columns

ColumnDescription
qidQuestion ID
traj_idxTrajectory index (0-0)
questionInput question
reference_answerGround truth answer
boxed_answerModel's extracted \boxed{} answer
correctGPT-4o judge verdict
judge_explanationJudge's reasoning
question_accuracyFraction of trajectories correct for this question (difficulty: 0=hardest, 1=easiest)
num_tool_callsNumber of tool calls in trajectory
tool_callsTool call log (JSON)
conversationFull trajectory with reasoning + tool responses (JSON)
contaminatedWhether search results contained evaluation dataset pages
latency_sGeneration time in seconds