scienceagentbench
ScienceAgentBench
ScienceAgentBench
Update 04/30/2026: To mitigate false negatives in evaluation, we have released a verified version of ScienceAgentBench. Please load our benchmark using the following code going forward and make sure you follow the latest instructions in our github repository:
from datasets import load_dataset
ds = load_dataset("osunlp/ScienceAgentBench", split="verified")
The advancements of language language models (LLMs) have piqued growing interest in developing… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/ScienceAgentBench.DataPRM-ScienceAgentBenchScienceAgentBench
ScienceAgentBench
The advancements of language language models (LLMs) have piqued growing interest in developing LLM-based language agents to automate scientific discovery end-to-end, which has sparked both excitement and skepticism about their true capabilities.
In this work, we call for rigorous assessment of agents on individual tasks in a scientific workflow before making bold claims on end-to-end automation.
To this end, we present ScienceAgentBench, a new benchmark for… See the full description on the dataset page: https://huggingface.co/datasets/BDDSSD/ScienceAgentBench.