datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MagicBrush
Dataset Card for MagicBrush
Dataset Summary
MagicBrush is the first large-scale, manually-annotated instruction-guided image editing dataset covering diverse scenarios single-turn, multi-turn, mask-provided, and mask-free editing. MagicBrush comprises 10K (source image, instruction, target image) triples, which is sufficient to train large-scale image editing models.
Please check our website to explore more visual results.
Dataset Structure
"img_id" (str):… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/MagicBrush.Multimodal-Mind2Web
Dataset Summary
Multimodal-Mind2Web is the multimodal version of Mind2Web, a dataset for developing and evaluating generalist agents
for the web that can follow language instructions to complete complex tasks on any website. In this dataset, we align each HTML document in the dataset with
its corresponding webpage screenshot image from the Mind2Web raw dump. This multimodal version addresses the inconvenience of loading images from the ~300GB Mind2Web Raw Dump.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Multimodal-Mind2Web.osumaps19866Mind2Web
Dataset Card for Dataset Name
Dataset Summary
Mind2Web is a dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website. Existing datasets for web agents either use simulated websites or only cover a limited set of websites and tasks, thus not suitable for generalist web agents. With over 2,000 open-ended tasks collected from 137 websites spanning 31 domains and crowdsourced action… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Mind2Web.SuperMemory-VQA
SuperMemoryVQA
SuperMemory-VQA is an egocentric visual question answering benchmark for
evaluating long-horizon memory in augmented reality assistant settings. The
dataset is designed around practical questions a person might ask a wearable
memory assistant, such as where an object was left, what someone said earlier,
whether a planned step was completed, or what happened next in a longer event.
The benchmark contains 4,853 human-verified question-answer pairs grounded in
52.9… See the full description on the dataset page: https://huggingface.co/datasets/OSU-AIoT-MLSys-Lab/SuperMemory-VQA.TravelPlanner
TravelPlanner Dataset
TravelPlanner is a benchmark crafted for evaluating language agents in tool-use and complex planning within multiple constraints. (See our paper for more details.)
Introduction
In TravelPlanner, for a given query, language agents are expected to formulate a comprehensive plan that includes transportation, daily meals, attractions, and accommodation for each day.
TravelPlanner comprises 1,225 queries in total. The number of days and hard constraints… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/TravelPlanner.ScienceAgentBench
ScienceAgentBench
Update 04/30/2026: To mitigate false negatives in evaluation, we have released a verified version of ScienceAgentBench. Please load our benchmark using the following code going forward and make sure you follow the latest instructions in our github repository:
from datasets import load_dataset
ds = load_dataset("osunlp/ScienceAgentBench", split="verified")
The advancements of language language models (LLMs) have piqued growing interest in developing… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/ScienceAgentBench.SkillHarm
SkillHarm
Lifecycle-Aware Skill-Based Attacks via Automated Construction
📄 Paper · 🌐 Project Page · 💻 GitHub · 🤗 Data
Agent skills occupy a privileged position in the agent workflow — agents are expected to implicitly follow and execute them — which makes third-party skills a vulnerable supply-chain attack surface. SkillHarm is a benchmark of skill-based attacks across the skill-use lifecycle, paired with a systematic taxonomy of 12 skill-relevant risks. Every attack is… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/SkillHarm.AttributionBench
Dataset Card for AttributionBench
Github repository: [Github]
Paper: AttributionBench: How Hard is Automatic Attribution Evaluation?
Point of Contact: Yifei Li
Dataset Overview
We constructed this dataset from multiple existing data sources in a unified format, in order to create a unified and diverse testbed for evaluating advanced attribution evaluation systems. The dataset contains both in-domain training set and id-domain and out-of-domain test set.
Usage… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/AttributionBench.Dreamer-V1-DataAfter heavier cleaning, the remaining data size is 3.12M.
WebDreamer: Model-Based Planning for Web Agents
WebDreamer is a planning framework that enables efficient and effective planning for real-world web agent tasks. Check our paper for more details.
This work is a collaboration between OSUNLP and Orby AI.
Repository: https://github.com/OSU-NLP-Group/WebDreamer
Paper: https://arxiv.org/abs/2411.06559
Point of Contact: Kai Zhang
Models
Dreamer-7B:
General… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Dreamer-V1-Data.osu-beatmaps
osu! Beatmaps Dataset (WebDataset)
A collection of ranked/loved osu! beatmaps with audio and chart data, in WebDataset format.
Dataset Variants
Variant
Audio Format
Description
original
MP3/OGG/WAV
Full quality original audio files
compressed
64kbps Mono Opus
Compressed audio for smaller download
from datasets import load_dataset
# Load original audio variant
ds = load_dataset("project-riz/osu-beatmaps", "original", streaming=True)
# Load compressed… See the full description on the dataset page: https://huggingface.co/datasets/project-riz/osu-beatmaps.HippoRAG_2
HippoRAG 2 is a powerful memory framework for LLMs that enhances their ability to recognize and utilize connections in new knowledge—mirroring a key function of human long-term memory.SMolInstructSMolInstruct is a large-scale instruction tuning dataset for chemistry tasks and centers around small molecules. It contains a total of 14 chemistry tasks and over 3 million samples. It is designed to be large-scale, comprehensive, and high-quality.Online-Mind2Web
Blog |
Paper |
Code |
Leaderboard
Online-Mind2Web
Online-Mind2Web is the online version of Mind2Web, a more diverse and user-centric dataset includes 300 high-quality tasks from 136 popular websites across various domains. The dataset covers a diverse set of user tasks, such as clothing, food, housing, and transportation, to evaluate web agents' performance in a real-world online environment.
News
[11/03/2025] We’ve updated 36 tasks that are… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Online-Mind2Web.UGround-V1-Data
Updates
[May 1, 2025] Bounding Box Data: We have added bounding box version of Web-Hybrid. For everyone's convenience, no conversation template is applied to this version of data. All the coordinates (x1, y1, x2, y2) are as always normalized to [0,999].
Notes for Requests
If you have applied for access to this dataset but have not received approval, please contact us via email (Boyu Gou) with your name, institution, and research purpose.
Typically, requests will be… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/UGround-V1-Data.QUEST-Mid-Training-Data
QUEST Mid-Training Data
Parquet shards for QUEST mid-training. One split is published:
context_summarization
Each row has a messages field: list[{"role": "...", "content": "..."}] in chat format.
Relevant Information Extraction
The relevant_info_extraction task is not released because it contains raw HTML content, which might raise legal concerns. We provide the following minimal example to illustrate the task format:
{
"input": [
{
"role": "system"… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/QUEST-Mid-Training-Data.ConflictQA data for ConflictQA.early-experience
Early Experience — Reproduction Data
Supervised fine-tuning data for reproducing Agent Learning via Early Experience across 8 agent environments. Each environment provides data for three training paradigms:
IL — Imitation Learning: expert
SR — Self-Reflection: expert + reflection
IWM — Implicit World Modeling: iwm (world model) → expert
Code: OSU-NLP-Group/EarlyExperience
Usage
from datasets import load_dataset
# load_dataset("osunlp/early-experience"… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/early-experience.QUEST-RL-Data
QUEST RL Data
Project Page | Paper | GitHub
Training split for the QUEST / DeepResearch RL recipe. Each row includes prompt, reward_model, extra_info, and related fields.
Dataset columns
Column
Description
data_source
Source tag (e.g. deepresearch_tasks)
prompt
list[{"role", "content"}] chat-style input
reward_model
Reward configuration (Python literal string; some rows embed numpy-like array(...) and need custom parsing)
extra_info
Extra metadata… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/QUEST-RL-Data.TableInstruct
TableLlama: Towards Open Large Generalist Models for Tables
Project Page: https://osu-nlp-group.github.io/TableLlama/
Paper: https://arxiv.org/abs/2311.09206
Model: https://huggingface.co/osunlp/TableLlama/
Code: https://osu-nlp-group.github.io/TableLlama/
Introduction
We introduce TableLlama, an open-source large generalist model specifically tailored for various table-based tasks. The TableLlama model is trained on TableInstruct Dataset, a meticulously curated… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/TableInstruct.bioscan-traits
Dataset Card for BIOSCAN-Traits
Dataset Details
Dataset Description
BIOSCAN-Traits is a trait-level annotation dataset for fine-grained insect imagery. Derived from BIOSCAN-5M, it provides morphology-centric natural language trait descriptions automatically generated by a two-stage pipeline: (1) a Sparse Autoencoder (SAE) trained on DINOv2 visual features identifies species-level salient visual parts (wings, legs, antennae, etc.), and (2) a Multimodal LLM… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/bioscan-traits.AutoElicit-Exec
AutoElicit-Exec Dataset
Project Page | Paper | GitHub
AutoElicit-Exec is a human-verified dataset of 132 execution trajectories exhibiting unintended behaviors from typical benign execution. All trajectories are elicited from frontier CUAs (i.e., Claude 4.5 Haiku and Claude 4.5 Opus) using AutoElicit, which perturbs benign instructions from OSWorld to increase the likelihood of unintended harm while keeping instructions realistic and benign. This dataset is designed to provide… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/AutoElicit-Exec.AttrScore We construct this dataset, which contains both training and test data for the evaluation of attribution.
The training data are repurposed from related tasks, such as question answering, fact-checking,
natural language inference, and summarization. The test data contains a set simulated from QA datasets
and a set manually curated from a generative search engine, New Bing.QUEST-SFT-Data-Objective-Script
QUEST SFT Data Objective Script
Project Page | Paper | GitHub
Supervised fine-tuning split for QUEST / DeepResearch objective tasks. Each row includes the user prompt, a rule-style reward_model, extra_info, and the objective task category. The corresponding objective evaluation scripts are provided separately under eval_scripts/.
This dataset follows the same broad schema style as osunlp/QUEST-RL-Data: each row includes prompt, reward_model, extra_info, and rl_task_category. The… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/QUEST-SFT-Data-Objective-Script.QUEST-SFT-Data-Objective
QUEST SFT Data (Objective)
Project Page | Paper | GitHub
Objective-style supervised fine-tuning trajectories for QUEST (tool-using assistant format). This dataset is part of the QUEST family, designed to train deep research agents with fully synthetic tasks.
Split: train
Columns: messages (list[{role, content}])
Load
from datasets import load_dataset
ds = load_dataset("osunlp/QUEST-SFT-Data-Objective", split="train", streaming=True)
row = next(iter(ds))… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/QUEST-SFT-Data-Objective.QUEST-SFT-Data-Open-ended
QUEST SFT Data (Open-ended)
Project Page | Paper | GitHub
Open-ended supervised fine-tuning trajectories for QUEST (tool-using assistant format). Split: train. Columns: messages (list[{role, content}]).
Load
from datasets import load_dataset
ds = load_dataset("osunlp/QUEST-SFT-Data-Open-ended", split="train", streaming=True)
row = next(iter(ds))
print(row.keys())
QUEST Family
Type
Resources
35B checkpoints
RL, MT+SFT, MT, SFT
30B checkpoints… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/QUEST-SFT-Data-Open-ended.osu-everything-tools
osu-everything
osu-everything contains the source tools and runbooks for maintaining the
compact v1 osu! dataset in:
hf://buckets/lekdan/osu-everything
The source checkout is a normal Git repo. Runtime dataset state lives outside
the checkout in a sibling dataset workspace.
~/osu_model/ # source checkout: code, docs, schemas, feature pipeline
~/osu_model_dataset/ # working dataset state: archives, data, fetcher DB, logs
~/osu_model_features/ # derived feature store output… See the full description on the dataset page: https://huggingface.co/datasets/lekdan/osu-everything-tools.osu
osu-dataset-builder Schema
This document describes the parquet file schemas generated by osu-dataset-builder.
Overview
The dataset consists of 14 parquet files organized into logical groups:
Group
Files
Description
Core
beatmaps, hit_objects, timing_points
Main beatmap data
Sliders
slider_control_points, slider_data
Slider curve details
Storyboard
storyboard_elements, storyboard_commands, storyboard_loops, storyboard_triggers
Storyboard animations
Events… See the full description on the dataset page: https://huggingface.co/datasets/lekdan/osu.TACO-Cobalt
TACO-Cobalt
Paper | GitHub
TACO-Cobalt is a cleaned version of that TACO code generation dataset crawled from programming competition websites.
Due to noise in the crawled data, we start from the TACO-verified subset and apply further cleaning to obtain 6,103 valid tasks, each with at least 8 test cases.
For each task, we sort the test cases’ difficulty by the possibility of Qwen2.5-Coder-7B-Instruct passing them out of 16 attempts.
We select the four easiest test cases as the… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/TACO-Cobalt.AgentCL
AgentCL: Evaluation Framework for Continual Learning in Agents
Datasets
agentboard_babyai, agentboard_scienceworld, and mmlu_pro are included as ready-to-use subsets. They are direct subsets from existing public datasets, with no modifications to the original data.
Our constructed codeeval-pro and browsecomp_plus streams are provided.
The metadata in source datasets only indicates the authorship for the source datasets, and does not imply the authorship of this… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/AgentCL.