datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Multimodal-Mind2Web
Dataset Summary
Multimodal-Mind2Web is the multimodal version of Mind2Web, a dataset for developing and evaluating generalist agents
for the web that can follow language instructions to complete complex tasks on any website. In this dataset, we align each HTML document in the dataset with
its corresponding webpage screenshot image from the Mind2Web raw dump. This multimodal version addresses the inconvenience of loading images from the ~300GB Mind2Web Raw Dump.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Multimodal-Mind2Web.mind2web-utg-crawl-outputsMind2Web
Dataset Card for Dataset Name
Dataset Summary
Mind2Web is a dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website. Existing datasets for web agents either use simulated websites or only cover a limited set of websites and tasks, thus not suitable for generalist web agents. With over 2,000 open-ended tasks collected from 137 websites spanning 31 domains and crowdsourced action… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Mind2Web.mind2web_multimodal_test_domain
Dataset Card for "Cross-Domain" Test Split in Multimodal Mind2Web
Note: This dataset is the test split of the Cross-Domain dataset introduced in the paper.
This is a FiftyOne dataset with 4050 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/mind2web_multimodal_test_domain.mind2web_multimodal_test_task
Dataset Card for Multimodal Mind2Web "Cross-Task" Test Split
Note: This dataset is the test split of the Cross-Task dataset introduced in the paper.
This is a FiftyOne dataset with 1338 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/mind2web_multimodal_test_task.MNL_Mind2Web_APPWorld
Mistake Notebook Learning (MNL) Dataset
Our work has been acepted by ACL 2026
Project Code | Paper
⭐ If you find this project helpful, please give us a star on GitHub! It means a lot to us.
Github:https://github.com/Bairong-Xdynamics/MistakeNotebookLearning
Mistake Notebook Learning (MNL) is a novel, training-free memory framework that enables Large Language Model (LLM) agents to systematically learn from their mistakes. This repository contains the training and evaluation… See the full description on the dataset page: https://huggingface.co/datasets/MultiSense/MNL_Mind2Web_APPWorld.mind2web_multimodal_test_website
Dataset Card for Multimodal Mind2Web "Cross-Website" Test Split
Note: This dataset is the test split of the Cross-Website dataset introduced in the paper.
This is a FiftyOne dataset with 1019 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/mind2web_multimodal_test_website.Online-Mind2Web
Blog |
Paper |
Code |
Leaderboard
Online-Mind2Web
Online-Mind2Web is the online version of Mind2Web, a more diverse and user-centric dataset includes 300 high-quality tasks from 136 popular websites across various domains. The dataset covers a diverse set of user tasks, such as clothing, food, housing, and transportation, to evaluate web agents' performance in a real-world online environment.
News
[11/03/2025] We’ve updated 36 tasks that are… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Online-Mind2Web.mind2web-utg-eval-trajectories
UTG Cross-Model Evaluation Trajectories (ms50)
Agent trajectories from cross-model experiments evaluating UI Transition Graph (UTG) augmentation on web navigation tasks.
Experiment Setup
Task set: 90 tasks (46 medium, 44 complex) from Mind2Web benchmark
Agent framework: browser-use v0.12.0
max_steps: 50 (increased from default 20 to avoid step-budget ceiling effects)
Judge: Gemini 2.5 Flash (automated screenshot-based SUCCESS/PARTIAL/FAILURE)
UTG source: Pre-computed UI… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations-utg-dev/mind2web-utg-eval-trajectories.MT-Mind2Web
MT-Mind2Web Dataset
MT-Mind2Web is constructed by using the single-turn interactions from Mind2Web, an expert-annotated web navigation dataset, as the guidance to construct conversation sessions.
Statistics
Train
Test-Task
Test-Website
Test-Subdomain
# Conversations
600
34
42
44
# Turns
2,896
191
218
216
Avg. # Turn/Conv.
4.83
5.62
5.19
4.91
Avg. # Action/Turn
2.95
3.16
3.01
3.07
Avg. # Element/Turn
573.8
626.3
620.6
759.4
Avg. Inst. Length
36.3… See the full description on the dataset page: https://huggingface.co/datasets/magicgh/MT-Mind2Web.Mind2Web-LiveDataset: Mind2Web-Live
Github: https://github.com/iMeanAI/WebCanvas
MM-Mind2Web-tilde_test_snapshot_20dist
MultiModal-Mind2Web~ (MM-Mind2Web~)
rabbit inc.
[Leaderboard & Blogpost to be released]
Configuration: test split, snapshot with seed 42, 20 distractors
Multimodal-Mind2Web is a dataset proposed by Boyuan et al.. It's designed for the development and evaluation of generalist web agents and includes various action trajectories of humans on real websites.
We've simplified the raw dump from both Multimodal-Mind2Web and Mind2Web into sequences of observation-action pairs. We've… See the full description on the dataset page: https://huggingface.co/datasets/rabbit-hmi/MM-Mind2Web-tilde_test_snapshot_20dist.ebrowse-v1.1-online-mind2web-subset
ebrowse v1.1 — Online-Mind2Web subset
20 full browsing-agent traces produced by ebrowse,
a token-efficient browser-control CLI for LLM agents. Each trace records a complete
task attempt on a live public website: every tool call, every tool output, a
screenshot and DOM snapshot per step, browser/console/network events, and the
agent's own messages.
These are raw research traces, not a cleaned benchmark. They exist so the
ebrowse trace viewer can replay exactly what the agent saw… See the full description on the dataset page: https://huggingface.co/datasets/jayoohwang/ebrowse-v1.1-online-mind2web-subset.minibench-multimodal-mind2webtraining_data_mind2webMagma-Mind2Web-SoM
Magma: A Foundation Model for Multimodal AI Agents
Jianwei Yang*1†
Reuben Tan1†
Qianhui Wu1†
Ruijie Zheng2‡
Baolin Peng1‡
Yongyuan Liang2‡
Yu Gu1
Mu Cai3
Seonghyeon Ye4
Joel Jang5
Yuquan Deng5
Lars Liden1
Jianfeng Gao1▽
1 Microsoft Research; 2 University of Maryland; 3 University of Wisconsin-Madison4 KAIST; 5 University of Washington
* Project lead † First authors ‡ Second authors ▽ Leadership
[arXiv Paper] [Project Page] [Hugging Face Paper] [Github Repo] [Video]… See the full description on the dataset page: https://huggingface.co/datasets/MagmaAI/Magma-Mind2Web-SoM.dcagent2-swebench-verified-random-100-folders-dcagent-neulab-mind2web-sandboxe-12131316Mind2Web-HTML-cleaned-lite-with-desc_w_taoMind2Web_train_llava
Mind2Web training set for the paper: Harnessing Webpage Uis For Text Rich Visual Understanding
🌐 Homepage | 🐍 GitHub | 📖 arXiv
Introduction
We introduce MultiUI, a dataset containing 7.3 million samples from 1 million websites, covering diverse multi- modal tasks and UI layouts. Models trained on MultiUI not only excel in web UI tasks—achieving up to a 48% improvement on VisualWebBench and a 19.1% boost in action accuracy on a web agent dataset Mind2Web—but also… See the full description on the dataset page: https://huggingface.co/datasets/neulab/Mind2Web_train_llava.Multimodal-Mind2Web
cua-lite/Multimodal-Mind2Web
cua-lite preprocessed version of Multimodal-Mind2Web (osunlp/Multimodal-Mind2Web). The screenshot-augmented Mind2Web web-agent benchmark: real human-annotated multi-step navigation episodes across 100+ websites. Only the train split is processed (the test_task/test_website/test_domain benchmark holdouts are never trained on). Episodes containing a native Mind2Web-2
Mind2Web 2
Mind2Web 2 is an evaluation framework for agentic search capabilities, featuring Agent-as-a-Judge methodology for comprehensive assessment of web automation agents.
Mind2Web 2 features realistic and diverse long-horizon web search tasks and a novel Agent-as-a-Judge framework to evaluate complex, time-varying, and citation-backed answers.
🔗 Links
🏠 Homepage
🏆 Leaderboard
📖 Paper
💻 Code
🔄 Changelog
Oct 23, 2025:
Updated several tasks… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Mind2Web-2.MM-Mind2Web-testmind2web-mcq-datasetmind2web-utg-crawli-Mind2Webnull
Mind2Web-cleaned-lite-value-model-w-cot-formatted
Dataset Card for "Mind2Web-cleaned-lite-value-model-w-cot-formatted"
More Information needed
Mind2Web-HTML-cleaned-lite-with-desc_w_tao_value_rationaleminibench-mind2webmind2web_conversationMultimodal-Mind2Web-filtered处理自https://huggingface.co/datasets/osunlp/Multimodal-Mind2Web
删除了训练集
删除了html信息
保留了pos_candidates的bbox信息（x, y, w, h），删除了其它pos_candidates和neg_candidates的信息
添加了图片width, height信息
删除了不包含pos_candidates或不包含screenshot的样本