datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Multimodal-Mind2Web
Dataset Summary
Multimodal-Mind2Web is the multimodal version of Mind2Web, a dataset for developing and evaluating generalist agents
for the web that can follow language instructions to complete complex tasks on any website. In this dataset, we align each HTML document in the dataset with
its corresponding webpage screenshot image from the Mind2Web raw dump. This multimodal version addresses the inconvenience of loading images from the ~300GB Mind2Web Raw Dump.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/osunlp/Multimodal-Mind2Web.mind2web_multimodal_test_domain
Dataset Card for "Cross-Domain" Test Split in Multimodal Mind2Web
Note: This dataset is the test split of the Cross-Domain dataset introduced in the paper.
This is a FiftyOne dataset with 4050 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/mind2web_multimodal_test_domain.mind2web_multimodal_test_task
Dataset Card for Multimodal Mind2Web "Cross-Task" Test Split
Note: This dataset is the test split of the Cross-Task dataset introduced in the paper.
This is a FiftyOne dataset with 1338 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/mind2web_multimodal_test_task.mind2web_multimodal_test_website
Dataset Card for Multimodal Mind2Web "Cross-Website" Test Split
Note: This dataset is the test split of the Cross-Website dataset introduced in the paper.
This is a FiftyOne dataset with 1019 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/mind2web_multimodal_test_website.ebrowse-v1.1-online-mind2web-subset
ebrowse v1.1 — Online-Mind2Web subset
20 full browsing-agent traces produced by ebrowse,
a token-efficient browser-control CLI for LLM agents. Each trace records a complete
task attempt on a live public website: every tool call, every tool output, a
screenshot and DOM snapshot per step, browser/console/network events, and the
agent's own messages.
These are raw research traces, not a cleaned benchmark. They exist so the
ebrowse trace viewer can replay exactly what the agent saw… See the full description on the dataset page: https://huggingface.co/datasets/jayoohwang/ebrowse-v1.1-online-mind2web-subset.minibench-multimodal-mind2webMagma-Mind2Web-SoM
Magma: A Foundation Model for Multimodal AI Agents
Jianwei Yang*1†
Reuben Tan1†
Qianhui Wu1†
Ruijie Zheng2‡
Baolin Peng1‡
Yongyuan Liang2‡
Yu Gu1
Mu Cai3
Seonghyeon Ye4
Joel Jang5
Yuquan Deng5
Lars Liden1
Jianfeng Gao1▽
1 Microsoft Research; 2 University of Maryland; 3 University of Wisconsin-Madison4 KAIST; 5 University of Washington
* Project lead † First authors ‡ Second authors ▽ Leadership
[arXiv Paper] [Project Page] [Hugging Face Paper] [Github Repo] [Video]… See the full description on the dataset page: https://huggingface.co/datasets/MagmaAI/Magma-Mind2Web-SoM.Mind2Web_train_llava
Mind2Web training set for the paper: Harnessing Webpage Uis For Text Rich Visual Understanding
🌐 Homepage | 🐍 GitHub | 📖 arXiv
Introduction
We introduce MultiUI, a dataset containing 7.3 million samples from 1 million websites, covering diverse multi- modal tasks and UI layouts. Models trained on MultiUI not only excel in web UI tasks—achieving up to a 48% improvement on VisualWebBench and a 19.1% boost in action accuracy on a web agent dataset Mind2Web—but also… See the full description on the dataset page: https://huggingface.co/datasets/neulab/Mind2Web_train_llava.MM-Mind2Web-testmind2web-mcq-datasetMultimodal-Mind2Web
cua-lite/Multimodal-Mind2Web
cua-lite preprocessed version of Multimodal-Mind2Web (osunlp/Multimodal-Mind2Web). The screenshot-augmented Mind2Web web-agent benchmark: real human-annotated multi-step navigation episodes across 100+ websites. Only the train split is processed (the test_task/test_website/test_domain benchmark holdouts are never trained on). Episodes containing a native minibench-mind2webMultimodal-Mind2Web-filtered处理自https://huggingface.co/datasets/osunlp/Multimodal-Mind2Web
删除了训练集
删除了html信息
保留了pos_candidates的bbox信息（x, y, w, h），删除了其它pos_candidates和neg_candidates的信息
添加了图片width, height信息
删除了不包含pos_candidates或不包含screenshot的样本
minibench-mm-mind2webMultimodal-Mind2Web-HTML-WM-messagesMultimodal-Mind2Web-HTML-WM-messages-testmirror-Multimodal-Mind2Web
Dataset Summary
Multimodal-Mind2Web is the multimodal version of Mind2Web, a dataset for developing and evaluating generalist agents
for the web that can follow language instructions to complete complex tasks on any website. In this dataset, we align each HTML document in the dataset with
its corresponding webpage screenshot image from the Mind2Web raw dump. This multimodal version addresses the inconvenience of loading images from the ~300GB Mind2Web Raw Dump.… See the full description on the dataset page: https://huggingface.co/datasets/alucent/mirror-Multimodal-Mind2Web.Multimodal-Mind2Web-HTML-WM-messages-filter-35000mind2web-skillmem-train
Mind2Web SkillMemory — Train split (memory pool)
Pre-processed Mind2Web train trajectories from the SkillMemory pipeline,
intended as a memory pool for retrieval-augmented web agents (e.g.
WebForge / GUIagent ablations replacing AgentNet computer-use memory).
Source: ~/SkillMemory/data/processed/train (already decomposed into
subtask intents + cropped to GT viewport blocks).
Layout
train/
trajectories/<aid>.json # 1009 task metadata files… See the full description on the dataset page: https://huggingface.co/datasets/ming9999/mind2web-skillmem-train.Multimodal-Mind2Web-HTML-WM