datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MolmoAct-Midtraining-Mixture
MolmoAct - Midtraining Mixture
Data Mixture used for MolmoAct Midtraining. Contains MolmoAct Dataset formulated as Action Reasoning Data.
MolmoAct is a fully open-source action reasoning model for robotic manipulation developed by the Allen Institute for AI. MolmoAct is trained on a subset of OXE and MolmoAct Dataset, a dataset with 10k high-quality trajectories of a single-arm Franka robot performing 93 unique manipulation tasks in both home and tabletop environments. It has… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct-Midtraining-Mixture.molmobot-data
MolmoBot-data
Training episode data (actions, visual inputs, and other sensor data) for 8 tasks on 2 robotic platforms:
DoorOpeningDataGenConfig
RBY1OpenDataGenConfig
RBY1PickDataGenConfig
FrankaPickOmniCamConfig
RBY1PickAndPlaceDataGenConfig
FrankaPickAndPlaceOmniCamConfig
FrankaPickAndPlaceColorOmniCamConfig
FrankaPickAndPlaceNextToOmniCamConfig
Please note that every package indexed by the parquet files can contain several instances of episode data.
We also provide an… See the full description on the dataset page: https://huggingface.co/datasets/allenai/molmobot-data.MolmoAct2-BimanualYAM-DatasetThis dataset was created using LeRobot.
MolmoAct2-BimanualYAM Dataset
This repository is the merged ckpt / merged LeRobot dataset artifact for the MolmoAct2-BimanualYAM Dataset, a large-scale collection of bimanual robot manipulation demonstrations collected for MolmoAct2. Across the full collection, MolmoAct2-BimanualYAM contains more than 720 hours of training demonstrations spanning diverse tabletop manipulation tasks.
Language Annotations
This dataset includes… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct2-BimanualYAM-Dataset.MolmoAct-DatasetThis dataset was created using LeRobot.
Dataset Description
This dataset contains MolmoAct Dataset in lerobot format. All contents in this dataset were collected in-house by Ai2.
Quick links:
📂 All Models
📂 All Data
📃 Paper
🎥 Blog Post
🎥 Video
Code
License and Use
This dataset is licensed under CC BY-4.0. It is intended for research and educational use in accordance with Ai2's Responsible Use Guidelines.
Citation
@misc{molmoact2025… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct-Dataset.MolmoAct-Pretraining-Mixture
MolmoAct - Pretraining Mixture
Data Mixture used for MolmoAct Pretraining. Contains a subset of OXE formulated as Action Reasoning Data along with auxiliary robot data and link to Multimodal Web data.
MolmoAct is a fully open-source action reasoning model for robotic manipulation developed by the Allen Institute for AI. MolmoAct is trained on a subset of OXE and MolmoAct Dataset, a dataset with 10k high-quality trajectories of a single-arm Franka robot performing 93 unique… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct-Pretraining-Mixture.Molmo2-SynMultiImageQA
Molmo2-SynMultiImageQA
Molmo2-SynMultiImageQA is a collection of synthetic multi-image question-answer pairs about various kinds of text-rich images, including charts, tables, documents, diagrams, etc.
The synthetic data is generated by extending the CoSyn framework into multi-image settings,
with Claude-sonnet-4-5 as the coding LLM to generate code that can be executed to render an image.
Then, we use GPT-5 to generate question-answer pairs with code (without using the rendered… See the full description on the dataset page: https://huggingface.co/datasets/allenai/Molmo2-SynMultiImageQA.Molmo2-ER-VST-P
Molmo2-ER · rayruiyang/vst_500k
500K perception QA over images normalized to a uniform virtual camera (single + multi-view).
This is a re-hosted, loader-ready subset of the upstream dataset, used to train allenai/Molmo2-ER-4B. Files mirror the upstream layout; nothing in the data has been modified.
⚠️ This dataset is released for non-commercial research use only, inheriting the most-restrictive license among its upstream sources. See the upstream repository for details.… See the full description on the dataset page: https://huggingface.co/datasets/allenai/Molmo2-ER-VST-P.molmoact2_so100_101_rbmMolmoWeb-SyntheticGround
MolmoWeb-SyntheticGround
This dataset was introduced in the paper MolmoWeb: Open Visual Web Agent and Open Data for the Open Web.
A dataset of webpage screenshots paired with synthetic grounding tasks. Each example asks a model to identify a target element on the page, with ground-truth bounding boxes and (for GPT examples) natural-language thoughts.
Dataset Usage
from datasets import load_dataset
# load the gpt subset
ds = load_dataset("allenai/MolmoWeb-SyntheticGround"… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoWeb-SyntheticGround.MolmoAct2-DROID-Dataset
MolmoAct2-DROID Dataset
This dataset was created using LeRobot.
Language Annotations
This dataset includes annotated language instructions in meta/tasks_annotated.parquet. The file is indexed by episode_index and has a task column containing our per-episode annotated instruction.
The standard LeRobot loader resolves a frame's language instruction through task_index: each data row stores a task_index, which is looked up in meta/tasks.parquet. When you use these… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct2-DROID-Dataset.EchoPolicy-0-MolmoSpaces-eval
EchoPolicy-0: MolmoSpaces Evaluation
Evaluation results, complete episode videos, trajectories and execution logs for EchoPolicy-0, developed by MagicLab.
Official submission: allenai/molmospaces#199
Evaluation version: echopolicy-ms-v15-20260918-full3915
Policy version: echo-tools-20260917-v15
Coverage: 3915 episodes across Close, Pick, Open and Pick & Place, including every success and failure.
Browse the videos
Open the episode viewer.
Each row represents one… See the full description on the dataset page: https://huggingface.co/datasets/ddffwyb/EchoPolicy-0-MolmoSpaces-eval.molmo2-tulu4-classifiedMolmoAct2-SO100_101-Dataset
MolmoAct2 SO-100/101 Dataset Manifest
This repository lists the SO-100 and SO-101 LeRobot datasets used for MolmoAct2 training. The datasets were collected from online LeRobot repositories, and this repository provides the annotated language instructions for each dataset.
The original dataset repositories are listed in repo_list.json. The list contains 1220 repositories from 377 users.
Language Annotations
Each dataset has a corresponding tasks_annotated.parquet… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct2-SO100_101-Dataset.MolmoPoint-GUISyn
MolmoPoint-GUISyn
MolmoPoint-GUISyn is a large-scale synthetic dataset of 36K GUI screenshots with dense pointing annotations for training GUI grounding agents. Each screenshot is a realistic simulation of a digital environment (desktop apps, mobile apps, websites) generated entirely from code, with an average of 54 annotated UI elements per image.
The data is generated using the MolmoPoint-GUISyn pipeline, with Claude Sonnet 4.6 as the coding LLM.
Quick links:
Model:… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoPoint-GUISyn.MolmoAct2-LIBERO-DatasetThis dataset was created using LeRobot.
MolmoAct2-LIBERO Dataset
This repository contains the merged LeRobot LIBERO dataset used for MolmoAct2 finetuning experiments. It combines all four LIBERO suites used in our finetuning setup: LIBERO-Spatial, LIBERO-Goal, LIBERO-Object, and LIBERO-Long / LIBERO-10.
The merged dataset contains 1,693 episodes and 273,465 frame-level training examples at 10 FPS. It is intended for finetuning only. We did not use this dataset in MolmoAct2… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct2-LIBERO-Dataset.Molmo2-VideoPoint
Molmo2-VideoPoint
Molmo2-VideoPoint is a dataset of video pointing data collected from human annotators.
It can be used to fine-tune vision-language models for video grounding by pointing.
Molmo2-VideoPoint is part of the Molmo2 dataset collection and was used to train the Molmo2 family of models.
Quick links:
📃 Paper
🎥 Blog with Videos
Usage
from datasets import load_dataset
# Load entire dataset
ds = load_dataset("allenai/Molmo2-VideoPoint", split="train")
#… See the full description on the dataset page: https://huggingface.co/datasets/allenai/Molmo2-VideoPoint.Spa_Bench_Full_MolmoAct2
Spa-Bench physical rollouts — MolmoAct2
Physical SO-101 rollout recordings for the MolmoAct2 Spa-Bench checkpoint at
epoch 12, step 76,596.
Scope and counts
This is a complete Spa-Bench evaluation: 900 episodes, comprising 300
primary withheld/OOD spatial trials and 600 diagnostic/control trials. The
repository contains 782,590 frames, 356 unique instruction strings, two
480×640 RGB streams, and six-dimensional state/action at 30 FPS.
Reported physical… See the full description on the dataset page: https://huggingface.co/datasets/justintiensmith/Spa_Bench_Full_MolmoAct2.MolmoWeb-HumanTrajs
MolmoWeb-HumanTrajs
A dataset of human collected web trajectories. Each example pairs an instruction with a sequence of webpage screenshots and the corresponding agent actions (clicks, typing, scrolling, etc.).
Dataset Usage
from datasets import load_dataset
# load a single subset
ds = load_dataset("allenai/MolmoWeb-HumanTrajs")
Working with images and trajectories
Each row has an images field (list of raw image bytes) and a corresponding… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoWeb-HumanTrajs.MolmoWeb-HumanSkills
MolmoWeb-HumanSkills
This dataset was introduced in the paper MolmoWeb: Open Visual Web Agent and Open Data for the Open Web.
A dataset of human collected web-navigation skills, where a skill is a trajectory for a very low level task (eg. find_and_open, fill_form). Each example pairs an instruction with a sequence of webpage screenshots and the corresponding agent actions (clicks, typing, scrolling, etc.).
Dataset Usage
from datasets import load_dataset
# load a… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoWeb-HumanSkills.MolmoWeb-SyntheticTrajs
MolmoWeb-SyntheticTrajs
A dataset of synthetic web-navigation trajectories. Each example pairs an instruction with a sequence of webpage screenshots and the
corresponding agent actions (clicks, typing, scrolling, etc.).
Dataset Usage
from datasets import load_dataset
# load a single subset
ds = load_dataset("allenai/MolmoWeb-SyntheticTrajs", "from_template")
# available subsets
for cfg in ["from_template", "task_seeded_wv", "task_seeded_om2w"… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoWeb-SyntheticTrajs.Molmo2-Cap
Molmo2-Cap
Molmo2-Cap is a dataset of very long (average 900 words), detailed video captions.
It can be used to pre-train and fine-tune vision-language models.
Molmo2-Cap is part of the Molmo2 dataset collection and was used to train the Molmo2 family of models.
Quick links:
📃 Paper
🎥 Blog with Videos
Data Format
Videos are stored as Youtube video ID that will need to be downloaded separately.
Clip Annotations
clip_timestamps: a list of video… See the full description on the dataset page: https://huggingface.co/datasets/allenai/Molmo2-Cap.Molmo2-MultiImagePoint
Molmo2 Multi-Image Pointing
This dataset contains multi-image pointing/counting metadata.
This dataset is generated by extending PixMo-Points using a semantic grouping algorithm designed to maximize coverage.
Molmo2-MultiImagePoint is a part of the Molmo2 dataset collection and was used to
provide the multi-image pointing capabilities of the Molmo2 family of models.
Quick links:
📃 Paper
🎥 Blog with Videos
Columns
image_urls: list of image URLs (original source… See the full description on the dataset page: https://huggingface.co/datasets/allenai/Molmo2-MultiImagePoint.abc-molmoact2-yam-mix
ABC + MolmoAct2 YAM Mix — bimanual manipulation (LeRobot v3.0)
21,767 episodes / 61,120,847 frames / 14 language-differentiated tasks @ 30 fps,
224x224, 14-DoF bimanual YAM (cameras: top, left_wrist, right_wrist).
▶ Browse episodes in the LeRobot Dataset Visualizer —
synchronized camera playback + action/state charts. The Dataset Viewer above
shows the numeric streams (state/action/indices) from the data parquets; camera
frames live in chunked video files that only the… See the full description on the dataset page: https://huggingface.co/datasets/kkipngenokoech/abc-molmoact2-yam-mix.Molmo2-VideoTrack
Molmo2-VideoTrack
Molmo2-VideoTrack is a dataset of video point tracking annotations collected from human annotators across 16 video datasets.
It can be used to fine-tune vision-language models for video object tracking via point trajectories.
Molmo2-VideoTrack is part of the Molmo2 dataset collection and was used to train the Molmo2 family of models.
Quick links:
📃 Paper
🎥 Blog with Videos
Usage
from datasets import load_dataset
# Load entire dataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/allenai/Molmo2-VideoTrack.MolmoAct2-RT-1-Dataset
MolmoAct2-RT-1 Dataset
This dataset was created using LeRobot.
Language Annotations
This dataset includes annotated language instructions in meta/tasks_annotated.parquet. The file is indexed by episode_index and has a task column containing our per-episode annotated instruction.
The standard LeRobot loader resolves a frame's language instruction through task_index: each data row stores a task_index, which is looked up in meta/tasks.parquet. When you use these… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct2-RT-1-Dataset.rollout_molmoact2_500eps_endeffector_eval_20260902_120317This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/mattpidden/rollout_molmoact2_500eps_endeffector_eval_20260902_120317.MolmoAct2-Bridge-Dataset
MolmoAct2-Bridge Dataset
This dataset was created using LeRobot.
Language Annotations
This dataset includes annotated language instructions in meta/tasks_annotated.parquet. The file is indexed by episode_index and has a task column containing our per-episode annotated instruction.
The standard LeRobot loader resolves a frame's language instruction through task_index: each data row stores a task_index, which is looked up in meta/tasks.parquet. When you use these… See the full description on the dataset page: https://huggingface.co/datasets/allenai/MolmoAct2-Bridge-Dataset.rollout_molmoact2_500eps_endeffector_incorrect_task_eval_20260904_104057This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/mattpidden/rollout_molmoact2_500eps_endeffector_incorrect_task_eval_20260904_104057.rollout_molmoact2_500eps_endeffector_hand_trace_eval_20260904_143115This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/mattpidden/rollout_molmoact2_500eps_endeffector_hand_trace_eval_20260904_143115.Molmo2-VideoCapQA
Molmo2-VideoCapQA
Molmo2-VideoCapQA is a dataset of multiple-choice video QA that only requires visual content.
It can be used to fine-tune vision-language models.
Molmo2-VideoCapQA is part of the Molmo2 dataset collection and was used to train the Molmo2 family of models.
Quick links:
📃 Paper
🎥 Blog with Videos
Data Format
Videos are stored as Youtube video ID that will need to be downloaded separately. We provide a mapping from their IDs to the original YouTube… See the full description on the dataset page: https://huggingface.co/datasets/allenai/Molmo2-VideoCapQA.