datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
worldmem_minecraft_datasetvae_cache_minecraft_480p_9sMinecraft-Skill-Data
Dataset Card for Minecraft Expert Skill Data
This dataset consists of expert demonstration trajectories from a Minecraft simulation environment. Each trajectory includes ground-truth skill segmentation annotations, enabling research into action segmentation, skill discovery, imitation learning, and reinforcement learning with temporally-structured data.
Dataset Details
Dataset Description
The Minecraft Skill Segmentation Dataset contains gameplay trajectories… See the full description on the dataset page: https://huggingface.co/datasets/dami2106/Minecraft-Skill-Data.oai_minecraft_npyminecraft-text-action-datasetminecraft-vpt-mp4
Minecraft VPT MP4 ArrayRecords
Minecraft gameplay clips and their aligned VPT-style actions, packaged as
sharded ArrayRecord files. This is the
raw-video dataset used by the Minecraft data path in dreamer4-jax-private.
Download
Install the Hugging Face CLI and download the repository to a local directory:
pip install -U huggingface_hub
hf download reactor-team/minecraft-vpt-mp4 \
--repo-type dataset \
--local-dir /path/to/mp4-arrayrecords
The downloaded… See the full description on the dataset page: https://huggingface.co/datasets/reactor-team/minecraft-vpt-mp4.minecraft-vla-collect
madokalif/minecraft-vla-collect
Minecraft agent rollout collections: episodes, logs, and videos across scripted tours, combat, build, harvest, threat and trade settings (~68 GB, 37.5k files).
Archived from a working directory so the local copy could be freed; see
MINECRAFT_DATA_INVENTORY.md in this repo for the full per-directory inventory as it stood
at archive time (2026-07-30).
deepseek-v2-codder-minecraft-apiminecraft3minecraft2gamewam-minecraft
GameWAM Minecraft Datasets
This repository contains three Minecraft gameplay datasets used by
GameWAM: A World Action Model for Video Games.
All three use LeRobot v2.1 format and
share a 22-D keyboard/mouse action space, a 6-D raw state (pitch, yaw, cursor
x/y, hotbar, and GUI-open state), and a 15-D canonical proprioceptive
representation.
Project page: https://yunncheng.github.io/GameWAM/Code: https://github.com/yunncheng/GameWAM
Dataset
Directory
Episodes
Frames… See the full description on the dataset page: https://huggingface.co/datasets/Yunncheng/gamewam-minecraft.minecraft-vla-stage1
Minecraft VLA Stage 1: Action Pretraining Data
Vision-Language-Action training data for Minecraft, processed from OpenAI's VPT contractor dataset.
Dataset Description
This dataset contains frame-action pairs from Minecraft gameplay, designed for training VLA models following the Lumine methodology.
Source
Original: OpenAI VPT Contractor Data (7.x subset)
Videos: 17,886 videos (330 hours of early-game gameplay)
Task: "Play Minecraft" with focus on first 30… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/minecraft-vla-stage1.MET-Bench-Minecraft
MET-Bench: Multimodal Entity Tracking for Evaluating the Limitations of Vision-Language and Reasoning Models
Vanya Cohen and Raymond Mooney · ICML 2026
Paper · Publication page · Load the dataset · Citation
Domains: Chess · Shell Game · Minecraft
MET-Bench evaluates entity state tracking across text and image modalities. This repository contains the Minecraft domain.
Minecraft
Minecraft is a state prediction task involving partial observations, dynamic… See the full description on the dataset page: https://huggingface.co/datasets/vanyacohen/MET-Bench-Minecraft.MET-Bench-Minecraft-Trajectories
MET-Bench: Multimodal Entity Tracking for Evaluating the Limitations of Vision-Language and Reasoning Models
Vanya Cohen and Raymond Mooney · ICML 2026
Paper · Evaluation code · Minecraft benchmark · Usage
Benchmark domains: Chess · Shell Game · Minecraft
Minecraft trajectories
This dataset contains the 462 source recordings used to construct the released MET-Bench Minecraft benchmark, comprising 462,235 captured observations. The recordings follow scripted… See the full description on the dataset page: https://huggingface.co/datasets/vanyacohen/MET-Bench-Minecraft-Trajectories.minecraft-vla-sftThis repository contains the dataset used for JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse.
Project Website
Minecraft-Skins-20M
Dataset Card for Minecraft Skins
Dataset Summary
This dataset contains 19,973,928 unique Minecraft player skins collected from various sources. Each skin is stored as a base64-encoded image with a unique identifier.
Dataset Structure
Data Fields
This dataset includes the following fields:
id: A randomly generated UUID for each skin entry. These UUIDs are not linked to any external APIs or services (such as Mojang's player UUIDs) and serve solely as… See the full description on the dataset page: https://huggingface.co/datasets/nyuuzyou/Minecraft-Skins-20M.minecraft-vla-stage2
Minecraft VLA Stage 2: Instruction-Following Data
Stage 2 of the TESS-Minecraft Vision-Language-Action training pipeline.
Overview
This dataset adds task instructions to the Stage 1 visuomotor data, enabling instruction-following training.
Data Format
Field
Type
Description
id
string
Unique sample ID
video_id
string
Source video name
frame_idx
int
Frame index within video
instruction
string
Task instruction (empty for continuation frames)… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/minecraft-vla-stage2.MinecraftWorldFilesminecraft-skins-1.1m-prerenderedminecraftminecraft_480p_9sMinecraft-Schematics
Minecraft Schematics Dataset
This dataset packages a curated collection of Minecraft structures in multiple formats, along with mapping files and a ready-to-use fine-tuning dataset formatted for OpenAI chat models (e.g. for training Gemma-4 / Llama-3 models).
Dataset Directory Structure
schematics/: The raw Minecraft schematic files (supporting .schem, .litematic, and legacy .schematic formats).
blueprints/: The schematic files converted into a clean, parsed 3D… See the full description on the dataset page: https://huggingface.co/datasets/Hack337/Minecraft-Schematics.Minecraft-winter-shaders-Lite
❄️ Minecraft Winter Shaders (Lite)
10,000+ winter Minecraft screenshots with Photon Shaders.
For generative models, style transfer, and high-quality vision tasks.
Structure
Single folder: winter/
Format: 960x540 JPEG
License
Dataset: CC BY NC 4.0.Shader: Photon Shaders by SixthSurge (commercial use of screenshots explicitly allowed).See PHOTON_SHADERS_LICENSE.txt.
minecraft5Minecraft-Skins-Captioned-1M
Dataset Card for Minecraft Skins
Dataset Summary
This dataset contains 981,079 unique Minecraft player skins collected from various sources. Each skin is stored as a base64-encoded image with a unique identifier.
Dataset Structure
Data Fields
This dataset includes the following fields:
hash: A data dependent hash. These hashes are generated from raw bytes and will be same if the skin is identical.
image: The skin image encoded in base64 format.… See the full description on the dataset page: https://huggingface.co/datasets/neurlang/Minecraft-Skins-Captioned-1M.minecraft-motion-action-datasetminecraft-skins-captioned-900k
🎮 minecraft-skins-captioned-900k
854,116 high-quality, captioned Minecraft player skins — deduplicated, Steve-model only, ready for text-to-image training.
📋 Dataset Summary
A rigorously filtered and quality-controlled version of neurlang/Minecraft-Skins-Captioned-1M specifically curated for training high-performance generative models that require precise UV topology constraints.
This dataset is optimized for models like ST-DiT (Sparse Template-Aware… See the full description on the dataset page: https://huggingface.co/datasets/summykai/minecraft-skins-captioned-900k.minecraft-vpt-tokenized
Minecraft VPT Tokenized Latents
Pre-tokenized latent episodes and aligned VPT-style actions derived from the
Minecraft VPT MP4 ArrayRecord dataset. The files are msgpack-encoded
ArrayRecord shards intended for dynamics and world-model training in
dreamer4-jax-private.
Download
pip install -U huggingface_hub
hf download reactor-team/minecraft-vpt-tokenized \
--repo-type dataset \
--local-dir /path/to/tokenized_data_500M
The expected layout is:… See the full description on the dataset page: https://huggingface.co/datasets/reactor-team/minecraft-vpt-tokenized.gamefactory-GF-Minecraft-data-2272Minecraft-vision-dataset
🌍 Minecraft Vision Dataset
Manually collected Minecraft screenshots for classification tasks.
Useful for training AI agents, bots, and computer vision models.
📦 What's inside
Biomes, dimensions, time of day, and more.
See the repository file browser for the full list of classes and current image counts.
📂 Dataset structure
The root contains one folder per class. New classes are added over time.
🔍 Current statistics
Exact numbers… See the full description on the dataset page: https://huggingface.co/datasets/Nininkkka/Minecraft-vision-dataset.