datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
minecraft-text-action-datasetminecraft-vla-stage1
Minecraft VLA Stage 1: Action Pretraining Data
Vision-Language-Action training data for Minecraft, processed from OpenAI's VPT contractor dataset.
Dataset Description
This dataset contains frame-action pairs from Minecraft gameplay, designed for training VLA models following the Lumine methodology.
Source
Original: OpenAI VPT Contractor Data (7.x subset)
Videos: 17,886 videos (330 hours of early-game gameplay)
Task: "Play Minecraft" with focus on first 30… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/minecraft-vla-stage1.MET-Bench-Minecraft
MET-Bench: Multimodal Entity Tracking for Evaluating the Limitations of Vision-Language and Reasoning Models
Vanya Cohen and Raymond Mooney · ICML 2026
Paper · Publication page · Load the dataset · Citation
Domains: Chess · Shell Game · Minecraft
MET-Bench evaluates entity state tracking across text and image modalities. This repository contains the Minecraft domain.
Minecraft
Minecraft is a state prediction task involving partial observations, dynamic… See the full description on the dataset page: https://huggingface.co/datasets/vanyacohen/MET-Bench-Minecraft.MET-Bench-Minecraft-Trajectories
MET-Bench: Multimodal Entity Tracking for Evaluating the Limitations of Vision-Language and Reasoning Models
Vanya Cohen and Raymond Mooney · ICML 2026
Paper · Evaluation code · Minecraft benchmark · Usage
Benchmark domains: Chess · Shell Game · Minecraft
Minecraft trajectories
This dataset contains the 462 source recordings used to construct the released MET-Bench Minecraft benchmark, comprising 462,235 captured observations. The recordings follow scripted… See the full description on the dataset page: https://huggingface.co/datasets/vanyacohen/MET-Bench-Minecraft-Trajectories.minecraft-vla-stage2
Minecraft VLA Stage 2: Instruction-Following Data
Stage 2 of the TESS-Minecraft Vision-Language-Action training pipeline.
Overview
This dataset adds task instructions to the Stage 1 visuomotor data, enabling instruction-following training.
Data Format
Field
Type
Description
id
string
Unique sample ID
video_id
string
Source video name
frame_idx
int
Frame index within video
instruction
string
Task instruction (empty for continuation frames)… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/minecraft-vla-stage2.minecraft-skins-1.1m-prerenderedminecraft-vla-sftThis repository contains the dataset used for JARVIS-VLA: Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse.
Project Website
minecraft-skins-captioned-900k
🎮 minecraft-skins-captioned-900k
854,116 high-quality, captioned Minecraft player skins — deduplicated, Steve-model only, ready for text-to-image training.
📋 Dataset Summary
A rigorously filtered and quality-controlled version of neurlang/Minecraft-Skins-Captioned-1M specifically curated for training high-performance generative models that require precise UV topology constraints.
This dataset is optimized for models like ST-DiT (Sparse Template-Aware… See the full description on the dataset page: https://huggingface.co/datasets/summykai/minecraft-skins-captioned-900k.minecraft-motion-action-datasetgamefactory-GF-Minecraft-data-2272minecraft-grounding-action-datasetminecraft-motion-coa-datasetminecraft-skins
minecraft-skins
A curated, high-quality dataset of Minecraft character skins with descriptive captions and tags for training generative AI diffusion models (such as Stable Diffusion 1.5, SDXL, and FLUX LoRAs).
Dataset Summary
Total Skins: 101
Image Resolution: 64x64 RGBA (standard Minecraft 1.8+ dual-layer UV layout)
Model Variants: Classic (4px arms) and Slim (3px arms)
Metadata Fields: image, username, uuid, caption, tags, model, quality
Quick… See the full description on the dataset page: https://huggingface.co/datasets/honvert/minecraft-skins.Minecraft-ChopTree
Minecraft Wood-Chopping Scenarios
Dataset d'imitation learning (behavioral cloning) enregistré dans
Minecraft Java 1.21.1 avec un mod Fabric custom.
50 épisodes de coupe d'arbres en survie, 20 Hz, avec randomisation
de l'inventaire, de la vie et de la faim au début de chaque épisode.
Structure
data/scenario-XXXX.parquet : un fichier par épisode, une ligne par tick
meta/scenario-XXXX.json : métadonnées de l'épisode (rayons, fréquence, dimension)… See the full description on the dataset page: https://huggingface.co/datasets/OpenBlock-Team/Minecraft-ChopTree.minecraft-translations
Minecraft Translations
This dataset contains translations to 113 languages extracted from Minecraft assets.
Full list of columns:
af_za
ar_sa
ast_es
az_az
ba_ru
bar
be_by
be_latn
bg_bg
br_fr
brb
bs_ba
ca_es
cs_cz
cy_gb
da_dk
de_at
de_ch
de_de
el_gr
en_au
en_ca
en_gb
en_nz
en_pt
en_ud
enp
enws
eo_uy
es_ar
es_cl
es_ec
es_es
es_mx
es_uy
es_ve
esan
et_ee
eu_es
fa_ir
fi_fi
fil_ph
fo_fo
fr_ca
fr_fr
fra_de
fur_it
fy_nl
ga_ie
gd_gb
gl_es
haw_us
he_il
hi_inhn_no
hr_hr
hu_hu
hy_am
id_id… See the full description on the dataset page: https://huggingface.co/datasets/NetherQuartz/minecraft-translations.minecraft_moddingminecraft-skins-20k-1024k-captionedspatial-minecraftMinecraft-Navigationminecraft-skins-100k-train-readyMinecraft-16x-Datasetgamefactory-GF-Minecraft-data-269minecraft-question-answer-500k
Minecraft Q&A 500k
This is adapted from naklecha with a lot of filtering.
You can find the exact process here
minecraft-grounding-coa-datasetMinecrafter
Dataset Card for "Minecrafter"
More Information needed
minecraft_builds
minecraft builds
this dataset contains Minecraft structure blueprints stored in a Parquet file.
each row includes the structure name, block coordinates (x, y, z), and block type (mat).
source: grabcraft.com
minecraft-skinminecraft-skins-legacy
Minecraft Skin Img2Img Dataset (Legacy Format)
Data Overview
[!NOTE]
This dataset contains skins in the pre-1.8 format, with an image size of 64px x 32px instead of the newer 64px x 64px.
This dataset contains character images paired with a Minecraft skin texture and their descriptions. Character images were generated by DALL-E, prompted with renderings of the Minecraft skin file. All images in this dataset are 1024px x 1024px.
Usage
You can load this dataset… See the full description on the dataset page: https://huggingface.co/datasets/deepwaterhorizon/minecraft-skins-legacy.minecraft-sampleSWE-bench_Verified_FullDataset Summary
SWE-bench Verified is a subset of 500 samples from the SWE-bench test set, which have been human-validated for quality. SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. See this post for more details on the human-validation process.
The dataset collects 500 test Issue-Pull Request pairs from popular Python repositories. Evaluation is performed by unit test verification using post-PR behavior as the reference solution.
The original… See the full description on the dataset page: https://huggingface.co/datasets/MinecraftBrawlStars9/SWE-bench_Verified_Full.