datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
worldmem_minecraft_datasetminecraft-vpt-mp4
Minecraft VPT MP4 ArrayRecords
Minecraft gameplay clips and their aligned VPT-style actions, packaged as
sharded ArrayRecord files. This is the
raw-video dataset used by the Minecraft data path in dreamer4-jax-private.
Download
Install the Hugging Face CLI and download the repository to a local directory:
pip install -U huggingface_hub
hf download reactor-team/minecraft-vpt-mp4 \
--repo-type dataset \
--local-dir /path/to/mp4-arrayrecords
The downloaded… See the full description on the dataset page: https://huggingface.co/datasets/reactor-team/minecraft-vpt-mp4.minecraft-vla-collect
madokalif/minecraft-vla-collect
Minecraft agent rollout collections: episodes, logs, and videos across scripted tours, combat, build, harvest, threat and trade settings (~68 GB, 37.5k files).
Archived from a working directory so the local copy could be freed; see
MINECRAFT_DATA_INVENTORY.md in this repo for the full per-directory inventory as it stood
at archive time (2026-07-30).
minecraft3minecraft2minecraft_480p_9sminecraft5minecraft_8minecraft-vla-videos
HEXTANT — Minecraft VLA Rollout Videos (VPT ranged combat)
OpenHA/HEXTANT 연구용 롤아웃 영상 공개셋. 원거리(활) 전투를 VPT
(MineStudio_VPT.rl_for_shoot_animals_2x)로 롤아웃한 게임플레이 녹화입니다.
학습용 parquet(text_action)·라벨은 별도 private repo에 있고, 여기는 영상만 공개합니다.
환경 다양성 (kite_ranged 시나리오)
바이옴(7): savanna · swamp · plains · desert · taiga · forest · lottery
몹(13): cow · skeleton · sheep · chicken · husk · pig · creeper · zombified_piglin · drowned · zombie · stray · spider · cave_spider
날씨(3):… See the full description on the dataset page: https://huggingface.co/datasets/madokalif/minecraft-vla-videos.minecraft4minecraft6minecraft_9minecraft10MinecraftReferenceVideosminecraft_simpleminecraft-latents
GF-Minecraft DC-AE-lite-f32c32 latents (data_2003)
Pre-encoded latent shards of KlingTeam/GameFactory-Dataset's GF-Minecraft/data_2003 split (mouse + keyboard captures), ready to stream directly into the diffusion world-model trainer in wendlerc/toy-wm-private on branch feat/minecraft.
Skip the encode step — the raw dataset is ~138 GB and encoding takes several hours on 4 A6000/A100 GPUs. These shards let you jump straight into training.
What's in here
32… See the full description on the dataset page: https://huggingface.co/datasets/chrisxx/minecraft-latents.open-p2p-minecraftOpenAI-Minecraft-Contractor
Contractor Demonstrations
Versions
4.16 Update: There is an issue with BFrame in videos, some data is incomplete.
Over the course of the project we requested various demonstrations from contractors
which we release as index files below. In general, major recorder versions change for a new
prompt or recording feature while bug-fixes were represented as minor version changes.
However, some
recorder versions we asked contractors to change their username when recording… See the full description on the dataset page: https://huggingface.co/datasets/zhwang4ai/OpenAI-Minecraft-Contractor.rollout_Minecraft1_20260622_231347This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/2006gk/rollout_Minecraft1_20260622_231347.Minecraft1_recover_20260622_235206This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/2006gk/Minecraft1_recover_20260622_235206.Minecraft_20260622_192114This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/2006gk/Minecraft_20260622_192114.minecraft7minecraft-trajectoryDescription:
samples_v1: 从 contractor 中提取出的 info 作为 instruction 的轨迹，每个 info 最多 10 条
samples_v2: sample_v1 的补充，扩展到每个 info 最多 100 条
samples_v3: samples_v1 和 samples_v2 的补充，扩展到每个 info 最多 200 条
samples_v4: 从 contractor 中提取的 craft_item info 的补充
samples_v5: smelt_item 的合成数据
minecraft-rollouts