datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
GameplayCaptions
Dataset Card for "Gameplay Captions"
More Information needed
GameplayCaptions-GPT-4VGameplayCaptions-Gemini-pro-visionGameplayCaptions-GPT-4V-V2Gameplay-Walkthrough-QAdoom-e1-internet-gameplayI've used the Inverse Dynamic Model, I've previously trained on manually recorded gameplay, on pure gameplay YouTube videos.
This dataset is in public domain, use it however you want.
Gameplay_Images
Gameplay Images
A dataset from kaggle.
This is a dataset of 10 very famous video games in the world.
These include
Among Us
Apex Legends
Fortnite
Forza Horizon
Free Fire
Genshin Impact
God of War
Minecraft
Roblox
Terraria
There are 1000 images per class and all are sized 640 x 360. They are in the .png format.
This Dataset was made by saving frames every few seconds from famous gameplay videos on Youtube.
※ This dataset was uploaded in January 2022. Game content updated after that… See the full description on the dataset page: https://huggingface.co/datasets/Bingsu/Gameplay_Images.gameplay_get_captionsDiogenes_Gameplay_raw_sample_v01
Diogenes Gameplay Raw Sample v01
Formerly DiogenesLab/Diogenes_COD_sample_v01 — old links redirect here.
A sample dataset. PC gameplay recordings with frame-aligned keyboard/mouse action
annotations, in two batches:
batch
recorded
video
audio
batch 1
2026-07-25/26
1920×1080 @ 30 fps, H.264
none
batch 2
2026-07-31
1920×1080 @ 60 fps, H.264
process-loopback system audio, 48 kHz stereo s16le (zstd-compressed PCM)
This is a sample — the recording output available… See the full description on the dataset page: https://huggingface.co/datasets/DiogenesLab/Diogenes_Gameplay_raw_sample_v01.footsies-gameplay
Footsies Dataset
Game frames from FOOTSIES game.
{episode}_{step}_{p1}_{p2}_{v1}_{v2}.png
can ignore p2, v1 and v2
Dataset Structure
data/train-00000-of-00001.parquet: Metadata
data/images/: PNG frames
Columns
Column
Type
Description
image
str
Path to image file
episode
int
Episode number (0-299)
step
int
Step within episode
p1_action
int
Player 1 action (0=none, 1=back, 2=forward)
p2_action
int
Player 2 action
Statistics… See the full description on the dataset page: https://huggingface.co/datasets/H1yori233/footsies-gameplay.doom-e1-gameplayThis is some crappy gameplay of me playing DOOM (1993) first episode ("Knee Deep in Hell") with VizDoom on a resolution of 320x240 to train an Inverse Dynamic Model on DOOM.
Use it how you want. The gameplay is in the public domain.
action-conditioned-gameplay-10000h
Action-Conditioned Gameplay Dataset
10,000 hours of rights-cleared gameplay trajectories with synchronized video, keyboard/mouse/controller inputs, camera motion, player state, object state, events, goals, rewards, and outcomes for world models and AI agents.
This repository contains the full technical specification, annotation schema, and sample metadata files (Parquet). The production dataset is rights-cleared and delivered directly to buyers. Request access to see the full… See the full description on the dataset page: https://huggingface.co/datasets/Datoric/action-conditioned-gameplay-10000h.connections-gameplay-sft
Connections Gameplay SFT Dataset
A dataset of 714 complete Connections puzzle playthroughs generated by a language model, intended for supervised fine-tuning. Each example is a full game trajectory — from the initial puzzle prompt to the final correct guess — with only winning, valid-guess-only rollouts included.
Puzzles are drawn from the train_sft split of ericbotti/connections-puzzles, intended as a warm-up for an RL training environment.
Overview
Total… See the full description on the dataset page: https://huggingface.co/datasets/ericbotti/connections-gameplay-sft.random-gameplay-imagesDiogenes_Gameplay_multigame_sample_v01
Diogenes Gameplay Multigame Sample v01
Human gameplay recordings across 19 PC titles, captured with the same
recording pipeline and the same per-frame action-annotation schema on every title.
The point of this sample is breadth: one session per title, identical schema,
per-title semantic action vocabularies (mappings/), and honest per-segment QC
columns computed from the shipped data itself.
19 sessions · 603 segments · 1.72 h · 205,091 annotated frames · 332,572 raw input… See the full description on the dataset page: https://huggingface.co/datasets/DiogenesLab/Diogenes_Gameplay_multigame_sample_v01.