datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
computer-use-large
Computer Use Large
A large-scale dataset of 48,478 screen recording videos (~12,300 hours) of professional software being used, sourced from the internet. All videos have been trimmed to remove non-screen-recording content (intros, outros, talking heads, transitions) and audio has been stripped.
Dataset Summary
Category
Videos
Hours
AutoCAD
10,059
2,149
Blender
11,493
3,624
Excel
8,111
2,002
Photoshop
10,704
2,060
Salesforce
7,807
2,336
VS… See the full description on the dataset page: https://huggingface.co/datasets/markov-ai/computer-use-large.vidore_v3_computer_scienceViDoRe V3 : Computer Science
This dataset, Computer Science, is a corpus of textbooks from the openstacks website, intended for long-document understanding tasks. It is one of the 10 corpora comprising the ViDoRe v3 Benchmark.
About ViDoRe v3
ViDoRe V3 is our latest benchmark for RAG evaluation on visually-rich documents from real-world applications. It features 10 datasets with, in total, 26,000 pages and 3099 queries, translated into 6 languages. Each query comes with… See the full description on the dataset page: https://huggingface.co/datasets/vidore/vidore_v3_computer_science.computer-use-data-psai
Computer Use Dataset - PSAI
A large-scale, multimodal dataset of human-computer interactions for training and evaluating AI agents.
🔗 Access Dataset: https://huggingface.co/datasets/anaisleila/computer-use-data-psai
📊 Dataset Overview
This dataset contains 3,167 completed tasks of human-computer interactions captured with video, screenshots, DOM snapshots, and detailed interaction events. Created by Paradigm Shift AI for advancing computer use AI agent research.… See the full description on the dataset page: https://huggingface.co/datasets/anaisleila/computer-use-data-psai.minecraft-vla-stage1
Minecraft VLA Stage 1: Action Pretraining Data
Vision-Language-Action training data for Minecraft, processed from OpenAI's VPT contractor dataset.
Dataset Description
This dataset contains frame-action pairs from Minecraft gameplay, designed for training VLA models following the Lumine methodology.
Source
Original: OpenAI VPT Contractor Data (7.x subset)
Videos: 17,886 videos (330 hours of early-game gameplay)
Task: "Play Minecraft" with focus on first 30… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/minecraft-vla-stage1.computer-use
Computer Use Trajectories
Successful computer-use agent trajectories collected on OSWorld tasks.
Dataset Details
Rows: 160 (one per task trajectory)
Steps: 1,378 total across all trajectories (avg ~8.6 steps/task)
Agent: Gemini 3 Flash Preview with linearized accessibility-tree grounding
Score filter: Only trajectories with score = 1.0 (fully successful)
Domains
Domain
Tasks
Description
chrome
21
Web browsing tasks in Google Chrome
gimp
15
Image… See the full description on the dataset page: https://huggingface.co/datasets/markov-ai/computer-use.minecraft-vla-stage2
Minecraft VLA Stage 2: Instruction-Following Data
Stage 2 of the TESS-Minecraft Vision-Language-Action training pipeline.
Overview
This dataset adds task instructions to the Stage 1 visuomotor data, enabling instruction-following training.
Data Format
Field
Type
Description
id
string
Unique sample ID
video_id
string
Source video name
frame_idx
int
Frame index within video
instruction
string
Task instruction (empty for continuation frames)… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/minecraft-vla-stage2.vidore_v3_computer_science_mteb_format
Vidore3ComputerScienceRetrieval
An MTEB dataset
Massive Text Embedding Benchmark
Retrieve associated pages according to questions.
Task category
t2i
Domains
Academic
Reference
https://huggingface.co/blog/QuentinJG/introducing-vidore-v3
Source datasets:
vidore/vidore_v3_computer_science
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/vidore/vidore_v3_computer_science_mteb_format.Computer-Science-Conversational-Dataset-Indicatari-vla-stage1-5hz
TESS-Atari Stage 1 (5Hz)
Human gameplay demonstrations from Atari games, formatted for Vision-Language-Action (VLA) model training.
Overview
Metric
Value
Source
Atari-HEAD
Games
11 (overlapping with DIAMOND benchmark)
Samples
~4M
Action Rate
5 Hz (1 action per observation)
Format
Lumine-style action tokens
Games Included
Alien, Asterix, BankHeist, Breakout, DemonAttack, Freeway, Frostbite, Hero, MsPacman, RoadRunner, Seaquest… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/atari-vla-stage1-5hz.tess-agentnet
TESS AgentNet Dataset
Computer use trajectories for training Vision-Language-Action models.
Features
image: Screenshot (PIL Image)
instruction: Task description
action_type: 0=MOUSE, 1=KEYBOARD
mouse_x, mouse_y: Normalized coordinates [0,1]
click_type: 0-8 (NO_CLICK, LEFT_CLICK, etc.)
keyboard_text: Text with special tokens
os_type: ubuntu, windows_macos
episode_id, step_idx: Episode structure
Click Types
Index
Type
Description
0
NO_CLICK… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/tess-agentnet.hey-computer-speech-commands
Hey Computer: Speech Command Recognition
Dataset Summary
A public, viewer-ready educational challenge dataset. Host-only scoring data and hidden targets are excluded.
Splits
Split
Examples
Description
train
13,192
Labeled training data
test
3,295
Public inputs with withheld target labels or annotations
Data Fields
Field
Type
audio
Audio
id
string
label
string (test sentinel: unlabeled)… See the full description on the dataset page: https://huggingface.co/datasets/hoangbang/hey-computer-speech-commands.atari-vla-stage1-15hz
TESS-Atari Stage 1 (15Hz)
Human gameplay demonstrations from Atari games with action chunking, formatted for Vision-Language-Action (VLA) model training.
Overview
Metric
Value
Source
Atari-HEAD
Games
11 (overlapping with DIAMOND benchmark)
Samples
~1.3M
Observation Rate
5 Hz
Action Rate
15 Hz (3 actions per observation)
Format
Lumine-style action tokens
Why Action Chunking?
VLA models run at ~5 Hz inference speed, but Atari runs at… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/atari-vla-stage1-15hz.synthetic-computers-at-scale
Synthetic Computers
Paper: Synthetic Computers at Scale for Long-Horizon Productivity Simulation (arXiv:2604.28181)
A dataset of 98 synthetic computer environments designed for research on
computer-use agents, long-horizon planning, and persona-grounded reasoning.
Each row describes a single fictional user's computer — including the user's
persona, professional context, monthly objectives, collaborators, project
portfolio, filesystem policy, full file listing, and a graph of file… See the full description on the dataset page: https://huggingface.co/datasets/microsoft/synthetic-computers-at-scale.computer-agent-arena
Computer Agent Arena: Evaluating Computer-Use Agents via Crowdsourcing from Real Users
Dataset Description
Computer Agent Arena is a comprehensive evaluation platform for multi-modal AI agents, particularly focusing on computer use and GUI interaction tasks. This dataset contains real interaction trajectories from various state-of-the-art AI agents performing complex computer tasks in controlled environments.
The dataset includes:
4,641 agent trajectories across diverse… See the full description on the dataset page: https://huggingface.co/datasets/xlangai/computer-agent-arena.GroceryInContextcomputer-use-large
Computer Use Large
A large-scale dataset of 48,478 screen recording videos (~12,300 hours) of professional software being used, sourced from the internet. All videos have been trimmed to remove non-screen-recording content (intros, outros, talking heads, transitions) and audio has been stripped.
Dataset Summary
Category
Videos
Hours
AutoCAD
10,059
2,149
Blender
11,493
3,624
Excel
8,111
2,002
Photoshop
10,704
2,060
Salesforce
7,807
2,336
VS… See the full description on the dataset page: https://huggingface.co/datasets/txchmechanicus/computer-use-large.csgo-vla-stage1-5hz
CS:GO VLA Stage 1 Dataset (5Hz Chunked)
Vision-Language-Action dataset for Counter-Strike: Global Offensive with action chunking, converted from the TeaPearce CS:GO dataset.
Overview
Frame rate: 5Hz (every 3rd frame)
Action chunking: 3 actions per sample (~200ms coverage)
Total samples: ~1.8M chunks
Split: train / test following Diamond split
Map: Dust2 deathmatch
Action Format
<|action_start|> m1_x m1_y [keys1] ; m2_x m2_y [keys2] ; m3_x m3_y [keys3]… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/csgo-vla-stage1-5hz.html-samplecomputer-agent-trajectoriesmmlu-computer_security
Dataset Card for "mmlu-computer_security"
More Information needed
tess-atari-5hz-384ovos-wake-word-bench-community-computer
OVOS wake_word bench — community-computer
Per-clip detection decisions predictions of the registered
OVOS Plugin Arena
wake_word fighters over
OpenVoiceOS/ovos-community-wakewords-dataset.
One dedicated repo per modality; one dataset split per language; one JSONL
file per fighter under predictions/<lang>/<competitor_id>.jsonl. Rows follow
the arena §3.2 contract (pinned dataset_revision, plugin_version,
latency_ms). Produced by the reproducible benchmark script in the arena… See the full description on the dataset page: https://huggingface.co/datasets/OpenVoiceOS/ovos-wake-word-bench-community-computer.IndustryCorpus2_computer_programming_code
IndustryCorpus2: Programming
This repository contains the IndustryCorpus2: Programming domain subset of BAAI/IndustryCorpus2.
Refer to the parent dataset card for data construction, intended use, limitations,
and licensing details.
Citation
If you use this dataset in your work, please cite IndustryCorpus2:
@misc{shi2024industrycorpus2,
title = {IndustryCorpus2},
author = {Xiaofeng Shi and Lulu Zhao and Hua Zhou and Donglin Hao},
year = {2024}… See the full description on the dataset page: https://huggingface.co/datasets/BAAI/IndustryCorpus2_computer_programming_code.task692_mmmlu_answer_generation_computer_security
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task692_mmmlu_answer_generation_computer_security
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task692_mmmlu_answer_generation_computer_security.ovos-wake-word-bench-picovoice-computer
OVOS wake_word bench — picovoice-computer
Per-clip detection decisions predictions of the registered
OVOS Plugin Arena
wake_word fighters over
Picovoice/wake-word-benchmark.
One dedicated repo per modality; one dataset split per language; one JSONL
file per fighter under predictions/<lang>/<competitor_id>.jsonl. Rows follow
the arena §3.2 contract (pinned dataset_revision, plugin_version,
latency_ms). Produced by the reproducible benchmark script in the arena repo;
the arena's… See the full description on the dataset page: https://huggingface.co/datasets/OpenVoiceOS/ovos-wake-word-bench-picovoice-computer.pi-computer-use-sessions
Coding agent session traces for thomasmustier/pi-computer-use-sessions
This dataset contains redacted coding agent session traces collected while working on https://github.com/tmustier/pi-computer-use. The traces were exported with pi-share-hf from local pi workspaces and filtered to keep only sessions that passed deterministic redaction, secret scanning, visual review where applicable, and LLM review.
Source git repo: https://github.com/tmustier/pi-computer-use
Data… See the full description on the dataset page: https://huggingface.co/datasets/thomasmustier/pi-computer-use-sessions.tess-atari-15hz-384
TESS-Atari Stage 1 - Preprocessed (15Hz, 384x384)
Training-ready version of the 15Hz dataset with images pre-resized to 384x384 (SmolVLM native resolution).
Overview
Metric
Value
Source
TESS-Computer/atari-vla-stage1-15hz
Samples
1,340,293
Image Size
384x384 (pre-resized)
Action Rate
15 Hz (3 actions per observation)
Format
Lumine-style action tokens
Why Preprocessed?
Training VLMs requires resizing images to the model's native… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/tess-atari-15hz-384.computer-vision
Hibou Computer Vision Dataset
The Hibou Project is a drone recognition and localization system.
It is designed to detect and localize drones in real-time, using a combination of audio and video.
Official code repo: Hibou Project
This dataset is designed to train YOLO-based models for drone detection.
Object Classes
ID
Class Name
Ratio
0
Drone
74.17%
1
Other
25.83%
Dataset Description
Column
Description
image
Image from the… See the full description on the dataset page: https://huggingface.co/datasets/Hibou-Foundation/computer-vision.task701_mmmlu_answer_generation_high_school_computer_science
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task701_mmmlu_answer_generation_high_school_computer_science
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task701_mmmlu_answer_generation_high_school_computer_science.vn-provinces-household-computer-rate
Vietnam household computer ownership rate
Vietnam household computer ownership rate. Geographic labels are English (UN/GSO style ASCII romanization). Tables cover provinces, regions and national total where present. Province names follow ar_core.vn_geo (historical 63-province system).
Figures
Hero
Comparison
Color key
Files
provinces (378 rows)
data/provinces.csv
data/provinces.dta
data/provinces.xlsx
national (6 rows)
data/national.csv… See the full description on the dataset page: https://huggingface.co/datasets/letrinhan/vn-provinces-household-computer-rate.