datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
OS-Atlas-data
GUI Grounding Pre-training Data for OS-ATLAS
This document describes the acquisition of the pre-training data used by OS-ATLAS OS-ATLAS: A Foundation Action Model for Generalist GUI Agents.
[🏠Homepage] [💻Code] [🚀Quick Start] [📝Paper] [🤗Models] [🤗ScreenSpot-v2]
Notes: In GUI grounding data, the position of the target element is recorded in the bbox key, represented by [left, top, right, bottom].
Each value is a [0, 1] decimal number indicating the ratio of the… See the full description on the dataset page: https://huggingface.co/datasets/OS-Copilot/OS-Atlas-data.ScreenSpot-v2OSReward
OSReward Benchmark
OSReward evaluates whether a multimodal judge can determine if a computer-use
agent completed a user's task. This release contains binary outcome labels only:
SUCCESS and FAIL.
The benchmark has two evaluation configurations:
Configuration
Trajectories
Unique task IDs
SUCCESS
FAIL
full
1,019
656
440
579
hard
284
257
86
198
Load the dataset
Load either benchmark configuration with datasets:
from datasets import load_dataset… See the full description on the dataset page: https://huggingface.co/datasets/OS-Copilot/OSReward.python-image-copilot-training-using-import-knowledge-graphs
Python Copilot Image Training using Import Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains a png file in the dbytes column.
Rows: 216642
Size: 211.2 GB
Data type: png
Format: Knowledge graph using NetworkX with alpaca text box
Schema
The png is in the dbytes column:
{
"dbytes": "binary"… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-image-copilot-training-using-import-knowledge-graphs.python-image-copilot-training-using-class-knowledge-graphs
Python Copilot Image Training using Class Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains a png file in the dbytes column.
Rows: 312277
Size: 304.3 GB
Data type: png
Format: Knowledge graph using NetworkX with alpaca text box
Schema
The png is in the dbytes column:
{
"dbytes": "binary"… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-image-copilot-training-using-class-knowledge-graphs.editbench
EditBench Dataset
This dataset contains code editing tasks extracted from the EditBench evaluation framework specifically designed for evaluating model performance on code editing tasks. It is provided as a test-only benchmark. Each sample includes:
Please check out https://github.com/waynchi/editbench for our full evaluation harness.
Core Files (Python)
original_code.py: Starting code file
highlighted_code.py: Specific section of code to be modified
instruction.txt:… See the full description on the dataset page: https://huggingface.co/datasets/copilot-arena/editbench.python-text-copilot-training-instruct-ai-research-2024-02-03
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Agora Open Source AI Research Lab:
Agora GitHub Organization
Agora Hugging Face
This dataset is the 2024-02-03 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-02-03.python-audio-copilot-training-using-class-knowledge-graphs
Python Copilot Audio Training using Class with Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each class method has a question and answer mp3 where one voice reads the question and another voice reads the answer. Both mp3s are stored in the parquet dbytes column and the associated source code file_path identifier.
Rows:… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-audio-copilot-training-using-class-knowledge-graphs.python-audio-copilot-training-using-function-knowledge-graphs
Python Copilot Audio Training using Global Functions with Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each global function has a question and answer mp3 where one voice reads the question and another voice reads the answer. Both mp3s are stored in the parquet dbytes column and the associated source code file_path… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-audio-copilot-training-using-function-knowledge-graphs.OS-Shepherd-100K
OSReward Data
OSReward is a multimodal reward-model dataset for judging whether GUI-agent
trajectories complete the user's task. The release contains:
Supervised fine-tuning data in a LLaMAFactory-compatible ShareGPT layout.
Deduplicated screenshots packaged in independently extractable tar shards.
Training and validation data for GRPO.
The expected answer contains a brief evidence-based analysis followed by a
final line in exactly one of these forms:
Judge: SUCCESS
Judge:… See the full description on the dataset page: https://huggingface.co/datasets/OS-Copilot/OS-Shepherd-100K.python-text-copilot-training-instruct
Python Copilot Instructions on How to Code using Alpaca and Yaml
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains python code, either a class method or a global function, imported modules, base classes (if any), exceptions (ordered based off the code), returns (ordered based off the code), arguments (ordered based off the… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct.python-audio-copilot-training-using-import-knowledge-graphs
Python Copilot Audio Training using Imports with Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each imported module for each unique class in each module file has a question and answer mp3 where one voice reads the question and another voice reads the answer. Both mp3s are stored in the parquet dbytes column and the… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-audio-copilot-training-using-import-knowledge-graphs.Causal-Copilot-Dataset
Causal-Copilot: An Autonomous Causal Analysis Agent
[Demo] •
[Code] •
[Technical Report]
News
04/16/2025: We release Causal-Copilot-V2 and the official Technical Report. The new version supports automatically using 20 state-of-the-art causal analysis techniques, spanning from causal discovery, causal inference, and other analysis algorithms.
11/04/2024: We release Causal-Copilot-V1, the first autonomous causal analysis agent.
Introduction
Identifying… See the full description on the dataset page: https://huggingface.co/datasets/Causal-Copilot/Causal-Copilot-Dataset.python-audio-copilot-training-using-inheritance-knowledge-graphs
Python Copilot Audio Training using Inheritance and Polymorphism Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each base class for each unique class in each module file has a question and answer mp3 where one voice reads the question and another voice reads the answer. Both mp3s are stored in the parquet dbytes column and… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-audio-copilot-training-using-inheritance-knowledge-graphs.python-audio-copilot-training-using-class-knowledge-graphs-2024-01-27
Python Copilot Audio Training using Class with Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each class method has a question and answer mp3 where one voice reads the question and another voice reads the answer. Both mp3s are stored in the parquet dbytes column and the associated source code file_path identifier.
Rows:… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-audio-copilot-training-using-class-knowledge-graphs-2024-01-27.python-image-copilot-training-using-inheritance-knowledge-graphs
Python Copilot Image Training using Inheritance and Polymorphism Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains a png file in the dbytes column.
Rows: 259017
Size: 135.2 GB
Data type: png
Format: Knowledge graph using NetworkX with alpaca text box
Schema
The png is in the dbytes column:
{… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-image-copilot-training-using-inheritance-knowledge-graphs.residual_copilot_assets
Residual Copilot — Assets
Robot and object mesh files for Isaac Lab simulation, part of Residual Shared Autonomy.
Contents
robot/
├── xarm7_gripper.usd # XArm 7 + gripper USD (Isaac Lab ArticulationCfg)
├── xarm7.urdf # XArm 7 URDF (Pinocchio IK solver)
└── meshes/ # Visual/collision meshes referenced by URDF
objects/
├── m36_nut.usd # M36 nut USD (NutThread task)
├── m36_bolt.usd # M36 bolt USD (NutThread… See the full description on the dataset page: https://huggingface.co/datasets/shashuo0104/residual_copilot_assets.trsi-calib-d-copilot-mai-20260622-195448python-image-copilot-training-using-class-knowledge-graphs-2024-01-27
Python Copilot Image Training using Class Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains a png file in the dbytes column.
Rows: 312836
Size: 294.1 GB
Data type: png
Format: Knowledge graph using NetworkX with alpaca text box
Schema
The png is in the dbytes column:
{
"dbytes": "binary"… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-image-copilot-training-using-class-knowledge-graphs-2024-01-27.python-image-copilot-training-using-function-knowledge-graphs
Python Copilot Image Training using Function Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains a png file in the dbytes column.
Rows: 134357
Size: 130.5 GB
Data type: png
Format: Knowledge graph using NetworkX with alpaca text box
Schema
The png is in the dbytes column:
{
"dbytes": "binary"… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-image-copilot-training-using-function-knowledge-graphs.python-copilot-training-from-many-repos-large
Python Copilot Large Coding Dataset
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains python code, either a class method or a global function, imported modules, base classes (if any), exceptions (ordered based off the code), returns (ordered based off the code), arguments (ordered based off the code), and more.
Rows: 2350782… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-copilot-training-from-many-repos-large.trsi-calib-c-copilot-gpt5-5-20260623-001054trsi-calib-b-copilot-opus-20260623-001053python-text-copilot-training-instruct-ai-research
Building an AI Copilot Dataset to help keep up with Leading AI Research
This is a specialized, instruction dataset for training python coding assistants on how to code from leading AI/ML open source repositories (2.3M coding samples).
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
This dataset holds the latest coding changes from >1159… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research.OS-Critic-Benchpython-text-copilot-training-instruct-ai-research-2024-02-10
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the multimodal Qwen AI project:
Qwen
Qwen Agent
Qwen VL Chat
Qwen Audio
This dataset is the 2024-02-10 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-02-10.OS-Genesis-mobile-data🫶 If you are interested in our work or find this data helpful, please consider using the following .bib when referencing our paper:
@article{sun2024genesis,
title={OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis},
author={Sun, Qiushi and Cheng, Kanzhi and Ding, Zichen and Jin, Chuanyang and Wang, Yian and Xu, Fangzhi and Wu, Zhenyu and Jia, Chengyou and Chen, Liheng and Liu, Zhoumianze and others},
journal={arXiv preprint arXiv:2412.19723}… See the full description on the dataset page: https://huggingface.co/datasets/OS-Copilot/OS-Genesis-mobile-data.python-text-copilot-training-instruct-ai-research-2024-02-11
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Autogen and multimodal Qwen AI project:
Qwen
Qwen Agent
Qwen VL Chat
Qwen Audio
This dataset is the 2024-02-11 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-02-11.python-copilot-training-on-ai-research-repos
Python Copilot AI Research Coding Dataset
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains python code, either a class method or a global function, imported modules, base classes (if any), exceptions (ordered based off the code), returns (ordered based off the code), arguments (ordered based off the code), and more.
Rows:… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-copilot-training-on-ai-research-repos.trsi-calib-d-copilot-mai-20260623-001056