datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
digital-coach
DigitalCoach Dataset
DigitalCoach is a multimodal expert-novice computer-use coaching dataset for studying how humans teach software skills through grounded dialogue.It contains 72 coaching sessions, 22,752 dialogue turns, and 28.1 hours of screen recordings, collected across 5 software applications in creativity, engineering, and productivity-oriented workflows.
Each session pairs one expert coach with one novice learner, and captures timestamped data:
dialogue transcripts… See the full description on the dataset page: https://huggingface.co/datasets/berkeley-hci/digital-coach.nptel_hciA11y-CUA
A11y-CUA Dataset
A11y-CUA is a multimodal desktop interaction dataset for accessibility-focused computer-use agent research. It contains real task trajectories recorded on Windows across two human user groups and two computer use agents (CUAs), each operating under standard and accessibility-specific conditions. Every session captures: timestamped keyboard and mouse events, browser interaction logs, accessibility trees, screen video, and system audio.
The dataset is… See the full description on the dataset page: https://huggingface.co/datasets/berkeley-hci/A11y-CUA.Reduced-A11y-CUA
Reduced A11y-CUA
A structured-data-only subset of the A11y-CUA dataset on HuggingFace.
What's different from the full dataset
The full A11y-CUA dataset includes screen recordings (screen.mp4), system audio (system_audio.wav), and microphone audio (mic_audio.wav) for every session. These files account for the large majority of the dataset's total size.
This reduced version strips all video and audio files. Every other file is identical and complete:… See the full description on the dataset page: https://huggingface.co/datasets/berkeley-hci/Reduced-A11y-CUA.soarm101-feeding-nutsThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so101_follower",
"total_episodes": 80,
"total_frames": 74355,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 30,
"splits": {
"train": "0:80"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/HCIS-Lab/soarm101-feeding-nuts.bi-yam-loosen-bottleThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "bi_yam_follower",
"total_episodes": 50,
"total_frames": 73268,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 30,
"splits": {
"train": "0:50"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/HCIS-Lab/bi-yam-loosen-bottle.bimanual-yam-tactile-vision-demoThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "bi_yam_follower",
"total_episodes": 2,
"total_frames": 1406,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hcisbmm/bimanual-yam-tactile-vision-demo.bimanual-yam-tactile-demoThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "bi_yam_follower",
"total_episodes": 2,
"total_frames": 1348,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:2"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hcisbmm/bimanual-yam-tactile-demo.HCIS-KitchenThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "franka_panda",
"total_episodes": 264,
"total_frames": 413340,
"total_tasks": 3,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:264"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/author31/HCIS-Kitchen.akan_tts_datasetbi-yam-toy-tapeThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "bi_yam_follower",
"total_episodes": 60,
"total_frames": 38643,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 30,
"splits": {
"train": "0:60"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/HCIS-Lab/bi-yam-toy-tape.ugspeech-akan-clean-100hrsyam-pink-utensil-into-cup_20260707_180338This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"left_joint_0.pos",
"left_joint_1.pos",
"left_joint_2.pos",
"left_joint_3.pos",
"left_joint_4.pos",
"left_joint_5.pos",
"left_gripper.pos"… See the full description on the dataset page: https://huggingface.co/datasets/hcisbmm/yam-pink-utensil-into-cup_20260707_180338.eval_yam-loosen-bottle_actThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "bi_yam_follower",
"total_episodes": 3,
"total_frames": 3117,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 30,
"splits": {
"train": "0:3"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hcisbmm/eval_yam-loosen-bottle_act.soarm101-ice-cream-feedingThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 5,
"total_frames": 4358,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:5"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/HCIS-Lab/soarm101-ice-cream-feeding.eval_act_yam-toy-tape-3This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "bi_yam_follower",
"total_episodes": 13,
"total_frames": 10919,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 500,
"fps": 30,
"splits": {
"train": "0:13"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hcisbmm/eval_act_yam-toy-tape-3.HCIS-FinalThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "franka_panda",
"total_episodes": 278,
"total_frames": 402905,
"total_tasks": 3,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:278"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/author31/HCIS-Final.HCIS-TestThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "franka_panda",
"total_episodes": 214,
"total_frames": 388090,
"total_tasks": 2,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:214"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/author31/HCIS-Test.bibletts-asante-twi-max29secs-total9hrs-sr22050
BibleTTS Asante Twi Dataset
Dataset Information
This dataset is derived from the BibleTTS corpus, specifically focusing on Asante Twi speech data. The original BibleTTS is a large, high-fidelity, multilingual, and uniquely African speech corpus.
Total Duration: {total_hours:.2f} hours ({total_hours*60:.1f} minutes)
Number of Files: {file_count:,}
Sample Rate: {sample_rate:,} Hz
Max File Duration: {max_duration:.1f} seconds
Format: WAV files with corresponding… See the full description on the dataset page: https://huggingface.co/datasets/hci-lab-dcug/bibletts-asante-twi-max29secs-total9hrs-sr22050.IIT-AFF-Dataset-Modified
Dataset Card for "IIT-AFF-Dataset-Modified"
More Information needed
digital-twin
Intelligent Home Care Living Lab — Digital Twin
A USD digital twin of the Intelligent Home Care Living Lab (alias: NYCU Room 305), built by the
Human-centered Intelligent System Lab (hcislab) — GitHub @HCIS-Lab.
Authored for NVIDIA Isaac Sim 5.1.
This is a foundation twin: it is the room only — no furniture, no robot, no task props. Every
project references it and layers its own content on top (a robot, furniture, graspables, sensors)
without editing the twin. That keeps one… See the full description on the dataset page: https://huggingface.co/datasets/hcislab/digital-twin.HCIS-CutleryArrangement-Taskbibletts-asante-twi-max29secs-total57hrs-sr22050HCIS-final-fixedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "franka_panda",
"total_episodes": 278,
"total_frames": 402905,
"total_tasks": 3,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:278"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/author31/HCIS-final-fixed.eval_smolvla_yam-loosen-bottleThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "bi_yam_follower",
"total_episodes": 0,
"total_frames": 0,
"total_tasks": 0,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/hcisbmm/eval_smolvla_yam-loosen-bottle.soarm101-feeding-nuts-2This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "so_follower",
"total_episodes": 40,
"total_frames": 33072,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:40"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/HCIS-Lab/soarm101-feeding-nuts-2.HCIS-Cutlery-V1yam-pink-utensil-into-cup_20260707_175706This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"left_joint_0.pos",
"left_joint_1.pos",
"left_joint_2.pos",
"left_joint_3.pos",
"left_joint_4.pos",
"left_joint_5.pos",
"left_gripper.pos"… See the full description on the dataset page: https://huggingface.co/datasets/hcisbmm/yam-pink-utensil-into-cup_20260707_175706.HCIS-ToyblockThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"robot_type": "franka_panda",
"total_episodes": 91,
"total_frames": 192010,
"total_tasks": 1,
"chunks_size": 1000,
"data_files_size_in_mb": 100,
"video_files_size_in_mb": 200,
"fps": 30,
"splits": {
"train": "0:91"
},
"data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/author31/HCIS-Toyblock.jed_akosua