Datasetpublic

cy0307/awesome-egocentric-atlas

Use this dataset from datasets import load_dataset ds = load_dataset("cy0307/awesome-egocentric-atlas", split="train") print(len(ds), "resources") print(ds[0]) papers = load_dataset( "csv", data_files="https://huggingface.co/datasets/cy0307/awesome-egocentric-atlas/resolve/main/awesome-egocentric-papers.csv", split="train", ) print(len(papers), "paper-linked resources") Each row is one catalogued resource. Columns: Column Description name Resource name… See the full description on the dataset page: https://huggingface.co/datasets/cy0307/awesome-egocentric-atlas.

sourceHugging Facemitupdated 1mo agoView on Hugging Face

18likes2.9kdownloads

Dataset Card

Use this dataset

python

from datasets import load_dataset

ds = load_dataset("cy0307/awesome-egocentric-atlas", split="train")
print(len(ds), "resources")
print(ds[0])

papers = load_dataset(
    "csv",
    data_files="https://huggingface.co/datasets/cy0307/awesome-egocentric-atlas/resolve/main/awesome-egocentric-papers.csv",
    split="train",
)
print(len(papers), "paper-linked resources")

Each row is one catalogued resource. Columns:

Column	Description
`name`	Resource name
`kind`	`dataset`, `benchmark`, `model`, `toolkit`, or `collection`
`released`	Release date as `YYYY` or `YYYY-MM`
`venue`	Publication venue (conference/journal or `arXiv`/`project page`)
`status`	Access state: `open`, `watch`, `partial`, `benchmark`, or `request`
`scope`	`egocentric` (first-person) or `adjacent` (related, not first-person)
`year`	Release year
`url`	Primary link
`paper`	Paper link, when available
`code`	Code link, when available
`license`	License, when known
`scale`	One-line description of scale / signal / contribution
`tasks`	Task families (`;` -separated)
`modalities`	Modalities (`;` -separated)

The paper-focused sheet is also mirrored as `awesome-egocentric-papers.csv`, containing only resources with a paper link. A richer nested-JSON version with research-area summaries and stats is in `site-data.json`. This dataset card and catalog are mirrored from the GitHub repository and the interactive site; see the full README below for the resource tables and figures.

<h1 align="center">Awesome Egocentric Atlas</h1>

Datasets, benchmarks, models, and tools for egocentric AI.

<a href="README.md">English</a> · <a href="README.zh.md">中文</a> · <a href="README.es.md">Español</a> · <a href="README.fr.md">Français</a> · <a href="README.de.md">Deutsch</a> · <a href="README.ja.md">日本語</a> · <a href="README.ko.md">한국어</a> · <a href="README.pt.md">Português</a> · <a href="CONTRIBUTING.md">Help translate</a> · <a href="https://chaoyue0307.github.io/awesome-egocentric-atlas/">Landing page</a> · <a href="https://huggingface.co/datasets/cy0307/awesome-egocentric-atlas">Hugging Face mirror</a>

Awesome Egocentric Atlas is a practical catalog of egocentric (first-person) datasets, benchmarks, models, and tools for egocentric vision, embodied AI and robotics, vision-language-action, world models, long-context memory, AR/VR, and hand-object interaction. Every entry shows its public-access status, so you can tell at a glance what you can download today and what is still just a paper.

Updated: 2026-08-23. Scope: the main atlas is human or animal first-person capture from head, glasses, headset, body, wrist, handheld, or synchronized ego-exo rigs (where the ego view is central). Related but non-egocentric resources — robot-only datasets, multi-view robotic benchmarks, autonomous-driving 4D data, and general long-video reasoning — are listed separately under Adjacent and Related Resources rather than in the main tables.

—How to Use This Atlas
—At a Glance
—Milestones
—Start Here
—Landscape Snapshot
—Creator and Release Notes
—Status Legend
—Fast Task Map
—Recipes and Reference
—Dataset Atlas
—Benchmarks and Derived Annotations
—Models, Tools, and Baselines
—Adjacent and Related Resources
—Surveys, Papers, and Context
—Workshops and Challenges
—Watchlist
—Inclusion Rules
—Contributing
—Cite This Atlas

How to Use This Atlas

Use the first two tables to orient yourself, then jump into the detailed atlas section that matches your task. Each entry is deliberately short: name, release date (year-month, from the first public/arXiv posting where known), venue (conference/journal where published, or arXiv/source for preprints), scale/signal, best use, and public status. For filtering by task, modality, status, or date, use the catalog in `data/resources.yml`, where every entry carries a released field; label meanings and filter groups are explained in docs/taxonomy.md. Task-oriented research recipes walk you from goal to experiment.

Prefer a browsable view? The interactive site lets you filter the catalog by task, status, and date in eight languages, and the same catalog is mirrored as a dataset on Hugging Face.

At a Glance

Signal	What it means for readers
866 egocentric resources	229 datasets, 142 benchmarks, 442 models, and 43 toolkits, plus 4 collection hubs — across vision, robotics, memory, and AR. 178 related non-egocentric resources are listed separately.
6 research areas	Foundation video, procedure/action, hands and 3D, memory/reasoning, robotics/VLA, and AR/wearable sensing.
5 access states	`open`, `request`, `benchmark`, `partial`, and `watch` keep availability visible before you plan experiments.
Machine-checked catalog	`data/resources.yml` is the source for type, year, status, URL, tasks, and provenance — and CI keeps the public artifacts in sync.
Reader-first tables	Each entry is short enough to scan, then links out to the official page, paper, code, or dataset portal.

Milestones

Representative works to read first, from early egocentric activity datasets to recent models and large-scale corpora.

Start Here

Reader	Best first move	Then go deeper
New to egocentric vision	Read Ego4D, EPIC-KITCHENS-100, Ego-Exo4D, and EgoSchema first.	Add EgoClip/EgoVLP for video-language, VISOR for masks, and HOT3D/HOI4D for 3D hand-object work.
Building a benchmark table	Start from the `open` and `benchmark` entries, then inspect `request` licenses.	Use the YAML catalog to filter by task and modality.
Building an assistant or agent	Start with Ego4D, Xperience-10M, EgoLife, TeleEgo, EgoSchema, MyEgo, EgoIntrospect, EgoBench, UCS-Bench, and Ego2Web.	Track Ego-R1, X-LeBench, EgoMemReason, StreamMemBench, ReFocus, EgoCoT-Bench, EgoBlind, EgoSound, and NoRA for reasoning diagnostics.
Building a robot/VLA system	Start with Xperience-10M, EgoDex, EgoVerse, Ego-Exo4D, HoloAssist, HOI4D, HOT3D, Assembly101, OpenEgo, UMI, FastUMI-100K, EgoPAT3D, HUG, and WiYH.	Add Dobb-E, FastUMI, MV-UMI, UMIGen, YUBI, Hoi!, EgoVLA, EgoEngine, HumanEgo, EgoGuide, Ego-Pi, EgoAERO, H2O, ARCTIC, HOMIE-toolkit, Xperience task baselines, and EgoHandTrajPred for policy/action supervision. For robot-only corpora see Adjacent and Related.
Studying hands, objects, and 3D	Start with HOT3D, HOI4D, H2O, ARCTIC, FPHA, EgoHOS, EgoBody, GIMO, EgoHumans, EgoTactile, and UnrealEgo.	Add EgoTracks/TREK-150 for tracking, EgoForce for monocular 3D hand pose, and EgoEMG/EgoEVHands/EgoPressDiff for emerging sensors and contact.

Landscape Snapshot

Area	What to compare	Canonical anchors
Foundation video	scale, license, modalities, benchmark maturity	Xperience-10M, Ego4D, Ego-Exo4D, EPIC-KITCHENS-100, EgoClip
Procedure and action	action granularity, mistakes, anticipation, gaze	EPIC-KITCHENS, Assembly101, MECCANO, EgoProceL, HoloAssist
Hand-object and 3D	hands, objects, contact, pose, meshes, scene geometry	HOT3D, HOI4D, H2O, ARCTIC, FPHA, EgoHOS
Memory and reasoning	clip length, grounding, QA type, streaming constraints	EgoSchema, EgoLife, TeleEgo, MyEgo, X-LeBench, EgoMemReason, UCS-Bench, StreamMemBench, EgoSound, Ego-R1
Robotics and VLA	action labels, hand trajectories, robot transfer, task diversity	Xperience-10M, EgoDex, EgoVerse, OpenEgo, UMI, FastUMI-100K, YUBI, EgoPAT3D, HUG, WiYH, EgoVLA
AR/wearable sensing	gaze, IMU, SLAM, audio, physiological signals	Project Aria, ADT, AEA, Nymeria, PROSE, EgoIntrospect, EasyCom

Origins. Egocentric vision grew out of wearable computing: Steve Mann pioneered continuous first-person ("sousveillance") cameras through the 1980s–90s and is widely credited with opening up the field, and Microsoft's SenseCam (2006) popularized lifelogging capture. The modern computer-vision wave began at the first IEEE Workshop on Egocentric Vision (CVPR 2009), where Spriggs, De La Torre, and Hebert introduced egocentric activity recognition on the CMU-MMAC database (2009) — the earliest dataset in this atlas — followed by the foundational GTEA (2011) and ADL (2012) datasets that launched first-person activity and object research.

Creator and Release Notes

Resource	Created / stewarded by	Release channel and access note
Xperience-10M	Ropedia	Released under `ropedia-ai` on Hugging Face. Full dataset uses controlled non-commercial access; the sample is open, HOMIE-toolkit loads/visualizes annotations, and Xperience task baselines provide a public evaluation artifact.
Ego4D	Meta AI / Facebook AI with an international academic consortium	Official data portal with license-gated downloads and benchmark tasks.
Ego-Exo4D	Meta and academic partners across the Ego-Exo4D consortium	Official data portal with application/download tooling for synchronized ego-exo data.
EPIC-KITCHENS	University of Bristol-centered EPIC-KITCHENS team	Official challenge/dataset portal with public annotations and benchmark splits.
Project Aria datasets	Meta / Project Aria	Official Project Aria dataset portal and tooling for AR-glasses sensor data.

Status Legend

Status	Meaning
`open`	Public download, public annotations, public code, or application-based access is clearly documented.
`request`	Public project exists, but dataset access requires license, form, approval, or institutional agreement.
`benchmark`	Mainly a benchmark, challenge, labels, or evaluation over existing videos.
`partial`	Some assets are public, but full raw data, license, or long-term availability is unclear.
`watch`	Recent or important resource whose release state still needs verification.

For how each status is decided, see docs/status_policy.md.

Fast Task Map

Research goal	Strong starting points
General egocentric foundation data	Xperience-10M, Ego4D, Ego-Exo4D, EPIC-KITCHENS-100, EgoClip
Ego-exo and skill learning	Ego-Exo4D, EgoExoLearn, Assembly101, Look and Tell
Robot manipulation / VLA	Xperience-10M, EgoDex, EgoVerse, OpenEgo, UMI, FastUMI-100K, YUBI, HUG, EgoVLA, EgoEngine, HumanEgo
Long-context QA and memory	EgoSchema, EgoLife, EgoMemReason, UCS-Bench, StreamMemBench, Ego-EXTRA, EgoStream, EgoBench
Hand-object and 3D HOI	HOT3D, HOI4D, EgoHOS, FPHA, HUG, EgoTactile, EgoForce, EgoEMG
Audio, speech, and social interaction	Ego4D AV tasks, EasyCom, EgoCom / audio-visual correspondence, AV-CONV
AR glasses and scene understanding	Project Aria Datasets, Aria Digital Twin, AEA, Nymeria, DTC

Recipes and Reference

Task-oriented starting points and the references that keep the catalog usable.

Research recipes — each gives a goal, what to start with, what to add, benchmarks, baselines, a minimum viable experiment, common pitfalls, and a reporting checklist:

—Long-context egocentric video QA
—Robot / VLA from egocentric human video
—Hand-object 3D HOI
—AR-glasses and wearable sensing
—Audio and social egocentric understanding

Reference

—Taxonomy — task/modality families and how to filter the catalog.
—Access Matrix — data, code, license, and leaderboard per key resource.
—Status Policy — how open/request/benchmark/partial/watch are decided.
—Resource Schema — every field in data/resources.yml.
—Maintenance — the add → verify → validate workflow.

Dataset Atlas

Foundational and Large-Scale Ego Video

Broad, large-scale first-person corpora that most egocentric pipelines pretrain or benchmark on.

Resource	Released	Venue	Scale / signal	Best for	Status
RekaDaily-10k (raw)	2026-08	Hugging Face	Current incremental release: 7,834 hours, 397,171 unscripted first-person videos, 9,836 WebDataset shards, and about 70 TB	Large-scale daily-life pretraining, action understanding, and human-to-robot learning	open
Ego500	2026-08	Hugging Face	Announced 500-hour corpus with 310K+ structured action annotations across 390 task-environment combinations and 60+ environments; manual access and upload are still pending	Dense work-procedure, hand-state, and task-phase modeling	request
EgoSuite-Open100K	2026-08	Hugging Face	Gated EgoDemo, EgoStandard, and EgoPro buckets advertise 100K hours across head and head-plus-wrist lines with hand/body pose and semantics; 100K remains the collection target, while current manifests are the release truth	Large-scale controlled-access egocentric pretraining and VLA data	request
Stereo-550	2026-07	Hugging Face	550 hours per camera, 1,462 calibrated stereo sessions, 209,315 dense action segments, and synchronized 6-axis IMU for 1,271 sessions	Calibrated stereo HOI, dense video-language learning, and sensor fusion	open
ACE-Data-0	2026-07	arXiv	Current public snapshot has 30 room-scale takes, 95,238 files, and about 3.61 TB of ego/exo video and multimodal sidecars; 150 hours and 75K+ interactions remain the full-release target	Multisensory household imitation, world models, and VLA research	partial
RetailSMV	2026-06	arXiv	32,105 captioned synchronized retail clips from five supermarkets with staff-perspective egocentric and exocentric views	Retail world-model adaptation and ego/exo viewpoint comparison	watch
EgoCS-400K	2026-06	arXiv	400K+ first-person Counter-Strike gameplay videos (10K hours, 13 maps) with aligned actions, player state, camera motion, and game events	Action-conditioned interactive world models from first-person gameplay	watch
HumanNet	2026-05	arXiv	1M-hour human-centric video corpus spanning first-person and third-person views, with captions, motion descriptions, hand/body signals, and a reported 1K-hour egocentric subset for VLA-style validation	Large-scale human-video pretraining, interaction understanding, and human-to-robot transfer	watch
EgoInteract	2026-05	arXiv	Public synthetic release with 10,534 episodes, about 1.9M frames, 61,211 NAO/HOI task frames, and released simulator/training code	Synthetic temporal segmentation, next-active-object detection, anticipation, and EHOI	open
PRISM	2026-03	arXiv	Paper reports 270K multi-view retail SFT samples; the gated PRISM-100K release provides 100K samples, 25,173 ego/exo videos, 21 task types, and about 772 GB	Embodied VLM spatial, physical, and action reasoning in retail settings	request
Ego-1K	2026-03	CVPR 2026	Nearly 1,000 synchronized multiview egocentric videos from a custom 12-camera plus VR-headset rig	Dynamic 3D/4D scene understanding and novel view synthesis from ego rigs	open
EgoCrowds / CrowdEraser	2026-03	arXiv	Semi-synthetic paired crowded/empty clips from real egocentric walking-tour video; CrowdEraser diffusion removes crowds for humanless walkthroughs	First-person walking-tour video editing and environment modeling	watch
Xperience-10M	2026-03	Hugging Face	Ropedia release on Hugging Face; 10M experiences, 10K hours, six video streams, audio, stereo depth, camera pose, hand/body mocap, IMU, hierarchical language, ~1 PB total	Embodied AI, world models, robot learning from human experience, sensor fusion, 3D/4D understanding	request
Xperience-10M Sample	2026-03	Hugging Face	Public sample episode for Xperience-10M with six rows on Hugging Face and `cc-by-nc-4.0` terms	Loader testing, demos, task-suite prototyping, annotation inspection	open
Egocentric-100K	2025-12	Hugging Face	100,405-hour manual-labor egocentric video corpus with 10.8B frames, 2.01M clips, intrinsics, and WebDataset shards	Industrial/manual-labor pretraining, robot learning, and active manipulation density analysis	request
Egocentric-10K	2025-11	Hugging Face	10,000-hour factory-only egocentric video corpus with 1.08B frames, intrinsics, and WebDataset-style shards	Industrial egocentric pretraining and active-manipulation density analysis	request
Voxel51 Egocentric-10K Subset	2025-11	Hugging Face	Open FiftyOne package for a Factory 51 subset of Egocentric-10K, with the card reporting 416 1080p head-mounted industrial clips	Lightweight inspection, prototyping, and visualization of Egocentric-10K-style factory footage	open
Look and Tell	2025-10	NeurIPS 2025 Workshop	25 participants, Project Aria plus stationary cameras, gaze/speech/video, 3D reconstructions	Referential communication across ego and exo viewpoints	watch
EgoBlind	2025-03	NeurIPS 2025 D&B	1,392 first-person videos from blind and visually impaired users, 5,311 questions posed or verified by blind users	Assistive egocentric VideoQA for blind users	watch
HD-EPIC	2025-02	CVPR 2025	41 hours, 9 kitchens, dense fine-grained labels, 3D fixture annotations, audio events, VQA	Detailed kitchen understanding, VQA, 3D-aware ego reasoning, audio-event recognition	open
EgoVid-5M	2024-11	NeurIPS 2025	5M curated egocentric clips at 1080p with fine-grained kinematic and high-level text action annotations (NeurIPS 2025)	Egocentric video generation and action-conditioned world models	open
Ego-Exo4D	2023-11	CVPR 2024	1,286 hours, 740 participants, synchronized first- and third-person views, audio, gaze, IMU, 3D point clouds, language commentary	Skilled activity, ego-exo transfer, cross-view translation, pose, proficiency, robotics	request
HoloAssist	2023-09	ICCV 2023	166 hours, 350 instructor-performer pairs, 7 synchronized streams from mixed-reality headsets	Interactive assistance, mistake detection, intervention prediction, procedural collaboration	open
EgoObjects	2023-09	ICCV 2023	Pilot release with 9K+ videos, 250 participants, 50+ countries, 650K annotations, 368 categories	Category and instance-level egocentric object understanding, continual object detection	open
EgoCom / Ego audio-visual correspondence	2023-07	CVPR 2024	Egocentric video with spatial audio for conversation and audio-visual correspondence tasks	Active speaker detection, spatial audio denoising, conversational graph reasoning	open
Assembly101	2022-03	CVPR 2022	4,321 videos, 8 static plus 4 egocentric views, 100K coarse and 1M fine action segments, 18M 3D hand poses	Procedural action, anticipation, mistake detection, cross-view transfer	open
Ego4D	2021-10	CVPR 2022	3,670+ hours from 900+ camera wearers, multiple countries, video/audio/gaze/stereo/3D/narrations depending on subset	Long-form ego video, episodic memory, social, hand-object, forecasting, audio-visual tasks	request
EasyCom	2021-07	arXiv	AR glasses egocentric multi-channel audio and wide-FOV RGB for noisy conversations	Speech enhancement, source localization, conversation assistance	open
EPIC-KITCHENS-100	2020-06	IJCV 2022	100 hours, 20M frames, 90K actions, 45 kitchens, narrations and dense action labels	Kitchen action recognition, anticipation, action detection, retrieval	open
From Third Person to First Person	2018-12	CVPR 2019	Ego/exo video synthesis and retrieval datasets plus baselines for bridging third-person social video to first-person views	Cross-view synthesis, retrieval, and ego-exo transfer	watch
EPIC-KITCHENS	2018-04	ECCV 2018	55 hours / 11.5M frames from 32 kitchens with narrations, 39.6K action segments, and object boxes	Original large-scale kitchen action recognition and anticipation benchmark	open

Robotics, Manipulation, and VLA

Egocentric human-manipulation and wrist-camera data aimed at vision-language-action models and human-to-robot transfer.

Resource	Released	Venue	Scale / signal	Best for	Status
Egocentric Manufacturing	2026-08	Hugging Face	259 gated clips, about 100 hours, 14 live-factory process domains, 256 workers, 3,911 scenes, and 2,081 entities	Industrial procedures, VLA pretraining, and authentic plant-floor HOI	request
Axis Ego Samples	2026-08	Hugging Face	Public 100-hour LeRobot egocentric sample across nine-plus work domains, plus ten episodes from a separate 150-hour delivery corpus; reuse terms are unstated	Broad workplace human-demonstration and loader experiments	open
RealFactory-Ego-Data	2026-08	Hugging Face	Five Apache-2.0 head-mounted episodes totaling about 14 minutes on a real touchscreen line, with RGB, 21-point hands, and frame-level actions	Compact industrial HOI and action-segmentation prototypes	open
BaseMatrix EGO Desktop Demo	2026-08	Hugging Face	One 70-second, 2,114-frame binocular ego/exo episode with camera pose, metric 3D hands, contact, and derived actions	Inspecting a dense stereo hand-retargeting schema	open
ActTrace Ego-Home	2026-08	Hugging Face	13 household episodes over 11 tasks and 18 minutes with RGB, LiDAR depth, 100 Hz IMU, metric hands, and IMU-verified 6-DoF head trajectories	Metric RGB-D/IMU household demonstrations and sensor fusion	open
Everyday Manipulation 3D	2026-08	Hugging Face	116 densely annotated episodes and 28,019 frames with depth, pose, MANO, contact, masks, and language; companion raw release has 1,513 clips, 10.28 hours, and 279 GiB	Dense 3D HOI evaluation plus raw RGB-D pretraining	open
VITRA Egocentric Instruction Sidecars	2026-08	Hugging Face	95,831 accepted hand-action instructions from 175,327 Qwen3.5 candidates across GigaHands, HOT3D, and OakInk2; source revisions and paths are preserved, but no standalone license is declared	VLA instruction tuning and annotation-sidecar research	partial
InHandPlus Wrist-View Camera and IMU Sample	2026-08	Hugging Face	Auto-gated 30-episode cloth-manipulation sample with wrist video, synchronized raw 6-axis IMU, and LeRobot v3 metadata under CC-BY-NC-4.0	Wrist-view sensor fusion and early loader experiments	request
InfoBayAI Egocentric Video Sample	2026-08	Hugging Face	10 gated MP4 samples and metadata totaling 31.87 GB; the much larger headline collection on the card is not present in the repository manifest	Inspecting first-person sample media without treating the advertised corpus as released	request
UniDataPro Egocentric Video Sample	2026-08	Hugging Face	One public 2.33 GB video/metadata/tracking preview for a commercial collection advertised as 4,050 hours of Pico, ZED, IMU, and motion-tracker capture	Evaluating a commercial multimodal manipulation-data offering	request
Nexdata 10,000-Hour Egocentric Video Dataset	2026-08	Hugging Face	Commercial specification for 10,000 hours of 4K stereo head video, wrist/ankle IMU, 76-point body pose, and dense actions; full media is available by request	Evaluating a large commercial multimodal capture offering	request
ROCO IROS 2026 UMI Dataset	2026-08	Hugging Face	4,928 public LeRobot episodes and 7.46M frames with stereo FPV, four hand cameras, and bimanual pose/gripper state; no card or license is supplied	Bimanual UMI/VLA loading and six-camera policy experiments	partial
DreamTraj / MOVE	2026-08	arXiv	5,038 language-conditioned egocentric 6-DoF object trajectories; predicts motion from unrendered video-diffusion latents at 4.6x the reported speed of generate-then-extract pipelines	Object-trajectory prediction and language-to-manipulation motion	watch
Ego2Robot	2026-08	arXiv	Converts egocentric human manipulation video into 18,561 hours of synthesized robot-format data across 15 morphologies	Scalable ego-to-robot pretraining and embodiment generalization	watch
RoboReact	2026-08	arXiv	Generates egocentric RGB-D manipulation plans, reconstructs interaction keyframes, retargets them to humanoids, and refines execution in closed loop	Generated-video-to-humanoid whole-body skill distillation	watch
SiMDex	2026-08	arXiv	Mines 1.49M task-relevant samples from roughly 32M egocentric clips and reports 47.7% to 61.1% dexterous success over equal-sized random sampling	Relevance-aware human-video curation for dexterous VLA post-training	watch
VLAff / EgoAffordance	2026-08	IROS 2026	204K episodes with 5.6M visual and 11.6M grasp/trajectory affordances; predicts heatmaps, grasps, and executable trajectories	Actionable affordance learning and human-to-robot transfer	watch
JoyAI-RA 0.5	2026-08	arXiv	Dual latent/canonical action alignment across human egocentric video, simulation, and robot data for a generalist VLWA policy	Scaling manipulation learning from heterogeneous human and robot experience	watch
HumynLabs Egocentric Sample Collection	2026-08	Hugging Face	Five CC-BY-4.0 packs covering head/dual-wrist video and IMU, stereo/mono labels, voice-over captions, and regional residential IMU	Small multimodal capture, sensor-fusion, and loader experiments	open
EgoGenesis	2026-07	arXiv	Geometry-aware egocentric world-action simulator; 400 generated plus 400 real trajectories improve reported OOD success from 77% to 84% single-arm and 53% to 70% dual-arm	Controllable ego-video generation and world-action-model augmentation	watch
Firstly Household Manipulation v0.1	2026-07	Hugging Face	12 open dishwashing/laundry episodes totaling 119.1 minutes and 214,437 frames, with 148 phase instructions and two-hand tracks	Compact LeRobot human-demonstration and annotation-pipeline testing	open
HiFi-UMI-2K	2026-07	arXiv	2,000 open hours from a high-fidelity UMI rig with head stereo-inertial SLAM, native inter-gripper pose, microsecond synchronization, and two ultra-wide cameras per hand	Robot-free VLA/WAM pretraining and direct manipulation-policy deployment	open
EgoRecovery	2026-07	arXiv	Egocentric human recovery demonstrations collected at more than 10x robot-teleoperation throughput and aligned with a shared corrective-intent space	Learning when and how robots should recover from manipulation failures	watch
DDD Egocentric Stereo Manipulation Sample	2026-07	Hugging Face	Ten open LeRobot v2 episodes (about 2.37 GB) with stereo first-person video and high-frequency hand/head pose	Stereo human-demonstration loaders, hand tracking, and sensor fusion	open
Showway Egocentric Origami Series	2026-07	Hugging Face	Nine open neck-mounted iPhone origami episodes across three LeRobot v3 repositories, with 3D hand joints and reviewed narration	Compact procedural HOI and imitation-learning experiments	open
EGXO Household Egocentric Video Evaluation	2026-07	Hugging Face	71 household-task videos totaling 10 hours; six previews are public and the complete 43.4 GiB media package is licensed by request	Household procedure and hand-object video evaluation	request
Let the Body Follow	2026-07	arXiv	Coupled egocentric TIAGo teleoperation maps head and arm motion to coordinated torso and mobile-base control, reducing explicit controls and user workload	Whole-body mobile-manipulator teleoperation and HRI	watch
EgoHTR	2026-07	arXiv	55 scene-aligned 4D human-terrain sequences across seven environments, totaling 1.37 hours and about 150K frames from eight participants, with multiview and mocap ground truth	Scene-aware human-motion reconstruction and humanoid terrain traversal	watch
Open-AoE	2026-07	Hugging Face	Nano 3-hour and tiny 100-hour subsets are released; about 323 hours were uploaded and 694 hours prepared by August 12, with 11.9 TB currently stored; 2,000 hours remains the roadmap	Large-scale hand-motion, camera-trajectory, and manipulation pretraining	partial
SenseXperience Raw MCAP Sample	2026-07	Hugging Face	12 public raw ROS 2 MCAP episodes with four synchronized cameras, head IMU near 120 Hz, calibration, metadata, and recorder diagnostics	Raw multimodal capture-pipeline and sensor-synchronization testing	open
Ego Data by Object	2026-07	Hugging Face	4,110 pick-and-place episodes over 51 object labels with egocentric RGB and HDF5 hand/body transforms, intrinsics, task attributes, and descriptions; source licensing is unclear	Object-conditioned hand tracking and imitation-data experiments	partial
EgoExo-Wreck	2026-07	Hugging Face	Commercial card reports 150+ synchronized ego-exo hours of object destruction, debris handling, cleanup, and reset; only a gated 1 GB evaluation sample is hosted	Physics-rich HOI, state-change, world-model, and cleanup demonstrations	request
Verbose Industrial Egocentric Sample Series	2026-07	Hugging Face	Six open industrial sample repositories with 53 first-person clips, 4.86 hours, and about 17.0 GB across textile, metal, electronics assembly, skilled work, and manufacturing operations	Compact industrial action and human-demonstration experiments	open
Verbose Cooking and Chopping Egocentric Video	2026-07	Hugging Face	11 head- or chest-mounted kitchen clips totaling 101.7 minutes in a public 6.09 GB ZIP, with audio and clip metadata across cooking and chopping	Open kitchen action, procedure, and hand-object video experiments	open
Verbose Household Cleaning Egocentric Video	2026-07	Hugging Face	12 head- or chest-mounted 1080p/30 fps cleaning clips totaling 104.3 minutes in a public 5.74 GB ZIP, with audio and three sub-activity labels	Open domestic-action and assistive-robotics video experiments	open
Flikforge Egocentric Household Tasks	2026-07	Hugging Face	Card claims about 100 rights-cleared household clips with dense customizable annotations, but the repository currently has no media or annotation files and its LICENSE is empty	Monitoring a prospective commercial household-demonstration sample release	watch
AgenticFocus	2026-07	arXiv	Restores occluded object geometry and full-hand motion from ordinary first-person human videos, then retargets and composites robot-aligned observations, actions, and states	Mixed-reality human-to-humanoid demonstration synthesis	watch
Hub Egocentric GoPro RGB+IMU	2026-07	Hugging Face	19 GoPro HERO13 manipulation clips totaling 3.72 hours, with RGB, approximately 200 Hz GPMF IMU, sensor sidecars, and MCAP	High-rate visual-inertial human-demonstration prototyping	partial
Hub Egocentric iPhone RGB+IMU	2026-07	Hugging Face	12 iPhone manipulation clips totaling 2.96 hours, with 1080p RGB, CoreMotion/ARKit IMU and attitude, intrinsics, sidecars, and MCAP	Smartphone RGB-IMU synchronization and sensor-fusion experiments	partial
Hub Egocentric Stereo RGB-D	2026-07	Hugging Face	9 ZED X Mini clips with stereo RGB, metric depth, IMU, 6DoF VIO, calibration, MCAP, and per-clip LeRobot v3.0 packages	RGB-D/VIO pipelines and multimodal imitation-learning loaders	partial
Hub Egocentric iPhone RGB	2026-07	Hugging Face	40 household-manipulation clips totaling 1.60 hours, delivered as 1080p H.264 RGB without IMU, depth, pose, or robot actions	Lightweight first-person manipulation-video baselines	partial
Datoric Residential Egocentric Video	2026-07	Hugging Face	Commercial card claims 100,000 hours of 1080p+ household first-person video from Brazilian homes with task, action, object, HOI, and completion-state annotations; public files are specification/sample metadata	Reviewing large-scale household capture specifications and requesting production data	request
Datoric Industrial Egocentric Video	2026-07	Hugging Face	Commercial card claims 50,000 hours of workplace first-person video across logistics, inspection, assembly, and cleaning; public files are specification/sample metadata	Reviewing industrial capture specifications and requesting production data	request
EgoWAM	2026-07	arXiv	Controlled human-robot WAM co-training study comparing pixel, DINO, and 3D-flow targets on three real bimanual tasks, with up to 4x OOD gains reported for DINO	Human-to-robot world-representation transfer	watch
USA Egocentric	2026-07	Hugging Face	Three iPhone kitchen-manipulation episodes with RGB videos, camera trajectories, hand outputs, action segments, object/contact tracks, and quality reports	Open manipulation-pipeline debugging and VLA annotation prototyping	open
RynnWorld-4D	2026-07	arXiv	4D embodied world model over future RGB, depth, and optical flow; reports Rynn4DDataset 1.0 with 254.4M+ frames across egocentric human and robotic manipulation videos	Robot manipulation world modeling and action-conditioned 4D prediction	watch
RynnWorld-Teleop	2026-07	arXiv	Digital-teleoperation world model driven by operator hand-pose streams to synthesize egocentric videos and action labels	Scalable bimanual robot data generation from human pose streams	watch
LingBot-VLA 2.0	2026-07	arXiv	VLA update trained with around 60K hours, including 10K hours of egocentric human videos and 50K hours of robot trajectories	Whole-body VLA pretraining and practical robot deployment	watch
LingBot-Video	2026-07	arXiv	MoE DiT video foundation model for embodied intelligence, adding robot-oriented manipulation/navigation and egocentric-perspective footage plus physically grounded rewards	Embodied video pretraining and robot/world-model generation	watch
LIME	2026-07	arXiv	Intent-aware camera-motion generator mined from egocentric video, pairing language intents with relative SE(3) target poses	Active camera control and next-view prediction for embodied agents	watch
H-Tac / TTP	2026-07	arXiv	160-hour egocentric human tactile-action dataset with 300+ tasks and 135K episodes, plus transferable tactile pretraining	Contact-rich dexterous manipulation from human tactile demonstrations	watch
HomER v2	2026-06	Hugging Face	765 public first-person household videos totaling 99.60 hours from 420 participants across 49 countries and 10 activity categories	Diverse everyday activity, hand-object interaction, video-language, and robot-learning research	open
Ego-Tactile Manipulation	2026-06	Hugging Face	72 public episodes totaling 1.28 hours and 138,629 frames with 1080p ego video, 160-channel dual-hand pressure, head/wrist IMU, and contact-derived action segments	Open tactile-grounded manipulation, contact, and action-segmentation research	open
EgoSteer	2026-06	arXiv	Open full-stack dexterous VLA system using 9.6K hours of curated egocentric human video, with EgoSmith, a robot stack, and two released 3B checkpoints	Steerable dexterous manipulation and human-video-to-robot pretraining	open
Poseidon First-Person Task Video Samples	2026-06	Hugging Face	Five auto-gated 1080p clips totaling 174 seconds across kitchen, gardening, maintenance, fine-motor, and cleaning tasks	Reviewing task diversity, hand-object visibility, and capture quality	request
WT-UMI	2026-06	arXiv	Wearable whole-body tactile UMI interface aligning human demonstrations and teleoperation through tactile images, contact forces, and end-effector poses	Contact-aware humanoid whole-body manipulation	watch
Human-as-Humanoid	2026-06	arXiv	Synchronized ego-exo human videos converted through motion recovery, IK, and FK-aware supervision into 60-DoF humanoid action chunks	Zero-shot humanoid VLA learning from human demonstrations	watch
UMI-Bench 1.0	2026-06	arXiv	Local-first real-robot evaluation protocol dedicated to UMI-style data collection, deployment, reset, logging, and task-factor analysis	Reproducible UMI policy benchmarking	watch
VISTA UMI	2026-06	arXiv	Adapts UMI data for VLA training with wrist-fisheye UMI-VQA visual grounding plus physics-based trajectory validation	Making UMI demonstrations usable for VLAs	watch
Pose6DAug	2026-06	arXiv	Physically plausible multi-view object-swapping augmentation for VLA policies under occlusion and egocentric viewpoints	Robust manipulation data augmentation	watch
MotionWAM	2026-06	arXiv	Real-time humanoid world-action model driven by a single egocentric camera and unified whole-body motion tokens	Humanoid loco-manipulation from egocentric perception	watch
OSCAR	2026-06	arXiv	Omni-embodiment action-conditioned video world model trained across robotics and egocentric human datasets	Robot policy evaluation and cross-embodiment world modeling	watch
Mem-World	2026-06	arXiv	Wrist-view-centered 4D surfel memory for persistent action-conditioned manipulation world models	Long-horizon wrist-camera world modeling	watch
PAIWorld	2026-06	arXiv	3D-consistent multi-view world foundation model across egocentric, eye-to-hand, and wrist cameras	Multi-view robotic world modeling	watch
Qwen-RobotManip	2026-06	arXiv	38,100-hour manipulation pretraining corpus converting egocentric hand demonstrations into robot trajectories across 15 platforms	Scalable VLA robot manipulation	watch
CAIP	2026-06	arXiv	Contrastive action-image pretraining from 32,041 hours of egocentric human video with 3D hand-keypoint action proxies	Action-centric visual pretraining for robotics	watch
YUBI	2026-06	arXiv	Reported 8,434 hours, 1.20M episodes, and 119 tasks from a finger-aligned UMI-style bimanual interface	Scalable bimanual dexterous data collection	watch
HumanoidUMI	2026-06	arXiv	VR-assisted egocentric data capture for whole-body humanoid imitation, using sparse human keypoint trajectories and wrist-view observations for whole-body transfer	Whole-body humanoid transfer from egocentric human demonstrations	watch
BRIDGE State-Gated Experts	2026-06	arXiv	Combines handheld UMI demonstrations with targeted teleoperated segments through state-gated diffusion-policy experts	Contact-rich manipulation with mixed UMI and teleoperation supervision	watch
ForceBand	2026-06	arXiv	10-hour wrist-worn sEMG, egocentric video, IMU, and fingertip-force dataset for force-enriched demonstrations	Force-aware human demonstration collection for robot policies	watch
EgoEngine	2026-06	arXiv	Converts egocentric human manipulation videos into high-fidelity robot observation videos and executable robot action trajectories	Human-to-robot data generation, dexterous imitation	watch
EgoAERO	2026-06	arXiv	Asset-free conversion from a single egocentric RGB-D demonstration; introduces EgoDex-R in the paper	Single-demo dexterous robot learning	watch
1M-HUGs / HUG	2026-06	arXiv	1M frames / 27.8 hours of smart-glasses human grasps over 6,707 object instances, plus HUG-Bench with 90 unseen objects	Human grasp modeling and zero-shot robot grasping	open
HALOMI	2026-06	arXiv	Extends UMI-style demonstration collection with egocentric head/wrist observations and head-hand trajectories for humanoid loco-manipulation	Active-perception humanoid manipulation from human demonstrations	watch
HumanoidArena	2026-06	arXiv	Benchmark for egocentric hierarchical whole-body learning across seven leg-critical humanoid-object and humanoid-scene interaction tasks	Whole-body humanoid control from egocentric perception	watch
EgoStation Smartphone Raw Catalog v1	2026-05	Hugging Face	Metadata for 5,962 episodes and 693.9 hours, with 16 downloadable full-quality samples carrying raw video, hand keypoints, and depth overlays	Inspecting smartphone capture quality and requesting selected raw demonstrations	partial
Luel IMU Egocentric 250k	2026-05	Hugging Face	Commercial card reports 250K hours and 46,340 tasks; the gated Hub repository exposes a verifiable 10-clip, 30-minute sample plus the full task taxonomy	Reviewing industrial RGB-IMU capture scope and requesting the proprietary full corpus	request
RoboX-EgoTask	2026-05	Hugging Face	17 public RGB-D task clips from five recordings with hand signals, camera pose, IMU, action segments, trajectories, segmentation, and quality reports	Open multimodal hand-object interaction and human-demonstration prototyping	open
SenseXperience UMI Human Demonstrations	2026-05	Hugging Face	1,051 public human-demonstration episodes, 220,963 frames, and 92 tasks with six synchronized head, wrist, depth, and gripper views plus 7-DoF poses in LeRobot format	Open human-to-robot transfer, bimanual manipulation, and VLA data	open
tau0-WM	2026-05	arXiv	Unified video-action world model trained on real-robot teleoperation, UMI-style interaction, egocentric human video, and rollout/failure data	Future-aware robot action generation and action-conditioned simulation	watch
SABER	2026-05	arXiv	100+ hours of in-store retail egocentric action capture paired with 360-degree exocentric context	Retail-domain VLA adaptation and manipulation data scaling	watch
Qwen-VLA	2026-05	arXiv	Unified VLA model trained over manipulation trajectories, egocentric human demonstrations, simulation, navigation, and trajectory supervision	General embodied action modeling	watch
Gaze2Act	2026-05	arXiv	Maps first-person gaze into robot perspective for target specification in interactive manipulation	Gaze-conditioned VLA manipulation	watch
DeMiAn	2026-05	arXiv	Dense multi-aspect language re-annotation over 1M robot clips and 50K EgoVerse human-egocentric videos	Language-dense robot policy learning	watch
Grid Egocentric Residential Samples	2026-05	Hugging Face	Public first-person residential manipulation preview pack with full-length 1080p household task videos	Household VLA capture-quality review and imitation-learning samples	open
Industrial Workplace Egocentric FHD Samples	2026-05	Hugging Face	21 rights-cleared head-mounted workplace clips with paired JSON metadata across factory, construction, services, retail, transit, and food-service domains	Industrial and vocational human-demonstration samples for VLA/WAM prototyping	open
UniDataPro Egocentric Video Preview	2026-05	Hugging Face	Commercial preview card for a claimed 4,050-hour VR-headset/Zed first-person robotics collection; public files expose a sample video, CSV, and tracking text	Market scan and modality reference for large-scale egocentric robotics capture	partial
Lo6yu Egocentric RGB-D + EMG/IMU	2026-05	Hugging Face	Eight household activity packages with RGB-D, wrist EMG/IMU, hand keypoints, masks, contact, per-finger force, semantic segments, and training-ready exports	Multimodal contact-aware imitation learning and sensor-fusion robot data	open
Egocentric Adjust Bottle LeRobot	2026-05	Hugging Face	LeRobot-format human_mano adjust-bottle task with 500 episodes and 36,897 frames across image, text, timeseries, and video exports	Loader tests and small-scale LeRobot manipulation policy prototyping	open
Low-Resolution Active Perception BC	2026-05	arXiv	Behavior cloning with low-resolution wrist-mounted egocentric RGB for object finding and grasp triggering	Active perception under low compute	watch
EgoSPT / SPOT	2026-05	arXiv	Egocentric spatially prompted manipulation trajectories with first-frame object/target grounding and 3D end-effector motion	Spatially grounded manipulation trajectory prediction	watch
Mobile UMI	2026-05	arXiv	Robot-free mobile manipulation capture with chest-centric context, wrist-centric interaction, decoupled kinematics, and latency-aware diffusion policy	Mobile manipulation from portable demonstrations	watch
BifrostUMI	2026-05	arXiv	VR-device robot-free demonstrations with sparse keypoint trajectories, wrist-mounted visual data, and humanoid retargeting	Humanoid whole-body demonstration collection	watch
UniT	2026-04	arXiv	Unified latent action tokenizer anchoring massive egocentric human data to humanoid policy learning and world modeling	Human-to-humanoid transfer	watch
XRZero-G0	2026-04	arXiv	VR and dual-gripper robot-free collection system with a reported 2,000-hour dataset and data-mixing study	Scalable robot-free manipulation data collection	watch
EgoVerse	2026-04	arXiv	1,362 hours, 80K episodes, 1,965 tasks, 240 scenes, 2,087 demonstrators	Human demonstration scaling for robot learning and VLA	watch
RoboX Egocentric Collection	2026-04	Hugging Face	7,342 auto-gated first-person clips across grasping, daily activities, scene capture, and navigation with metadata, hand keypoints, object tracks, action segments, IMU, and camera pose	Multi-task robotics imitation data and navigation/manipulation benchmark prototyping	request
EgoLive	2026-04	arXiv	Large-scale real-world task-oriented egocentric routines for robot manipulation	Home service, retail, and real-world work-task manipulation	watch
GazeVLA	2026-04	arXiv	VLA policy that pretrains on large-scale egocentric human data to capture gaze, intention, and action before robot fine-tuning	Gaze- and intent-aware robot manipulation	watch
WARPED	2026-04	arXiv	Wrist-aligned rendering converts monocular egocentric human demonstrations into robot policy observations with 3D Gaussian Splatting	Cross-embodiment imitation from human ego video	watch
UMI-3D	2026-04	arXiv	Extends UMI with wrist LiDAR, LiDAR-centric SLAM, synchronized sensing, and spatiotemporal calibration for robust 3D demonstration capture	3D-aware UMI data collection	watch
OmniUMI	2026-04	arXiv	Adds RGB, depth, trajectory, tactile sensing, grasp force, and external wrench signals to a human-aligned UMI-style handheld system	Contact-rich multimodal UMI demonstrations	watch
HRDexDB	2026-04	arXiv	1.4K human/robot grasping trials, tactile, multiview video, egocentric video streams	Cross-domain dexterous grasp learning	watch
UMI-Underwater	2026-03	arXiv	Transfers on-land handheld human demonstrations to underwater grasping through depth-based affordance representations	Underwater manipulation without underwater teleoperation	watch
HoMMI	2026-03	arXiv	Whole-body mobile manipulation interface augmenting UMI with egocentric sensing, relaxed head actions, and cross-embodiment hand-eye policy design	Robot-free mobile manipulation demonstrations	watch
Ego-Exo Manufacturing	2026-03	Hugging Face	Gated 275-hour shoe-manufacturing corpus with 40 synchronized ego-exo groups, 80 standalone ego sessions, 137 exo sessions, and procedure/proficiency/mistake annotations	Industrial ego-exo learning, skill assessment, and procedural modeling	request
OBayData Egocentric Dexterous Manipulation Demo	2026-03	Hugging Face	500 train and 100 test first-person manipulation sessions with head and wrist videos, 3D hand joints, camera extrinsics, action labels, and language annotations	Quality review and prototyping for dexterous human-demonstration pipelines	open
HuMI	2026-02	arXiv	Portable robot-free whole-body demonstration interface with open code and seven Hugging Face datasets across kneeling, squatting, tossing, walking, and bimanual tasks	Humanoid whole-body manipulation from portable human demonstrations	open
Seesaw Video Subset	2026-02	Hugging Face	763 gated LeRobot-compatible indoor episodes, including 243 ego and 520 exo views, 321,178 frames, four room settings, and smartphone AR camera pose	Ego-exo robot learning, action recognition, and data-conversion experiments	request
CoMe-VLA	2026-02	arXiv	Cognitive and memory-aware VLA that learns active-perception strategies from large-scale egocentric human data	Non-Markovian active perception and manipulation	watch
EgoAVFlow	2026-02	arXiv	Learns manipulation and active camera control from egocentric human videos through shared 3D flow	Active-vision robot policy transfer	watch
EgoHumanoid	2026-02	arXiv	Co-trains humanoid VLA policies from robot-free egocentric human demonstrations and limited robot data	Humanoid loco-manipulation	watch
EgoActor	2026-02	arXiv	Grounds high-level instructions into spatially aware egocentric humanoid actions	Humanoid task planning and action grounding	watch
AoE: Always-on Egocentric	2026-02	arXiv	Always-on egocentric human-video collection pipeline and corpus for embodied AI	Scaling human-video data for robot learning	watch
EgoScale	2026-02	arXiv	20,854 hours of action-labeled egocentric human video with a human-to-robot two-stage transfer recipe and a log-linear data-scaling law	Scaling dexterous manipulation from human video	watch
Manus Egocentric Sample	2026-01	Hugging Face	20 LeRobot-format manipulation episodes with egocentric RGB/depth, Manus glove tracking, IMU, timeseries, and videos	Open egocentric manipulation loader tests and sensor-fusion policy prototyping	open
HoyerChou Egocentric Bimanual Videos	2026-01	Hugging Face	YOTO/YOTO++ release with three raw first-person bimanual videos, 119 segmented demos, 241 rows, MANO hand meshes, 3D keypoints, calibration, and camera/world transforms	One-shot bimanual manipulation and hand-trajectory extraction from human video	open
Egocentric Specialties	2026-01	Hugging Face	Auto-gated skilled-jewelry manufacturing data card describing synchronized egocentric and top-down footage for diamond setting and specialty workflows	Skilled industrial manipulation and fine-grained manufacturing demonstrations	request
X-Humanoid	2025-12	arXiv	Applies human-to-humanoid video translation to 60 hours of Ego-Exo4D, releasing 3.6M robotized humanoid frames	Humanoid video/world-model data generation	watch
DreamTacVLA	2025-12	arXiv	Grounds VLA policies in contact physics using tactile images, wrist-camera local vision, and third-person macro vision	Contact-rich VLA manipulation	watch
TacThru-UMI	2025-12	arXiv	See-through-skin tactile-visual sensing plus a UMI-style imitation-learning framework for multimodal manipulation	Tactile-visual robot policy learning	watch
Mitty	2025-12	arXiv	Diffusion Transformer for translating human demonstrations into robot-execution videos, with synthesis from large egocentric datasets	Human-to-robot video generation	watch
World in Your Hands / WiYH	2025-12	arXiv	Reported 1,000-hour egocentric manipulation ecosystem with multiview/depth/hand/wrist signals	VLA, manipulation representation learning	watch
Hoi!	2025-12	arXiv	3,048 force-grounded articulated-manipulation sequences over human, wrist-camera, UMI, and Hoi-gripper embodiments	Force/contact-grounded cross-view manipulation	watch
METIS	2025-11	arXiv	Dexterous VLA pretrained on multi-source egocentric human and robotic data unified under one action space	Dexterous egocentric-VLA pretraining	watch
Let Me Show You / RfV	2025-11	IROS 2025	Retrieves egocentric human demonstrations and extracts affordance masks plus hand trajectories for robot policies	Retrieval-from-video robot manipulation	watch
UMIGen	2025-11	arXiv	Cloud-UMI plus visibility-aware egocentric point-cloud generation	3D-aware cross-embodiment imitation learning	watch
EgoMI	2025-10	arXiv	Egocentric manipulation interface capturing synchronized end-effector and active-head trajectories for semi-humanoid robots	Active-vision whole-body manipulation	watch
FARM	2025-10	arXiv	Force-aware diffusion policy using a modified tactile UMI gripper and matched actuated deployment gripper	Force- and tactile-grounded manipulation	watch
UMI-on-Air	2025-10	arXiv	Embodiment-aware diffusion policy adapting handheld UMI demonstrations to aerial manipulators	UMI transfer under embodiment dynamics	watch
EmbodiSwap	2025-10	arXiv	Photorealistic robot overlays over ego-centric human video for synthetic robot imitation datasets	Zero-shot human-video-to-robot imitation	watch
Hand-VLA Pretraining	2025-10	arXiv	Converts unscripted real-life egocentric hand videos into 1M VLA-aligned action-language episodes	Robot pretraining from human activity video	watch
ActiveUMI	2025-10	arXiv	Portable VR/UMI-style system capturing active egocentric perception for bimanual manipulation	Active-perception UMI-style data collection	watch
FastUMI-100K	2025-10	arXiv	100K+ UMI-style trajectories across 54 tasks in LeRobot format with multi-view wrist fisheye images	Large-scale UMI-style robot policy training	open
BiNoMaP	2025-09	arXiv	Category-level bimanual non-prehensile primitives learned from egocentric hand trajectories and transferred across two dual-arm platforms	Contact-rich bimanual skill extraction from human demonstrations	watch
EMMA	2025-09	IEEE RA-L 2025	Co-trains egocentric human mobile-manipulation data with static robot data to avoid mobile teleoperation bottlenecks	Scalable mobile manipulation from human data	watch
OpenEgo	2025-09	arXiv	1,107 hours unified across six public egocentric datasets, 290 manipulation tasks, hand-pose/action primitives	Standardized dexterous manipulation pretraining and evaluation	watch
MV-UMI	2025-09	arXiv	Multi-view UMI adds third-person context to wrist egocentric observations	Broad-scene context for cross-embodiment manipulation	watch
InterVLA	2025-08	ICCV 2025	11.4 hours and 1.2M frames with 2 ego and 5 exo views, human/object motions, and verbal commands	Vision-language-action and motion estimation	watch
EgoVLA	2025-07	arXiv	VLA training from egocentric human videos plus robot fine-tuning	Human-video-to-robot policy transfer	watch
EgoDex	2025-05	ICLR 2026	829 hours of Apple Vision Pro egocentric video, 194 tabletop tasks, 3D hand/finger tracking	Dexterous manipulation, imitation learning, human-to-robot hand trajectory prediction	watch
TASTE-Rob	2025-03	CVPR 2025	100,856 task-oriented ego-centric hand-object interaction videos aligned with language instructions	Video generation and robot imitation learning	watch
Kaiwu	2025-03	IEEE RA-L 2025	11,664 synchronized assembling-action instances with first-person video, gaze, hand motion, pressure, audio, EMG, mocap, and robot streams	Multimodal robot learning and HRI	watch
Humanoid-VLA	2025-02	arXiv	Combines language, egocentric scene perception, and motion control for universal humanoid control	Humanoid VLA and motion generation	watch
Immersive Social Interaction with VR and LLM-Assisted Humanoids	2024-11	Humanoids 2024 Workshop	Apple Vision Pro teleoperation with egocentric robot feedback, voice locomotion, wrist/finger retargeting, and RGB, gaze, command, joint-state, and hand-motion recording on a Unitree H1	Immersive humanoid teleoperation and multimodal demonstration capture	watch
EgoMimic	2024-10	ICRA 2025	Project Aria human videos with 3D hand tracking and robot co-training for unified imitation learning	Human-to-robot imitation from ego video	watch
FastUMI	2024-09	arXiv	UMI redesign reporting 10K+ real-world trajectories across 22 everyday tasks	Faster hardware-independent UMI-style collection	watch
R+X	2024-07	ICRA 2025	Retrieves executable skills from long unlabeled first-person videos of everyday human tasks	Robot imitation from everyday first-person video	watch
UMI on Legs	2024-07	arXiv	Combines handheld UMI task demonstrations with simulation-trained whole-body controllers for quadruped manipulation	Mobile cross-embodiment UMI transfer	watch
EgoPAT3D / EgoPAT3Dv2	2024-03	ICRA 2024	1M+ RGB-D/IMU frames in the original task; v2 expands egocentric 3D action-target prediction	Human-robot interaction, 3D target anticipation, manipulation safety	open
Universal Manipulation Interface / UMI	2024-02	RSS 2024	Hand-held GoPro gripper interface and policy stack for in-the-wild robot teaching	Wrist-view robot-free demonstrations and cross-embodiment policy transfer	open
Dobb-E / HoNY	2023-11	arXiv	13 hours, 22 homes, 5,620 trajectories, and 1.5M RGB-D frames collected with an iPhone-based Stick	Household robot learning from handheld in-home demonstrations	partial
VIP	2022-09	ICLR 2023	Value-Implicit Pre-Training learns visual rewards and representations from Ego4D human video for few-shot robot manipulation	Ego4D-to-robot reward learning	open
R3M	2022-03	CoRL 2022	Robot-manipulation visual representation pretrained on Ego4D with time-contrastive and video-language objectives	Ego4D-to-robot representation learning	open

Hands, Objects, Pose, Tracking, and 3D

Fine-grained hand, object, contact, and 3D-pose datasets, including emerging event-camera and EMG sensing.

Resource	Released	Venue	Scale / signal	Best for	Status
HOPE	2026-08	arXiv	Predicts temporally evolving per-vertex hand pressure and contact from monocular hand-centric video by unifying tactile-glove, planar-pressure, and contact labels	Dynamic hand-object pressure and contact estimation	watch
EgoGVAE	2026-07	ECCV 2026	Guided variational autoencoder reconstructing full-body meshes from head pose with one-step sampling and reported 50x+ faster inference than diffusion alternatives	Fast head-conditioned ego-body mesh recovery	watch
ReViV	2026-07	ECCV 2026	Open unified feed-forward reconstruction of camera trajectory, gaze, body, hands, depth, and view dynamics from one monocular egocentric RGB stream	Holistic and efficient viewer-plus-scene 4D reconstruction	open
Egocentric Hand Benchmark Annotations	2026-07	Hugging Face	100,427 public annotation rows combining 95,002 EgoDaily hand boxes and 5,425 HOT3D virtual hand-crop camera records; source RGB is not redistributed	Hand detection, crop-camera geometry, and derived benchmark evaluation	partial
EgoExoMoCap	2026-07	ECCV 2026	Distributed mocap from two or more smart-glasses wearers, fusing head/wrist tracking with context-aware image features on two in-the-wild datasets	Lightweight ego-exo body-motion reconstruction	watch
Exo2EgoPose	2026-07	ACM MM 2026	Vision-language-guided egocentric 3D hand-pose forecasting using reconstructed exocentric demonstrations, evaluated on AssemblyHands, Ego-Exo4D, EgoMe-pose, and CALVIN	Hand forecasting and human-to-robot transfer	watch
EgoObjects-Camera	2026-07	Hugging Face	Public camera-parameter annotations for 241,554 EgoObjects images across 49 shards, including roll, pitch, vertical field of view, and radial distortion; reuse terms are undeclared	Camera-aware egocentric modeling and dataset analysis	partial
TSR-Ego	2026-07	arXiv	Causal temporal stereo refinement with fisheye deformable cross-attention for online egocentric 3D body pose on UnrealEgo2 and UnrealEgo-RW	Head-mounted stereo pose recovery under occlusion and truncation	watch
CI-HOI / DEHOI	2026-07	arXiv	Cue-isolated HOI evaluation and inpainted testbed that separate hand and object evidence to expose contextual shortcuts in egocentric VLMs	Diagnosing hand- versus object-centric action understanding	watch
Whareformer	2026-07	ECCV 2026	First learned OSNOM tracker, trained on 56 videos and tested on 260 long EPIC-KITCHENS-100, IT3DEgo, and HD-EPIC videos; MIT code, weights, features, and training data are live	Long-term 3D object tracking through occlusion and out-of-view periods	open
HandsOnWorld	2026-07	arXiv	Camera-disentangled hand-controlled egocentric video generation with EgoVid-Pro: 103K clips and roughly 12M frames of protagonist-only hand trajectories	Unconstrained egocentric HOI video generation	watch
SAGE Action-Gaze	2026-07	arXiv	Unified action-gaze recognition and anticipation framework linking HOI, gaze, egocentric close-up views, and exocentric context	Joint gaze/action anticipation for human behavior understanding	watch
Ego3DLM / Ego-Human Motion Prediction	2026-07	ECCV 2026	3D-aware LLM that jointly predicts past/future human pose and narration from egocentric video, three-point tracking, and scene features on Nymeria	Scene-grounded egocentric human-motion forecasting	watch
Humanola Egocentric Hand-Pose Sample	2026-07	Hugging Face	LeRobot v2.1 sample with 9 episodes, 48,272 frames, four synchronized cameras, 200 Hz IMU, 3D hands, poses, overlays, and Rerun logs	Open hand-pose and sensor-fusion loader tests for manipulation pipelines	open
InterPet4D	2026-06	Hugging Face	6.8M-frame human-dog capture; public 10.7 GB v1 provides 227 headset clips with audio and aligned SMPL, MANO, pet skeleton, SMAL, and MERT features	Multimodal 4D human-pet interaction, motion generation, and pose analysis	open
ViDiHand	2026-06	arXiv	4D two-hand motion reconstruction from full-frame egocentric video using video-diffusion representations; evaluated on ARCTIC, HOT3D, and HOI4D	Egocentric hand-motion reconstruction without detector/test-time optimization	watch
HT-Bench / HandTouch	2026-06	arXiv	10M RGB frames and 7.8M tactile frames across 226 dexterous full-hand tasks	Vision-tactile representation learning for dexterous manipulation	watch
EPIC-Contact / HOPformer	2026-06	ECCV 2026	2.3K EPIC-KITCHENS clips and 62.3K frames with dense 3D hand-object contact, posed meshes, code, dataset, and checkpoints	In-the-wild egocentric 3D hand-object pose and contact estimation	open
GhostHandEgoNet	2026-06	IJPRAI 2026	Lightweight egocentric hand-pose and hygiene-monitoring pipeline with 2.56M parameters for embedded deployment	Egocentric hand pose under occlusion and clinical hand-hygiene monitoring	watch
Hand-4DGS	2026-06	arXiv	Feed-forward 4D hand reconstruction from egocentric video with 3D Gaussian Splatting, evaluated on H2O and ARCTIC	Fast egocentric 4D hand reconstruction	watch
A multimodal RGB/events FPV hand dataset	2026-06	arXiv	Synthetic event-based first-person hand detection from EgoHands plus v2e	Event/RGB hand detection benchmarking	watch
SIGNIQ Egocentric Hand-Pose Sample	2026-06	Hugging Face	Five short head-mounted GoPro manipulation clips with Label Studio import metadata for hand-pose and action-segment annotation	Hand-pose annotation and manipulation-labeling workflow tests	open
Leveraging Synthetic Data for Egocentric HOI Detection	2026-05	IJCV 2026	Synthetic-data augmentation study for hand-object interaction detection on VISOR, EgoHOS, and ENIGMA-51	HOI detection when real labels are scarce	watch
Causal-Inspired Fourier Egocentric Action Recognition	2026-05	IEEE TCSVT 2026	Causal-inspired Fourier representation learning for wearable IMUs and egocentric action recognition	Wearable-sensor action recognition	watch
EgoSMPLX	2026-05	ICIP 2026	Prior-guided whole-body human mesh recovery from monocular head-mounted images, with public code and train/test pseudo-ground-truth packages	Egocentric whole-body mesh recovery	partial
DexGloveHOI	2026-05	arXiv	100K+ synchronized egocentric vision and on-glove IMU samples with marker-based mocap 3D hand-pose ground truth across dexterous daily manipulation	Vision-IMU fusion for 3D hand tracking	watch
EggHand	2026-05	CVPR 2026 Findings	Multimodal foundation model for egocentric hand-pose forecasting over EgoExo4D-style video-language and action signals	Hand-pose forecasting and VLA action decoding	watch
Map-Mono-Ego	2026-05	ICIP 2026	Map-grounded global human-pose estimation from monocular egocentric video, with AIST-Living paired ego video and scanned environments	Scene-aware egocentric body-pose localization	watch
EgoForce Motion	2026-05	arXiv	Online full-body motion reconstruction from sparse egocentric observations using diffusion forcing	Real-time egocentric body-motion reconstruction	watch
EgoEMG	2026-05	arXiv	41 participants, bilateral EMG, IMU, RGB, external RGB-D, mocap hand labels	EMG plus egocentric vision hand pose estimation	watch
EgoTouch / TouchAnything	2026-05	arXiv	302 tasks, 4,530 episodes with egocentric + dual wrist cameras, bimanual 3D hand pose, and dense tactile pressure maps	Tactile estimation and contact modeling from egocentric video	watch
EgoEVHands	2026-05	arXiv	Stereo event-camera egocentric hand dataset with 5,419 sequences and 3D/2D keypoints	Event-based bimanual hand pose and gesture recognition	watch
TouchMoment	2026-04	CVPR 2026 Findings	4,021 egocentric videos, 8,456 annotated hand-object contact moments	Precise contact moment detection	watch
TAIHRI	2026-04	arXiv	Task-aware VLM for localizing task-relevant 3D human keypoints in close-range egocentric HRI	Metric-scale human keypoints for robot interaction	watch
MASS	2026-04	arXiv	Deformable surfel splatting for high-fidelity 3D hand reconstruction and rendering from egocentric monocular video	Egocentric hand reconstruction and rendering	watch
E-3DPSM	2026-04	arXiv	Event-driven continuous pose state machine for real-time egocentric 3D human-pose estimation from head-mounted devices	Event-based egocentric body-pose estimation	watch
EgoFun3D	2026-04	arXiv	271 egocentric videos with 3D geometry, part segmentation, articulation and function-template annotations	Interactive 3D object modeling from ego video	watch
FunRec	2026-04	CVPR 2026	Functional 3D digital-twin reconstruction from in-the-wild egocentric RGB-D interaction videos with articulated parts and kinematics	Interactive scene reconstruction for simulation and robot learning	watch
DP-DeGauss	2026-04	ICASSP 2026	Dynamic probabilistic Gaussian decomposition for egocentric 4D scene reconstruction, separating background, hands, and objects	First-person 4D interaction reconstruction	watch
Towards Egocentric 3D Hand Pose Estimation in Unseen Domains	2026-03	WACV 2026	Egocentric 3D hand-pose estimation study focused on unseen-domain generalization	Robust first-person hand tracking outside training domains	watch
Controllable Egocentric Video Generation	2026-03	ECCV 2026	Occlusion-aware sparse 3D hand-joint control for realistic egocentric hand-object video generation, with a 1M-clip annotation pipeline	Controllable egocentric HOI video generation and world models	watch
SHOW3D	2026-03	CVPR 2026	Open 20-hour, 2,137-scene release from 38 participants with eight exo and two Quest 3 ego views, 3D hand/object annotations, shape, and text	In-the-wild 3D hand-object reconstruction	open
EgoXtreme	2026-03	CVPR 2026	Smart-glasses egocentric 6D object-pose dataset across industrial, sports, and rescue scenes with extreme motion blur, dynamic lighting, and occlusion	Robust 6D object pose under extreme egocentric conditions	watch
AG-EgoPose	2026-03	arXiv	Attention-guided egocentric 3D human-pose estimation from fisheye camera input with dual motion/spatial streams	Fisheye egocentric 3D pose estimation	watch
FEEL	2026-03	arXiv	Open 26-trial, 700-episode benchmark with 39,497 Aria frames and synchronized gaze, hand, IMU, and Paxini grasp-force totals; the paper reports a larger full collection	Contact-rich physical action and hand-object understanding	open
Person Identification from Egocentric HOI	2026-02	CCF TPCI 2026	Uses 3D hand pose from egocentric human-object interactions as a person-identification signal	Identity and behavior analysis from first-person hand-object motion	watch
WHOLE	2026-02	arXiv	Joint world-grounded hand-object motion reconstruction from egocentric video using a generative HOI prior	Holistic hand-object pose recovery	watch
AirGlove	2026-02	ICASSP 2026	Egocentric 3D hand-tracking evaluation for gloved hands and sensing-glove appearance shifts	Gloved-hand tracking for teleoperation	watch
Eva-3M / EvaPose	2026-02	CVPR 2026	3.0M+ egocentric HPE frames, including 435K keypoint-visibility labels, plus visibility-aware pose estimation	Visibility-aware egocentric human pose	watch
WristPP	2026-02	CHI 2026 submission	Wrist-worn wide-FOV RGB system with a 133K-frame pose-pressure dataset from 20 subjects	Mobile hand pose and pressure interaction	watch
Egocentric Affordance Dataset	2026-02	Hugging Face	48,400 egocentric frames across household, workshop, factory, gardening, and healthcare scenes with Qwen3-VL affordance labels	Affordance and HOI supervision for embodied VLMs and robot learning	open
Industrial Egocentric HOI Detection	2026-01	ICIAP 2025 Workshops	Real-time egocentric hand-object interaction detection for industrial workflows	Industrial first-person HOI monitoring	watch
EgoGrasp	2026-01	arXiv	World-space open-vocabulary hand-object interaction reconstruction from dynamic egoview videos	HOI reconstruction in world coordinates	watch
GlovEgo-HOI	2026	VISAPP 2026	Synthetic-to-real egocentric HOI detection for gloved industrial hand-object scenarios	Industrial HOI detection under gloves and domain shift	watch
OpenTouch	2025-12	arXiv	5.1 hours of synchronized egocentric video-touch-pose data, 2,900 curated clips, text annotations, and retrieval/classification benchmarks	Full-hand tactile grounding for real-world interaction	watch
ROHIT	2025-12	arXiv	Reconstructs objects along hand-interaction timelines from HOT3D and EPIC-KITCHENS stable-grasp clips	Object pose propagation through hand contact	watch
EvHand-FPV	2025-09	arXiv	Event-based first-person 3D hand-tracking dataset and lightweight wrist-ROI tracking framework	Low-power FPV hand tracking	watch
THOR	2025-07	arXiv	Thermal-guided adaptive RGB sampling for wearable hand-object monitoring, using about 3% of RGB frames while preserving activity segments	Low-power longitudinal hand-object activity recognition	watch
Egocentric HOI Detection	2025-06	Expert Systems with Applications	New benchmark and method for detecting hand-object interactions in egocentric video	Egocentric hand-object interaction detection	open
EgoBrain	2025-06	ICLR 2026	Complete gated 1.6 TB release from 40 participants with synchronized 4K egocentric video, EEG, IMU, interval markers, and surveys	Brain-signal plus first-person action understanding	request
EventEgoHands	2025-05	ICIP 2025	Event-based egocentric 3D hand mesh reconstruction benchmark over N-HOT3D	Low-light and motion-blur hand reconstruction	watch
EgoEvGesture	2025-03	arXiv	First large-scale egocentric event-camera gesture-recognition dataset with a head-motion-robust 7M-param model (62.7% unseen-subject accuracy)	Event-camera egocentric gesture recognition	watch
EgoCast	2024-12	WACV 2025	3D pose forecasting from egocentric video and proprioceptive data on Ego-Exo4D / Aria Digital Twin	Future body-pose forecasting	watch
Ego3DT	2024-10	ACM MM 2024	Zero-shot 3D reconstruction and tracking of all objects in ego-centric videos	3D object tracking in first person	watch
EgoPressure	2024-09	CVPR 2025	5.0 hours, 21 participants, a moving egocentric camera plus 7 stationary RGB-D cameras, hand pose meshes, and fine-grained per-contact touch pressure (CVPR 2025)	Hand pressure and pose from egocentric vision	open
AFF-ttention / STAformer	2024-06	ECCV 2024	Affordance- and attention-aware short-term object-interaction anticipation from egocentric video	Short-term active-object anticipation	watch
HOT3D	2024-06	CVPR 2025	833+ minutes, 3.7M+ images, Aria and Quest 3, gaze, point clouds, 3D hand/object/camera poses	3D hand-object tracking for AR/VR manipulation	open
EMAG	2024-05	ECCV 2024 HANDS Workshop	Ego-motion-aware 2D hand forecasting for egocentric videos	Hand trajectory anticipation	watch
UnrealEgo2 / UnrealEgo-RW	2024-01	CVPR 2024	Expanded stereo egocentric pose datasets from synthetic and real-world capture	Stereo egocentric human pose estimation	watch
POV-Surgery	2023-07	MICCAI 2023	Egocentric surgical video for hand and tool pose estimation during operating-room activities	Surgical hand-tool pose and interaction understanding	open
EgoHumans	2023-05	ICCV 2023	125K+ egocentric images for in-the-wild multi-human tracking, 2D/3D pose, and mesh recovery	Egocentric multi-human perception and tracking	partial
AssemblyHands	2023-04	CVPR 2023	3.0M 3D hand-pose annotations from Assembly101, including 490K egocentric images	Egocentric 3D hand pose and pose-aware action recognition	benchmark
EgoTracks	2023-01	NeurIPS 2023	Long-term egocentric visual object tracking annotations from Ego4D	Tracking, re-detection, embodied object persistence	open
EgoGTA / EgoPW-Scene	2022-12	CVPR 2023	Synthetic EgoGTA plus EgoPW-based in-the-wild scene-aware pose data	Scene-aware egocentric 3D human pose estimation	partial
EgoHOS	2022-08	ECCV 2022	11,243 egocentric images with detailed hand/object/contact segmentation	Hand-object segmentation and contact-aware foreground modeling	open
UnrealEgo	2022-08	ECCV 2022	Synthetic stereo eyeglass egocentric dataset with 3D human pose	Robust egocentric 3D human motion capture	open
Ego2HandsPose	2022-06	WACV 2024	Extension of Ego2Hands with 3D global hand pose annotations	Two-hand RGB 3D pose in global coordinates	open
ARCTIC	2022-04	CVPR 2023	2.1M frames with bimanual hand/object meshes, articulated objects, and contact	Dexterous bimanual manipulation, contact, reconstruction	request
GIMO	2022-04	ECCV 2022	Ego-centric views, gaze, scene scans, and high-quality body pose sequences	Gaze-informed human motion prediction in scene context	open
HOI4D	2022-03	CVPR 2022	2.4M RGB-D egocentric frames, 4,000+ sequences, 800 objects, 16 categories, 610 rooms	4D human-object interaction, segmentation, object pose tracking, action segmentation	open
EgoBody	2021-12	ECCV 2022	HoloLens2 egocentric RGB/depth/eye/head/hand data with 3D body pose and shape	Egocentric human pose, shape, motion, social interaction	open
TREK-150	2021-08	project page	150 egocentric tracking sequences derived from EPIC-KITCHENS	Single-object tracking in egocentric video	open
H2O: Two Hands and Objects	2021-04	ICCV 2021	Synchronized multi-view RGB-D, two-hand 3D pose, 6D object pose, object meshes, camera poses	First-person two-hand interaction recognition and pose-aware HOI	open
Ego2Hands	2020-11	arXiv	Large-scale composited RGB egocentric two-hand segmentation/detection	Robust unconstrained hand segmentation	open
xR-EgoPose	2019-07	GitHub	Synthetic egocentric body pose data for XR headset viewpoints	3D human pose from head-mounted cameras	open
You2Me	2019-04	CVPR 2020	Infers second-person body pose in egocentric video from first- and second-person interaction cues	Social interaction and body-pose inference	watch
H+O	2019-04	CVPR 2019 Oral	Unified egocentric RGB model for 3D hand pose, object pose, interactions, objects, and actions	3D hand-object pose and interaction recognition	watch
EgoReID Dataset	2018-12	arXiv	900 identities, about 10.2K tracks, 176K detections from mobile first-person cameras with sensor metadata	Person re-identification from first-person video	watch
EgoYouTubeHands / HandOverFace	2018-03	CVPR 2018	In-the-wild egocentric hand-segmentation study built around EgoYouTubeHands, HandOverFace, EGTEA, and EgoHands-style supervision	Hand detection and segmentation beyond lab capture	watch
FPHA / First-Person Hand Action	2017-04	CVPR 2018	100K+ RGB-D frames, 45 action classes, 26 objects, 3D hand/object poses	3D hand pose and first-person hand action recognition	open
EgoGesture	2017-01	IEEE TMM 2018	24K+ gesture samples, 3M RGB-D frames, 50 subjects, 83 static and dynamic gestures across six scenes	First-person gesture recognition for wearable interaction	open
Left/Right Hand Segmentation	2016-07	CVIU 2016	Classic first-person method that separates left/right hands and handles hand-to-hand occlusion with temporal superpixels	Hand segmentation and wearable interaction	watch
EgoHands	2015-12	ICCV 2015	48 Google Glass videos, 4,800 annotated hand images	Hand detection and segmentation	open
BEOID	2014-09	BMVC 2014	Gaze-tracked egocentric video of 8 users interacting with objects across 6 everyday locations (kitchen, workspace, printer, corridor, gym)	Discovering task-relevant objects and interaction modes	open

Daily Life, Memory, Assistance, and QA

Long-horizon daily-life capture and the question-answering benchmarks that probe memory, reasoning, and assistance.

Resource	Released	Venue	Scale / signal	Best for	Status
EgoMonth	2026-08	arXiv	300+ hours from 20 participants over 20-120 days and 1,443 human-crafted questions across 14 tasks; annotations, scripts, and eight sample videos are public while full raw video uses controlled access	Month-scale episodic memory, spatial reasoning, and personal QA	partial
SERUM	2026-07	COLM 2026	Extracts compact action and intent state models from 61 egocentric videos spanning coding, cooking, physical activity, and daily life	Interpretable user workflows, intent modeling, and personalized assistants	watch
Sidewalk Moments	2026-07	arXiv	61 first-person city walks segmented into more than 50K ten-second clips with video, averaged-image, audio, and text representations	Human-aligned urban engagement and multimodal temporal-compression studies	watch
Vinci2 / EgoServe	2026-07	ECCV 2026	3,000+ proactive-service instances across 11 categories and four memory horizons, with public annotations and the training-free EgoMemo agent	Deciding when an egocentric assistant should intervene and grounding its response in memory	open
VEGAS	2026-07	arXiv	Training-free gaze-aware caption metric plus egocentric activities and instructional slides paired with synchronized gaze and reference captions	Human-attention-aligned caption evaluation and retrieval	watch
CoMind	2026-07	ECCV 2026	Dual-view collaborative-cooking dataset with two synchronized head-mounted cameras, two exo views, audio, gaze, 3D scene/object scans, and social/interaction annotations; download is marked coming soon	Social reasoning, joint attention, handover prediction, and collaborative assistance	watch
Imprint	2026-07	arXiv	Online interaction-centric memory compression for seven-day egocentric QA on EgoLifeQA	Scalable long-horizon memory retrieval	watch
EgoGapBench	2026-07	arXiv	Diagnostic benchmark for whether models select actions from the camera-wearer's perspective in multi-agent scenes	Egocentric perspective and action-selection evaluation	open
EgoSafetyBench	2026-06	arXiv	1,200 egocentric robot-view safety scenarios with half-second labels and misleading in-scene text controls	Streaming VLM safety-guard evaluation	watch
BinaryTracking / GangnamLoop	2026-06	arXiv	Spatial QA and navigation benchmark for long egocentric robot routes, with GangnamLoop route data and the BinTrack implementation	Open spatial memory and navigation QA	open
Egocentric Pedestrian Crossing Intention	2026-06	arXiv	VLM-based VQA formulation for predicting pedestrian crossing intention from short egocentric traffic-safety clips	Wearable traffic-safety intent reasoning	watch
Face-vs-Body HRI Tracking	2026-06	arXiv	Custom-annotated egocentric HRI dataset from a social robot, comparing face and body tracking under occlusion and engagement changes	Social-robot engagement tracking	watch
VL-MemKnG / WalkieKnowledgeT+	2026-06	arXiv	Hybrid spatio-temporal knowledge graph plus segment memory for QA over long egocentric navigation trajectories	Navigation memory, long-horizon evidence retrieval, spatial QA	watch
V-RAGBench / CARVE	2026-06	arXiv	Query, evidence-chunk, answer triplets for decoupled retrieval/generation evaluation in long egocentric video RAG	Retrieval-augmented long-video QA	watch
OVO-S-Bench	2026-06	arXiv	1,680 spatial-reasoning questions over 348 continuous egocentric videos with query timestamps and evidence intervals	Streaming spatial intelligence, tracking, simulation, and allocentric mapping	watch
Astra	2026-06	arXiv	Agentic VLM spatial reasoning with a view-consistent world simulator that supplies imagined novel-view evidence from limited egocentric observations	World-model-augmented spatial reasoning	watch
VLESA	2026-06	arXiv	Goal-conditioned safety annotations over egocentric human-activity video for safety-aware assistance	Safety monitoring and situated activity assistance	open
Causal-Plan-1M	2026-06	arXiv	Reported million-scale corpus of explicit causal reasoning traces over egocentric videos	Causal and planning-oriented ego reasoning	watch
NoRA	2026-06	arXiv	1,420 first-person video clips for normative action reasoning with fact-reason-action support graphs	Grounded reasonableness and safety-oriented action generation	watch
Pause and Think	2026-06	arXiv	Reasoning-centric training data and benchmark for video-grounded assistive action suggestions	Scene-grounded assistance, planning, and temporal consistency	watch
SuperMemory-VQA	2026-06	arXiv	52.9 hours of AI-glasses activity with RGB, audio, gaze, IMU, SLAM, and 4,853 human-verified QA pairs across object/location/intent/scene memory	Long-horizon memory for AR assistants	watch
VISTA Daily Assistance	2026-05	arXiv	Generative egocentric-video framework for reactive and proactive daily-assistance scenarios	Synthetic training/evaluation for assistants	watch
EgoMemReason	2026-05	COLM 2026	500 public questions over week-long EgoLife video with entity, event, and behavior memory, open evaluation code, and a leaderboard	Memory-driven reasoning across sparse evidence over hours or days	open
EgoBench	2026-05	arXiv	1,045 egocentric-video-grounded interactive tasks with tools and simulated users	Tool-using multimodal agents with dynamic interaction	watch
EyeCue	2026-05	arXiv	Gaze-empowered egocentric video model plus CogDrive annotations for driver cognitive-distraction detection	Gaze-context reasoning for safety and internal-state inference	watch
Minerva-Ego	2026-05	arXiv	Complex egocentric visual-reasoning benchmark with multi-step multimodal questions, dense reasoning traces, and spatiotemporal object masks	Grounded multi-step egocentric reasoning	open
EgoPro-Bench	2026-05	arXiv	Personalized proactive-interaction benchmark with 2,400 evaluation videos, 12K+ training videos, and 12 domains	Streaming intent prediction and proactive assistance timing	watch
Pro2Assist	2026-05	arXiv	Continuous step-aware proactive assistance with multimodal egocentric perception and an AR-glasses testbed	Real-time step-aware assistance	watch
Personal Visual Context Learning / Personal-VCL-Bench	2026-05	arXiv	Smart-glasses benchmark for using wearer-specific visual context at prompt time over continuous first-person streams	Personalized visual memory for wearable assistants	watch
AwareLLM	2026-05	arXiv	Proactive assistant using egocentric vision, pupillometry, gaze, posture, heart activity, and LLM reasoning	Personalized productivity and cognitive-state-aware assistance	watch
GazeMind / CogLoad-Bench	2026-05	arXiv	Gaze-guided LLM agent and cognitive-load benchmark for smart glasses, reporting 152 participants, 40+ hours, and 10K+ annotations	Cognitive-load-aware assistance	watch
Dietary Behavior Change Receptivity	2026-05	arXiv	Pilot AIM-2 wearable-camera study using egocentric eating images to infer receptivity for just-in-time dietary interventions	Nutrition and health-oriented lifelogging	watch
EgoBabyVLM	2026-05	arXiv	Benchmarking cross-modal learning from naturalistic infant and adult egocentric video, including Machine-DevBench and challenge tracks	Developmental egocentric VLM learning	watch
EgoIntrospect	2026-05	arXiv	180 hours from 60 subjects with synchronized video, audio, gaze, motion, physiological signals	Internal-state reasoning, affect, intent, cognitive memory for wearable assistants	watch
EgoExoMem	2026-05	arXiv	2.6K MCQs across synchronized ego-exo videos	Cross-view memory reasoning	watch
EgoCoT-Bench	2026-05	arXiv	3,172 verifiable QA pairs over 351 videos with operation-centric rationale annotations	Grounded chain-of-thought and evidence consistency	open
EgoStream	2026-05	arXiv	2,250 questions across seven memory dimensions with Answer Validity Windows, expanded to 8,528 recall-conditioned evaluations over streams up to 45.3 hours	Diagnosing streaming episodic memory	watch
ChildLens	2026-04	Behavior Research Methods 2026	109 hours of vest-mounted first-person video/audio from 62 children aged 3-5 at home, with five location and 14 activity classes	Naturalistic child activity, temporal localization, and voice analysis	request
MyEgo	2026-04	CVPR 2026	541 long videos and 5K personalized questions about the camera wearer, belongings, activities, and past	Personalized ego-grounding and long-range memory QA	open
EgoEsportsQA	2026-04	arXiv	1,745 QA pairs from professional first-person shooter matches across three games	High-speed first-person perception and tactical reasoning in virtual environments	watch
EgoEverything	2026-04	arXiv	5,000+ MCQ pairs over 100+ hours, with gaze-attention-inspired question generation for AR	Human-behavior-inspired long-context egocentric understanding	watch
LifeDialBench / EgoMem	2026-04	arXiv	Real egocentric video plus synthetic lifelog conversations	Online temporal-causality evaluation for continuous lifelog memory systems	watch
EgoScreen-Emotion / ESE	2026-04	arXiv	224 egocentric screen-view movie trailers with 28,667 confidence-aware emotion keyframe annotations	Emotion understanding from realistic first-person viewing	watch
Gaze-to-Guidance Assistants	2026-04	arXiv	Gaze-grounded multimodal LLM assistance study using egocentric video and gaze overlays to infer reading difficulty	Cognitive-need-aware wearable assistance	watch
EgoTL	2026-04	arXiv	Think-aloud (say-before-act) egocentric capture with word-level spoken reasoning and metric-scale spatial annotations over 100+ household tasks	Long-horizon reasoning for VLMs and world models	watch
EgoSelf	2026-04	arXiv	Personalized egocentric assistant framework with graph memory	Personalization from long-term egocentric interaction memory	watch
MA-EgoQA	2026-03	arXiv	1.7K questions over multiple long-horizon egocentric streams	Multi-agent egocentric memory reasoning	open
Ego2Web	2026-03	CVPR 2026	Egocentric videos paired with web tasks requiring physical-scene understanding and online execution	Web agents grounded in first-person physical context	watch
Egocentric Co-Pilot	2026-03	WWW 2026	Web-native smart-glasses agent framework with temporal reasoning, context compression, speech/gaze intent, and streaming WebRTC/WebSocket pipelines	Assistive always-on egocentric web agents	watch
LifeEval	2026-03	arXiv	4,075 QA pairs over continuous first-person streams for real-time task-oriented human-AI collaboration in daily life	Daily-life multimodal assistance evaluation	watch
Gesture-Based Egocentric Video QA	2026-03	CVPR 2026	Egocentric video QA grounded in the camera wearer's pointing and deictic gestures	Gesture-grounded referential QA	watch
EgoIntent	2026-03	arXiv	3,014 steps over 15 daily-life scenarios for local intent (What), global intent (Why), and next-step plan (Next)	Step-level intent and anticipatory assistance	watch
SAW-Bench	2026-02	ICML 2026	Open 3.30 GB benchmark with 786 Ray-Ban Meta smart-glasses videos and 2,071 QA pairs across six situated-awareness tasks	First-person situated spatial reasoning	open
EgoGraph	2026-02	arXiv	Dynamic temporal knowledge graph framework evaluated on EgoLifeQA and EgoR1-bench	Training-free ultra-long egocentric video memory	watch
EgoSound	2026-02	CVPR 2026	7,315 validated QA pairs across 900 Ego4D/EgoBlind videos	Sound perception, spatial localization, causal audio-visual reasoning	watch
SuperGlasses	2026-02	CVPR 2026 Findings	Evaluates VLMs as intelligent agents for AI smart glasses across realistic wearable assistant tasks	Smart-glasses agent evaluation	watch
Gazeify Then Voiceify	2026-01	IUI 2026	Displayless smart-glasses object referencing via gaze selection, mask generation, voice descriptions, and conversational correction	Gaze-plus-voice physical object grounding	watch
Wearable Product Localization	2026-01	arXiv	Wearable assistive shopping system combining detection, VLM guidance, spatialized sonification, and corrective feedback	Product search and navigation for blind or low-vision users	watch
Egocentric Clinical Intent	2026-01	arXiv	Benchmark for egocentric clinical intent understanding by medical multimodal LLMs over first-person clinical procedure video	Clinical intent reasoning for medical assistants	watch
WearVox	2026-01	arXiv	Egocentric multichannel voice-assistant benchmark for spoken interaction grounded in first-person audio-visual context	Wearable voice-assistant evaluation	watch
Screen Detection from Egocentric Streams	2026	IEEE TMM 2026	Multi-view vision-language approach for detecting screens in egocentric image streams	Wearable screen understanding and assistive scene-text interfaces	watch
Ego-EXTRA	2025-12	WACV 2026	50 hours of expert-trainee procedural assistance dialogue, 15K+ VQA pairs	Egocentric video-language assistants and expert feedback	open
WearVQA	2025-11	NeurIPS 2025	2,520 image-question-answer triplets across 7 domains and 10 task types under occluded/low-light/blurry wearable capture (NeurIPS 2025)	Smart-glasses VQA under real wearable conditions	watch
WAGIBench	2025-10	NeurIPS 2025 Spotlight	29 hours, 348 participants, and 3,477 multimodal recordings for assistive wearable-agent goal inference	Inferring user goals from wearable context	watch
TeleEgo	2025-10	arXiv	Streaming omni-modal benchmark with 3,291 QA items across memory, understanding, and cross-memory reasoning	Real-time egocentric AI assistant evaluation	watch
CIVIL Lifelog Retrieval	2025-10	arXiv	Captioning-integrated visual lifelog retrieval that describes wearable-camera images as first-person experiences before text-query matching	Personal memory search and lifelog retrieval	watch
EgoNight	2025-10	ICLR 2026	EgoNight-VQA (3,658 QA / 90 videos / 12 types) plus day-night retrieval and depth, with day-night aligned video	Nighttime / low-light egocentric robustness	open
EgoMemory	2025-09	OpenReview	165,795 user-specific object annotations over 245 videos from 45 participants	Memory-augmented personalized retrieval	watch
HowToDIV	2025-08	arXiv	507 conversations, 6,636 QA pairs, 24 hours of egocentric instructional task-assistance video clips	Instructional dialogue and procedural video reasoning	watch
EgoTrigger / HME-QA	2025-08	ISMAR 2025 / TVCG	Audio-driven smart-glasses capture strategy plus 340 first-person HME-QA pairs from full-length Ego4D videos	Energy-efficient memory assistance and audio-triggered capture	watch
EgoCross	2025-08	AAAI 2026	Cross-domain benchmark over surgery, industry, extreme sports, and animal perspectives; the EgoVis 2026 challenge drew 1,500+ submissions and 130+ participants	Cross-domain egocentric QA generalization	watch
ProMemAssist	2025-07	UIST 2025	Smart-glasses proactive assistance via real-time working-memory modeling from multimodal wearable signals	Timely proactive assistance	watch
EgoPrivacy	2025-06	ICML 2025	Benchmark for privacy leakage from first-person cameras across demographic, individual, and situational inference	Privacy risk evaluation for ego video	watch
EASG-Bench	2025-06	ICCV 2025 Workshop	Egocentric action-scene-graph-based video QA benchmark	Scene graph, temporal order, and relation-aware QA	open
Ego-R1	2025-06	arXiv	Ego-CoTT-25K, Ego-QA-4.4K, and Ego-R1 Bench for tool-augmented ultra-long egocentric QA	Chain-of-tool-thought reasoning over days/week-long videos	watch
ExAct	2025-06	arXiv	Video-language benchmark for expert action analysis and feedback over skilled activity	Expert feedback and skill assessment	open
EgoTextVQA	2025-06	CVPR 2025	Egocentric scene-text-aware video QA across housekeeping and driving scenes (CVPR 2025)	Reading and reasoning over first-person scene text	open
Week-Long First-Person Video Privacy Probe	2025-04	arXiv	54-hour wearable-camera study of what foundation models infer from week-long first-person video	Personal privacy and lifelogging risk	watch
EgoLog	2025-04	arXiv	Audio-IMU daily-log system for contextual and spatial activity logging with ubiquitous wearables	Fine-grained lifelogging	watch
EgoLife	2025-03	CVPR 2025	300 hours from six participants over one week with Meta Aria, third-person references, and EgoLifeQA	Long-term daily-life memory, personalized assistants, ultra-long QA	partial
EgoToM	2025-03	arXiv	Theory-of-mind QA over Ego4D-style egocentric videos	Goals, beliefs, and next-action reasoning for camera wearers	watch
X-LeBench	2025-01	EMNLP 2025	432 simulated life logs over Ego4D-like footage, spanning 23 minutes to 16.4 hours	Extremely long egocentric video understanding	watch
VidEgoThink	2024-10	ICLR 2025	Ego4D-based benchmark for video QA, hierarchy planning, visual grounding, and reward modeling	Embodied egocentric video understanding	benchmark
Egocentric 360 VideoQA for Visual Impairment	2024-05	arXiv	360-degree egocentric wearable-camera VideoQA dataset for multiple real-world obstacles faced by people with visual impairments	Assistive VideoQA and full-surround wearable perception	watch
EgoThink	2023-11	CVPR 2024	First-person VQA benchmark covering six capability groups and twelve dimensions	First-person perspective reasoning for VLMs	benchmark
EgoSchema	2023-08	NeurIPS 2023	5K+ curated multiple-choice QA pairs over 250+ hours from Ego4D clips	Very long-form video-language understanding	open
EgoTaskQA	2022-10	NeurIPS 2022	Diagnostic QA benchmark for task dependencies, effects, intents, beliefs, and counterfactuals in ego video	Task-step reasoning and procedural QA	benchmark
AssistQ	2022-03	ECCV 2022	531 question-answer samples from 100 newly filmed instructional videos	Assistance-oriented video QA and affordance-centric task completion	open
MMAC Captions	2021-09	ACM MM 2021	Sensor-augmented egocentric-video captioning data around CMU-MMAC-style multimodal activity streams	Video captioning with RGB, audio, IMU, and text	watch
EgoVQA	2019-10	ICCV 2019	600+ QA pairs over egocentric videos; an early first-person VideoQA benchmark	Classic egocentric VideoQA	open
HARMONIC	2018-07	IJRR 2022	Assistive eating HRI data with head-mounted ego video, gaze, EMG, joystick, robot state, and third-person stereo views	Intention prediction and shared-autonomy assistance	watch
First-Person Stories	2017-07	ICIAP 2017 Workshop	45K+ egocentric photo-stream images labeled for lifestyle patterns such as eating, socializing, and sedentary behavior	Lifelogging and daily-life behavior analysis	watch

Action, Procedure, Lifelogging, and Classic FPV

Procedural and activity datasets, from modern industrial assembly to the classic first-person-vision benchmarks that started the field.

Resource	Released	Venue	Scale / signal	Best for	Status
EventKitchen	2026-08	ECCV 2026	5.5 hours from 10 participants in 13 kitchens with stereo events, synchronized RGB/depth/IMU, 10,762 action segments, and 13,482 boxes	Neuromorphic action recognition, object detection, and stereo depth	watch
Wearable Gait MoCap with Shank-Mounted Ego Cameras	2026-07	Scientific Data 2026	Wearable motion-capture dataset for gait analysis using IMUs and shank-mounted egocentric cameras	Wearable gait analysis and motion-sensor fusion	watch
Poseidon Egocentric Activity Video Samples	2026-06	Hugging Face	Two auto-gated 1080p, 30 fps everyday activity clips with task, scene, technical, feature, and PII metadata	Lightweight activity-footage and capture-quality review	request
OR-Action	2026-06	arXiv	Fine-grained multi-role operating-room action benchmark over public ego-exocentric OR video and scene-graph state changes	Surgical workflow and temporal action understanding	watch
SkillSpotter	2026-06	ECCV 2026	Pose-aware multi-view skilled-action detection and grading over Ego-Exo4D proficiency demonstrations, transferring to HoloAssist	Ego-exo skill detection, grading, and coaching	open
TimeScribe Egocentric Annotations	2026-06	Hugging Face	Manual dense-caption annotations for eight egocentric videos with frame-aligned micro-events, Chinese captions, English mirrors where available, source clips, and JSON manifests	Human captioning gold data for hand-action and procedure understanding	request
Egocentric Nursing Competency Assessment	2026-05	CVPR 2026 Workshop	3.8 hours of egocentric nursing-simulation video with 493 annotated actions tied to instructor-rated competency	Medical skill assessment	watch
IMPACT-HOI	2026-05	arXiv	Mixed-initiative tool for constructing onset-anchored HOI event graphs in egocentric procedural video	Structured HOI annotation for robot learning	watch
EgoMAGIC	2026-04	arXiv	3,355 egocentric field-medicine videos over 50 tasks from a head-mounted stereo camera with audio; 1.95M labels, 124 objects, action-detection challenge (Zenodo)	Field-medicine perception, action and object detection	open
PIE-V	2026-04	arXiv	Mistake-aware procedural egocentric-video benchmark injecting plausible mistakes and recovery corrections across Ego-Exo4D scenarios	Procedural mistake detection and recovery reasoning	watch
IMPACT	2026-04	arXiv	Ego-exo RGB-D industrial assembly dataset with bimanual, state, and anomaly annotations	Industrial assembly, procedural state tracking, anomaly recovery	watch
ENIGMA-360	2026-03	arXiv	Ego-exo dataset for human behavior understanding in industrial scenarios with 360-degree and egocentric capture	Industrial ego-exo behavior understanding	watch
SAVA-X	2026-03	CVPR 2026	Ego-to-exo imitation-error detection over asynchronous, length-mismatched egocentric and exocentric videos, evaluated with EgoMe	Cross-view imitation-error detection	watch
Motion Focus Recognition	2026-01	arXiv	Real-time motion-focus recognition for fast-moving egocentric video using camera-pose foundation-model features and sliding-batch inference	Sports/fast-motion FPV intent and edge analysis	watch
SmartSeg	2026	Pervasive and Mobile Computing 2026	Non-parametric temporal segmentation method for wearable-camera video streams	Lifelog event segmentation and procedure understanding	watch
PEDESTRIAN	2025-12	arXiv	340 first-person pavement videos covering 29 urban sidewalk obstacle types, with deep-learning detection baselines	Pedestrian-safety obstacle detection from first-person video	watch
Mistake Attribution / MATT	2025-11	CVPR 2026	Fine-grained mistake attribution over EPIC-KITCHENS-M and Ego4D-M with semantic, temporal, and spatial labels	Procedural mistake understanding	watch
SkillSight	2025-11	arXiv	First-person skill assessment using video+gaze teacher models distilled to a low-power gaze-only student	Gaze-based skill learning	watch
IndEgo	2025-11	NeurIPS 2025	~197h egocentric (plus ~97h exocentric) industrial collaborative work over assembly, logistics, inspection, and repair; gaze, narration, sound, motion, hand pose, point clouds	Industrial egocentric assistants and procedure understanding	open
EgoEMS	2025-11	AAAI 2026	High-fidelity multimodal egocentric data for cognitive assistance in emergency medical services, capturing time-critical team actions	Real-time medical procedural assistance	watch
EgoExOR	2025-05	arXiv	Ego-exo operating-room dataset with wearable RGB/gaze/hand/audio, RGB-D cameras, ultrasound, and scene graphs	Surgical activity understanding	watch
Physical Activity VLM Annotation	2025-05	arXiv	VLM and discriminative-model study for reducing annotation burden in free-living wearable-camera physical-activity datasets	Health-oriented activity labeling from lifelog images	watch
LSC-ADL	2025-04	ACM MM 2025	ADL annotations over lifelogging data generated with clustering plus human review	Activity-aware lifelog retrieval	open
EgoSurgery	2025-03	Healthcare Technology Letters	EgoSurgery-Phase (surgical phase recognition) and EgoSurgery-HTS (pixel-wise hand-tool segmentation of 14 tools) from egocentric open-surgery video (MICCAI 2024)	Surgical phase, hand, and tool understanding	open
EgoMe	2025-01	arXiv	Real-world following-me dataset pairing exocentric demonstrations with egocentric imitation across everyday tasks	Egocentric imitation and cross-view following	watch
Audio-Narrated FPV Domain Generalization	2024-09	arXiv	Multimodal first-person action-recognition framework using audio narrations and audio-visual consistency to improve ARGO1M domain generalization	Robust first-person action recognition across environments	watch
PARSE-Ego4D	2024-06	arXiv	Ego4D personal action recommendation annotations with 18K+ candidate suggestions and human preference evaluation	Action recommendations for assistants	watch
Object-Aware Egocentric Online Action Detection	2024-06	CVPR 2024 EgoVis Workshop	Object-aware streaming action detection module for first-person video, validated on EPIC-KITCHENS-100	Online action detection	watch
EgoExo-Fitness	2024-06	ECCV 2024	Synchronized ego and exo fitness videos with keypoint verification, execution comments, and quality scores (ECCV 2024)	Ego-exo full-body action quality and skill assessment	open
AIM-2 Food Ingestion Environment	2024-05	arXiv	Two-stage transfer-learning method for recognizing food-ingestion environment from AIM-2 egocentric wearable-camera images	Nutrition and free-living dietary context recognition	watch
EgoPack / Backpack Full of Skills	2024-03	CVPR 2024	Multi-task egocentric video understanding with portable task perspectives across downstream skills	Multi-skill egocentric understanding	watch
EgoExoLearn	2024-03	CVPR 2024	120 hours of egocentric plus demonstration videos with gaze and annotations	Bridging asynchronous ego and exo procedural activity	open
Visual Experience Dataset / VEDB	2024-02	Journal of Vision	240+ hours egocentric video with gaze/head tracking in classic literature	Lifelogging, attention modeling, visual experience statistics	partial
CaptainCook4D	2023-12	NeurIPS 2024 Datasets and Benchmarks	Procedural-activity dataset for understanding execution errors, later reused in multimodal temporal-action-segmentation work	Procedural errors and mistake-aware activity understanding	watch
IndustReal	2023-10	WACV 2024	Industrial-like egocentric procedure-step recognition dataset with execution errors	Industrial procedure recognition and error handling	watch
EGOFALLS	2023-09	ICPR 2024	Visual-audio egocentric dataset and benchmark for fall detection	Wearable fall detection and safety monitoring	watch
WEAR	2023-04	IMWUT 2024	22 participants, 18 outdoor workouts, synchronized egocentric video and 3D acceleration across 11 locations (IMWUT 2024)	Vision-plus-inertial outdoor activity recognition	open
FT-HID	2022-09	GitHub	90K+ RGB-D first- and third-person human interaction samples from 109 subjects	FPV/TPV aligned human interaction analysis	open
EgoProceL	2022-07	ECCV 2022	62 hours, 130 subjects, 16 procedural tasks	Key-step localization and procedure learning from ego videos	open
KrishnaCam / OAK	2021-08	ICCV 2021	Long-running Google Glass daily-life stream; OAK adds 17.5 hours of object annotations	Continual object detection, summarization, personal visual memory	partial
Home Action Genome / HOMAGE	2021-05	CVPR 2021	27 participants, synchronized ego and third-person views, 12 sensor types, hierarchical activity/action labels	Compositional multi-view home activity understanding	open
MECCANO	2020-10	WACV 2021	Industrial-like motorbike model assembly with RGB/depth/gaze variants	EHOI, active object detection, anticipation, industrial procedures	open
EgoK360	2020-10	ICIP 2020	First-person 360-degree activity videos	360-degree egocentric activity recognition	partial
LEMMA	2020-07	ECCV 2020	Multi-view multi-agent multi-task daily activities across 14 kitchens/living rooms with dense atomic-action and HOI labels (ECCV 2020)	Multi-agent compositional activity understanding	open
EGO-CH	2020-02	Pattern Recognition Letters 2020	27+ hours, 70 subjects, two cultural sites, 26 environments, 200+ points of interest	Cultural heritage visitor behavior, localization, retrieval, preference prediction	partial
Charades-Ego	2018-04	CVPR 2018	68.8 hours paired first-person and third-person videos, 68K+ activity instances	Ego-exo domain transfer, action recognition, localization, captioning	open
GTEA	2018-01	project page	Early first-person cooking/activity dataset	Classic ego action recognition and object interaction	open
GTEA Gaze / EGTEA Gaze+	2018-01	project page	Meal-prep egocentric videos with gaze and action annotations	Gaze-aware action recognition and hand-object attention	open
Ego2Top	2016-07	ECCV 2016	50 top-view videos, 188 egocentric videos, 166K frames, and 100K body detections	Matching camera wearers across egocentric and overhead views	watch
Ego-Engagement	2016-04	ECCV 2016	Egocentric video dataset and model for whether the wearer is engaged with people or objects	Engagement, attention, and object/person interaction cues	watch
Wrist-mounted ADL	2015-11	CVPR 2016	Synchronized head and wrist wearable-camera daily activities	Comparing head- versus wrist-mounted first-person views	open
Ego-Object Discovery / EDUB	2015-04	arXiv	4,912 egocentric daily-life photo-stream images from four users for unsupervised object discovery	Lifelog object discovery and detection	open
HUJI EgoSeg	2014-06	project page	Long egocentric videos for temporal segmentation	Egocentric event segmentation	partial
DogCentric	2014-01	project page	Dog-mounted first-person activity videos	Animal egocentric activity recognition	open
JPL First-Person Interaction	2013-01	IEEE	First-person videos of people interacting with a humanoid observer	Human interaction recognition from first person	partial
First-Person Social Interactions	2012-06	CVPR 2012	Day-long head-mounted video of 8 subjects at a theme park, annotated for social interactions, roles, attention, and turn-taking	First egocentric social-interaction dataset	open
ADL Dataset	2012-06	project page	Unscripted daily activity recordings with activity/object/hand annotations in classic literature	Daily living action recognition and object interaction	partial
UT Ego	2012-06	project page	Long daily egocentric videos in classic summarization work	Temporal segmentation and summarization	partial
CMU-MMAC	2009-06	CMU tech report 2009	Multimodal kitchen-activity database: head-mounted egocentric video plus body IMUs, motion capture, and audio for 43 subjects cooking 5 recipes	One of the first egocentric activity datasets	open

Project Aria, AR/VR, and 3D Scene Resources

AR-glasses and headset data with gaze, SLAM, and digital-twin annotations for scene-level perception.

Resource	Released	Venue	Scale / signal	Best for	Status
SoniSpeech	2026-08	arXiv	Open 34-hour eyewear dataset with 18,000 voiced/silent utterances, synchronized ultrasound, audio, and frontal video, covering 5,356 words	Open-vocabulary wearable silent-speech interfaces	open
Assistant Placement Aria	2026-08	ICRA 2026	Synthetic and real scenes with 2D images, 3D point clouds, text, and three virtual-placement tasks under global, local, and human constraints	Egocentric spatial placement assistance and AR scene reasoning	watch
CRAFT Wearable Creative AI	2026-07	arXiv	Smart-glasses creative-AI probe informed by nine interviews, co-design with 16 participants, and 24 real-world sessions with eight writers	Context-aware wearable creativity and in-situ human-AI collaboration	watch
SHARE User-Centric AR SLAM	2026-07	arXiv	User-prioritized edge SLAM for commercial AR headsets and a ground robot, reporting 13.22 ms AR latency and sub-2-centimeter tracking	Responsive AR interfaces in shared human-robot workspaces	watch
Ego6D Nymeria Features	2026-07	Hugging Face	Public Nymeria-derived 3D scene voxels, 20 Hz head 6-DoF windows over 149 scenes, and world-frame SMPL body pose over 148 scenes; terms are research-only and incomplete	Joint 3D scene, localization, and body-pose experiments	partial
HeadRoom	2026-07	arXiv	Edge smart-glasses pipeline estimates visual and auditory channel availability from egocentric video/audio; a 25-participant study tests adaptive notification routing	Perceptual-load-aware wearable assistance	watch
Future-Privileged Causal Ego Gaze	2026-07	arXiv	Causal egocentric gaze-estimation study showing future-aware training improves online models on EGTEA Gaze+ and Ego4D	Real-time gaze modeling for AR/wearables	watch
EgoMed-IEMIS	2026-06	arXiv	523 smart-glasses videos and 173,657 frames with referred medical-target masks across five imaging modalities and five viewing scenes	Interactive egocentric medical segmentation and assistance	partial
AmbientEye	2026-06	arXiv	Passive-IR smart-glasses pupil-segmentation dataset for natural ambient illumination without active IR lighting	Energy-efficient eye tracking for all-day AR	watch
Single-View Mesh Rotation Stress Test	2026-06	arXiv	Controlled camera-rotation stress test for single-view mesh reconstruction on Aria Digital Twin and Franka wrist-camera sequences	Robust 3D reconstruction under ego/wrist camera motion	watch
EPIC Efficient Egocentric Perception	2026-06	arXiv	Smart-AR-glasses perception framework using gaze, pose, and inertial signals to reduce high-resolution egocentric memory and energy cost	Efficient always-on AR perception	watch
EgoTraj	2026-05	arXiv	75 Meta Quest Pro navigation sequences with RGB, head pose, gaze, scene labels	Egocentric human trajectory prediction and assistive navigation	open
RoSHI	2026-04	arXiv	Wearable suit fusing sparse IMUs with Project Aria to estimate global 3D pose and body shape for in-the-wild human data	Robot-oriented human data capture	watch
GIST	2026-04	arXiv	Semantic-topology pipeline converting mobile point clouds into lightweight maps for search, localization, zone classification, and egocentric routing instructions	Assistive navigation in cluttered real spaces	watch
VueBuds	2026-03	arXiv	Camera-integrated wireless earbuds streaming low-power binocular views to on-device VLMs	Earbud-scale wearable visual intelligence	watch
Pandora Articulated 3D Scene Graphs	2026-03	BMVC 2025	Recovers articulated 3D scene graphs from Project Aria egocentric exploration for robot mapping	Articulation-aware scene graphs	watch
EgoPoseVR	2026-02	IEEE VR 2026 / IEEE TVCG	Open 388 GB synthetic RGB-D/HMD corpus with 18,235 motion clips, seven VR scenes, SMPL body parameters, and pose labels	Sensor-free full-body VR pose estimation	open
ARGaze	2026-02	arXiv	Autoregressive transformer for causal online gaze estimation from first-person video and bounded recent gaze context	Streaming gaze estimation for AR/wearables	watch
EgoCampus	2025-12	arXiv	Egocentric pedestrian eye-gaze dataset and model over campus walking routes with synchronized first-person video and gaze	Pedestrian gaze and trajectory prediction	watch
Eyes on Target	2025-11	RAAI 2025	Depth-aware gaze-guided object detection for egocentric video	Gaze-aware object perception	watch
egoEMOTION	2025-10	NeurIPS 2025	50+ hours from 43 participants with Project Aria video, gaze, PPG, IMU, emotion, and personality labels	Affect and personality from wearable capture	watch
Aria Gen 2 Pilot Dataset	2025-10	arXiv	Incremental multimodal daily-activity release captured with Aria Gen 2 glasses across cleaning, cooking, eating, playing, and outdoor walking	Aria Gen 2 wearable sensing	open
LookOut / Aria Navigation Dataset	2025-08	ICCV 2025	4 hours of Project Aria real-world navigation recordings for future 6D head-pose trajectory prediction	Humanoid and assistive egocentric navigation	watch
Aria STDR	2025-07	arXiv	Project Aria egocentric scene-text dataset studying lighting, distance, resolution, and gaze-guided OCR	Scene text recognition under wearable capture	watch
Photoreal Scene Reconstruction from an Egocentric Device	2025-06	SIGGRAPH 2025	Project Aria egocentric-device reconstruction study improving pose and exposure treatment for pixel-accurate photoreal HDR scene reconstruction	Photoreal AR scene reconstruction	watch
Egocentric Event-Based Ping Pong	2025-06	CVPRW 2025	Event-camera first-person table-tennis system and dataset for high-speed ball trajectory prediction under low latency and motion blur	Event-based wearable sports perception	watch
Reading in the Wild	2025-05	NeurIPS 2025	100 hours of Project Aria reading/non-reading video with RGB, gaze, head pose, and reading-type labels	Reading recognition for always-on smart glasses	watch
Digital Twin Catalog / DTC	2025-04	CVPR 2025	2,000 scanned objects with DSLR and egocentric AR-glasses image sequences	Digital-twin object reconstruction and evaluation	open
Head+Cane Camera Navigation	2025-04	arXiv	Synchronized head- and cane-mounted camera comparison for blind last-mile navigation, SLAM, and 3D reconstruction across five real environments	Hybrid wearable/cane sensing for assistive navigation	watch
egoPPG	2025-02	ICCV 2025	Heart-rate estimation from eye-tracking cameras in egocentric systems to add physiological-state awareness to downstream wearer-behavior tasks	Physiology-aware egocentric perception	watch
HEADS-UP	2024-09	arXiv	Head-mounted egocentric dataset for pedestrian trajectory prediction and collision-risk assessment in blind-assistance systems	Trajectory prediction for blind assistance	watch
Smart-Glasses Engagement Prediction	2024-09	arXiv	34-participant smart-glasses dyadic-conversation dataset with self-reported engagement ratings and LLM multimodal-transcript fusion	Social interaction and engagement reasoning from wearable capture	watch
Nymeria	2024-06	ECCV 2024	300 hours, 264 participants, 50 locations, Aria, motion, language, observer view	Egocentric motion, language grounding, body tracking, daily activity	open
Aria Everyday Activities / AEA	2024-02	arXiv	143 daily activity sequences across five indoor locations with trajectories, point clouds, gaze, speech	Everyday AR perception, scene reconstruction, prompted segmentation	open
SANPO	2023-09	WACV 2025	Scene understanding, accessibility, and human navigation data for egocentric navigation and spatial assistance	Navigation and assistive scene understanding	watch
Project Aria Datasets	2023-08	arXiv	Official portal for Aria-based datasets and tooling	AR glasses, wearable sensing, scene reconstruction, gaze, SLAM	open
Aria Digital Twin / ADT	2023-06	arXiv	200 real-world activity sequences, raw Aria streams, 6DoF poses, object poses, depth, segmentation, synthetic renderings	Egocentric 3D machine perception and digital-twin evaluation	open

Benchmarks and Derived Annotations

Evaluation suites and label sets built on top of the raw datasets above.

Benchmark	Released	Venue	Built on	Tasks	Status
SLT 2026 SmartGlasses Challenge	2026-08	IEEE SLT 2026 Challenge	106.98 hours and 714 four-channel smart-glasses audio sessions released to registered participants	Timestamped speaker-attributed ASR and spoken-language understanding for dyadic dialogue and multi-party meetings	request
HandEdit	2026-08	arXiv	300K+ clips and 200M+ editing instances from EgoDex, ARCTIC, OakInk2, HOI4D, and HO-Cap	URDF-conditioned human-to-robot hand/arm editing across 26 embodiments, with public data and evaluation code	open
H2R-Bench	2026-08	arXiv	Egocentric human manipulation demonstrations paired with target robot embodiments	Five-dimension evaluation of 11 video generators across six manipulation families and two robot embodiments	watch
GST-Bench	2026-08	arXiv	6,790 minutes of synthetic video with human-verified questions	Global spatial VQA from egocentric streams, unseen viewpoints, and top-down scenes	watch
EgoAfford	2026-08	arXiv	15.5K human-verified generated images plus 102 real egocentric images across 26 tasks	Joint next-step planning and role-specific affordance segmentation	open
HumanCLAW-Bench	2026-07	arXiv	1,218 egocentric find-navigate-interact episodes across 41 indoor scenes	VLM embodied decision-making, navigation, interaction, and body self-awareness	watch
EgoSafe-Bench	2026-07	arXiv	3,000 first-person mobile clips paired with hierarchical QA chains	12,000 evaluations of evidence anchoring, blind spots, intent, causality, and safety reasoning	watch
VIABench	2026-07	arXiv	First-person videos recorded or shared by blind and visually impaired people	Proactive navigation reminders, assistive VideoQA, and vision-guided interaction in streaming and offline settings	watch
EgoProceVQA	2026-07	arXiv	1,272 clips from CaptainCook4D, EPIC-Tent, Assembly101, and EgoOops	3,600 QA pairs over 31 procedures and six key-step question types, plus EgoProceGen and EgoProceAgent	watch
EgoPolice	2026-07	arXiv	Public police body-worn camera footage	Second-by-second high-stakes action labels, action classification, and multiple-choice QA over egocentric police-civilian interactions	watch
S-EMBER	2026-07	arXiv	Ray-Ban Meta smart-glasses video	3,141 videos, 388 hours, and 9,448 QA pairs for streaming episodic-memory retrieval with temporal evidence	watch
R3D-Bench / R3D	2026-07	arXiv	Aria Digital Twin	3,033 quantitative 3D spatial-reasoning questions over 57 posed egocentric RGB-D video sequences plus a tool-calling baseline	watch
EgoInertia-MI	2026-07	arXiv	Synchronized egocentric video and wearable IMU	Motor-impairment assessment across 19 simulated upper- and lower-body activities and severity levels	watch
LongEgoRefer	2026-07	ECCV 2026	Ego4D	1,498 long-form video referring expressions with average 45-minute videos, sparse object occurrences, and spatio-temporal grounding	open
SG-Ego / GLEN	2026-07	arXiv	Ego4D	Spatio-temporal scene-graph annotations, graph-text/action alignment, and activity-driven graph-edit forecasting	watch
NormAct	2026-06	arXiv	Egocentric embodied-planning scenes	Hidden social-norm compliance, goal achievement, and task-success evaluation for embodied planners	watch
BinaryTracking / GangnamLoop	2026-06	arXiv	Long egocentric robot routes	Spatial QA, metric coordinate retrieval, and navigation with an open BinTrack implementation	open
UMI-Bench 1.0	2026-06	arXiv	UMI-style tabletop manipulation	Local-first real-robot benchmark protocol for UMI policy deployment and task-factor analysis	watch
EgoSAT	2026-06	arXiv	Streaming first-person interaction clips	Egocentric long-horizon interaction QA and decision-level alignment in real-time video streams	watch
NetraLink Assistive AI	2026-06	arXiv	Head-mounted GoPro assistive scenarios	Object recognition, scene-text QA, and multilingual visual reading for wearable MLLMs	watch
VEGA / VEGA-Bench	2026-06	arXiv	In-the-wild egocentric navigation video with monocular geometry	250K scenes and about 5M navigation goals for obstacle-aware VLA evaluation	watch
UCS-Bench / DirectMe	2026-06	ICML 2026	170+ hours of egocentric observations	8.1K+ timestamped questions for continual spatial intelligence and streaming spatial memory	open
StreamMemBench	2026-06	arXiv	EgoLife egocentric streams	Initial/follow-up task sequences testing evidence recall, feedback incorporation, and future assistance	watch
V-RAGBench / CARVE	2026-06	arXiv	Long egocentric video chunks	Decoupled retrieval and generation evaluation for VideoRAG over evidence chunks	watch
VL-MemKnG / WalkieKnowledgeT+	2026-06	arXiv	Long egocentric navigation trajectories	Temporally distributed spatial-memory QA with hybrid graph and segment retrieval	watch
OVO-S-Bench	2026-06	arXiv	Continuous egocentric streams	Hierarchical spatial intelligence across perception, tracking, simulation, and allocentric mapping	watch
SpatialWorld	2026-06	arXiv	Interactive real-world spatial tasks under vision-only partial observability	Egocentric evidence gathering and spatial-agent evaluation	watch
HumanoidArena	2026-06	arXiv	Egocentric humanoid-control tasks	Hierarchical whole-body learning from egocentric perception	watch
Plan, Watch, Recover / EgoProactive	2026-06	arXiv	EgoProactive plus Pro2Bench over five established benchmarks	Proactive procedural assistance, out-of-plan detection, and recovery guidance	watch
Wearable AI Dataset	2026-05	ECCV 2026 Workshop	2,100 gated head-mounted videos split evenly across EgoConv, EgoLongQA, and EgoProactive	Conversational QA, long-video QA, and streaming proactive-assistant evaluation with a bundled starter kit	request
PCSR-Bench	2026-05	arXiv	2,600 omnidirectional indoor images	84,373 perspective-conditioned spatial-reasoning questions, including egocentric rotation and limited-FOV visibility	watch
EgoBench	2026-05	arXiv	Egocentric video tasks	Interactive multimodal tool-using agents	watch
Minerva-Ego	2026-05	arXiv	Egocentric videos with dense reasoning traces and object masks	Multi-step egocentric visual reasoning	open
EgoPro-Bench	2026-05	arXiv	Personalized egocentric video streams	Proactive interaction, intent timing, and personalized assistance	watch
EgoBabyVLM	2026-05	arXiv	Naturalistic infant and adult egocentric videos	Developmental cross-modal VLM evaluation	watch
Beyond Motion Primitives	2026-05	arXiv	Ego4D	Head-mounted IMU benchmark for behavioral activity recognition on smart glasses	watch
Ego-METAS	2026-05	arXiv	EgoExo4D, CMU-MMAC, CaptainCook4D	Online multimodal, energy-aware temporal action segmentation across RGB, audio, gaze, IMU, and monochrome streams	watch
EgoProx	2026-05	CVPR 2026	Egocentric 3D proximity QA	Intention, exploration, exploitation, and chain-of-actions spatial reasoning for MLLMs	watch
BARISTA	2026-05	arXiv	185 coffee-preparation videos	Scene graphs, masks, tracks, boxes, hand-object interactions, activities, and process-step reasoning	watch
TAVIS	2026-05	arXiv	IsaacLab active-vision imitation tasks	Headcam vs fixed-cam evaluation, wrist/head active vision, and anticipatory-gaze metric	watch
MM-Conv	2026-05	arXiv	6.7 hours of egocentric VR interaction	Referential communication with speech, motion, gaze, and 3D scenes	watch
EgoExoMem	2026-05	arXiv	Synchronized ego-exo videos	Cross-view memory QA	watch
EgoMemReason	2026-05	COLM 2026	Week-long EgoLife video; 500 public questions plus evaluation code and leaderboard	Entity, event, and behavior memory reasoning	open
Personal Visual Context Learning / Personal-VCL-Bench	2026-05	arXiv	Continuous smart-glasses streams	Prompt-time wearer-specific visual-context learning for personalized LMM assistants	watch
GazeMind / CogLoad-Bench	2026-05	arXiv	Smart-glasses gaze and cognitive-load annotations	Gaze-guided LLM agent evaluation for personalized cognitive-load assessment	watch
VIGIL	2026-05	arXiv	Egocentric RGB embodied-agent episodes	Terminal-commitment scoring that separates world completion from success reporting	watch
Ego2World	2026-05	arXiv	HD-EPIC	Executable symbolic worlds from egocentric cooking video for belief-state planning	watch
EgoLink 2026	2026-04	ACM MM 2026	E3 plus public challenge videos and labels	Egocentric social reasoning, emotion/intent/causality QA, and interactive tool-using agents	open
SpaMEM	2026-04	arXiv	Procedural embodied environments	Dynamic spatial-memory updates from action-conditioned egocentric observations	watch
WhissleAI Egocentric Activity Sample	2026-04	Hugging Face	19 public first-person clips	Ego4D-style narrations, NLQ, moment annotations, FHO actions, metadata, and taxonomy for prototyping	open
EgoPoint-Bench	2026-04	ACL 2026	Simulated and real egocentric pointing samples	11K+ QA items for referential reasoning and pointing-grounded object disambiguation	watch
PIE-V	2026-04	arXiv	Ego-Exo4D-style procedural scenarios	Mistake-aware procedural-video evaluation with recovery corrections	watch
ReFocus / EM-QnF	2026-04	CVPR 2026	Egocentric episodic-memory NLQ with user feedback	Interactive feedback refinement for ambiguous memory queries	watch
EgoEsportsQA	2026-04	arXiv	First-person esports video	Fast virtual first-person perception and reasoning	watch
Audio Hallucination in Egocentric Video	2026-04	ICASSP 2026	Egocentric audio-visual video	300 videos / 1,000 sound-focused questions probing audio hallucination in AV-LLMs	watch
EXPLORE-Bench	2026-03	arXiv	Real first-person videos	Egocentric long-horizon scene-state prediction and reasoning	watch
LifeEval	2026-03	arXiv	Continuous first-person streams	Real-time task-oriented human-AI collaboration in daily life	watch
SAVA-X	2026-03	CVPR 2026	EgoMe and asynchronous ego/exo video pairs	Ego-to-exo imitation-error detection	watch
HOI-Synth	2026-03	arXiv	VISOR, EgoHOS, and ENIGMA-51 with synthetic hand-object interaction annotations	Synthetic HOI detection and benchmarking under synthetic augmentation	watch
MA-EgoQA	2026-03	arXiv	Multi-agent egocentric streams	Social, task coordination, theory-of-mind, temporal, environment QA	open
Ropedia Xperience-10M Task Suite	2026-03	Hugging Face	Xperience-10M Sample	12 embodied-AI task contracts, sample baselines, and evaluation protocol	open
Kriya-Egocentric-100K	2026-03	Hugging Face	Egocentric-100K preview subset	Action100M-style hierarchical temporal action trees and LLM-generated action descriptions	open
EgoAVU	2026-02	CVPR 2026	Egocentric audio-visual narrations	EgoAVU-Instruct (3M QAs) and EgoAVU-Bench (3K QAs) for audio-visual understanding (CVPR 2026 highlight)	open
SAW-Bench	2026-02	ICML 2026	Ray-Ban Meta smart-glasses video	Observer-centric situated awareness and physically grounded spatial reasoning	open
Ego4OOD	2026-01	arXiv	Egocentric video action recognition	Covariate-shift benchmark for egocentric domain generalization	watch
Sanpo-D	2026-01	arXiv	Sanpo egocentric navigation video	Spatial-conditioned reasoning over long first-person videos with fine-grained spatial re-annotation	watch
Egocentric-100K Evaluation	2025-12	Hugging Face	Egocentric-100K, Ego4D, EPIC-KITCHENS-100	30K-frame comparison of hand visibility and active manipulation density	open
Know-Show	2025-12	arXiv	Charades, Action Genome, Ego4D	Spatio-temporal grounded reasoning with joint answer and evidence localization	watch
Egocentric-10K Evaluation	2025-11	Hugging Face	Egocentric-10K, Ego4D, EPIC-KITCHENS-100	30K-frame comparison of hand visibility and active manipulation density with published labeling prompts	open
CS3 Collision Sound Source Segmentation	2025-11	arXiv	EPIC-CS3 and Ego4D-CS3	Audio-conditioned segmentation of objects responsible for collision sounds in egocentric video	watch
EgoExo-Con	2025-10	arXiv	Synchronized ego-exo videos	View-invariant temporal verification and grounding; introduces View-GRPO	watch
LaMAria	2025-09	ICCV 2025	Glasses-like wearable capture	City-scale egocentric visual-inertial SLAM with centimeter ground truth	open
EgoIllusion	2025-08	EMNLP 2025	Egocentric video	Hallucination benchmark probing fabricated objects, actions, and sounds in multimodal models	watch
EgoCross	2025-08	AAAI 2026	Cross-domain ego clips; the EgoVis 2026 challenge drew 1,500+ submissions from 130+ participants	Surgery, industry, extreme sports, animal-perspective QA, and two Codabench tracks	watch
EgoExoBench	2025-07	arXiv	Public ego-exo video datasets	7,300+ QA pairs over semantic alignment, viewpoint association, and temporal reasoning	watch
EASG-Bench	2025-06	ICCV 2025 Workshop	Egocentric action scene graphs	Relation and temporal QA	open
HD-EPIC VQA Challenge	2025-02	CVPR 2025	HD-EPIC	Recipe, ingredient, nutrition, fine-grained action, 3D perception, object motion, gaze	benchmark
EFM3D	2024-06	arXiv	Project Aria egocentric 3D data	3D object detection and surface regression for egocentric foundation models	watch
Ego4D-OSCA	2024-05	arXiv	Ego4D	Object-state-change anticipation annotations and visual-language baseline for procedural video	watch
EgoHOIBench / EgoNCE++	2024-05	ICLR 2025	Multiple egocentric HOI sources	Open-vocabulary hand-object interaction understanding	open
EgoPlan-Bench	2023-12	IJCV	Egocentric planning tasks	Multimodal LLM planning over human-level egocentric scenarios	open
Ego-Exo4D Benchmarks	2023-11	project page	Ego-Exo4D	Fine-grained activity, proficiency, cross-view translation, 3D pose, object correspondence	benchmark
Egocentric Pedestrian Trajectory Benchmark	2023-10	ICRA 2024	Egocentric pedestrian video	Trajectory prediction with scale- and motion-aware evaluation	watch
EPIC-Aff / Multi-label Affordance Mapping	2023-09	ICCV 2023	EPIC-KITCHENS	Metric, spatial, multi-label affordance annotations for interaction-grounded segmentation and navigation maps	benchmark
RefEgo	2023-08	ICCV 2023	Ego4D	12K+ clips / 41 hours for first-person referring-expression comprehension and referred-object tracking	open
EgoSchema	2023-08	NeurIPS 2023	Ego4D	Very-long-form multiple-choice video QA	open
EgoAdapt 2023	2023-07	arXiv	Ego4D	Real-world online adaptation benchmark over 50 user streams and 2,740 action labels	benchmark
EPIC-Fields	2023-06	NeurIPS 2023	EPIC-KITCHENS	3D fields and scene-level spatial reasoning over kitchen video	open
MMG-Ego4D	2023-05	CVPR 2023	Ego4D	Missing-modality and cross-modal zero-shot generalization benchmark over video, audio, and IMU	benchmark
Fine-Grained Affordance Annotation	2023-02	WACV 2023	Egocentric HOI videos	Fine-grained affordance labels for hand-object interaction	watch
EPIC-Sounds	2023-02	ICASSP 2023	EPIC-KITCHENS	Audio event recognition in egocentric kitchen video	open
VISOR	2022-09	NeurIPS 2022	EPIC-KITCHENS	Manual and dense masks, hand/object segmentation, active object relations	open
EgoClip / EgoMCQ	2022-06	NeurIPS 2022	Ego4D	3.8M clip-text pairs and MCQ development benchmark for egocentric VLP	open
AssistSR	2021-11	EMNLP 2021	Instructional daily-item video segments	Task-oriented question-driven video segment retrieval for personal assistants	watch
Ego4D Benchmarks	2021-10	project page	Ego4D	Natural Language Query, Moment Query, episodic memory, state change, long-term anticipation, social/audio, hand-object	benchmark
TREK-150	2021-08	project page	EPIC-KITCHENS	Egocentric single-object tracking	open
EPIC-KITCHENS Challenges	2018-04	project page	EPIC-KITCHENS / EPIC-KITCHENS-100	Recognition, detection, anticipation, retrieval, domain adaptation	benchmark

Models, Tools, and Baselines

Open models, baselines, and loaders you can build on directly.

Egocentric Foundation Models and Assistants

Resource	Released	Venue	What it contributes	Link
Vision-Language Models for Egocentric Video	2026-08	arXiv	Critical survey connecting egocentric VLMs across hand-object interaction, temporal and graph reasoning, efficient long-video processing, wearable assistance, and embodied AI	Paper
EgoGazeLite	2026-08	ECCV 2026 WearableAI Workshop	Software-only gaze prediction for token-efficient wearable MLLM input with 15.7M parameters and 21.6 ms/frame end-to-end latency	Paper
Long-Tail Embodied Urban Navigation from In-the-Wild Videos	2026-08	arXiv	Scalable annotation of web egocentric video with metric trajectories and navigation semantics for VLA training and rare-failure analysis	Paper
Hand-Object Interaction in the Age of Large Foundation Models	2026-07	arXiv	Systematic survey of foundation-model priors across six HOI tasks and eight geometric, semantic, and visual prior types, including embodied transfer	Paper
Ego Scene Augmentation (ESA)	2026-07	arXiv	Explicit ego-element graph that fuses visual-foundation-model signals for spatially structured scene understanding and stronger indoor/outdoor EgoTextVQA	GitHub
Worldscape-MoE	2026-07	arXiv	MoE diffusion world model sharing dynamics across camera trajectories, robot actions, and egocentric hand-joint control; code and weights are announced but not yet released	Project
UNIEGO	2026-06	arXiv	Unified egocentric encoder distilled from nine teachers spanning ego-exo views, RGB, depth, skeleton, and foundation-model representations	Paper
ActiveMimic	2026-06	arXiv	Egocentric human-video pretraining with active-perception signals for manipulation and VLA transfer	Paper
Wh0	2026-06	arXiv	World-model framework that synthesizes realistic hand-object manipulation sequences to bootstrap scalable egocentric data and policy learning	Paper
VLESA	2026-06	arXiv	Goal-conditioned safety-assistance evaluation and baseline for monitoring human activities from egocentric video	GitHub
Continual Child-View Learning	2026-06	arXiv	Chronological multimodal learning from a child's egocentric video and speech stream	Paper
Objects Before Words	2026-06	arXiv	Object-first language grounding from child-view egocentric video	Paper
Watch Remember Reason	2026-06	arXiv	Human-view long-video understanding framework for MLLM watching, memory, and reasoning	Paper
World Action Models	2026-05	arXiv	Survey and taxonomy linking VLA, world models, portable human demonstrations, simulation, and internet-scale egocentric video	Paper
Pro2Assist	2026-05	arXiv	Continuous step-aware proactive assistance framework with multimodal egocentric perception and AR-glasses evaluation	Paper
T-REN	2026-04	arXiv	Text-aligned region-token encoder that reduces long-video token counts and improves Ego4D video object localization	GitHub
V-JEPA 2.1	2026-03	arXiv	Dense self-supervised video representation model with strong Ego4D STA, EPIC-KITCHENS anticipation, and robot-grasping transfer results	Paper
EgoViT Object SSL	2026-03	CVPR 2026	Self-supervised object representation learning from continuous, uncurated first-person video	Paper
RynnBrain	2026-02	arXiv	Open embodied foundation model family with variants for egocentric understanding, localization, physical reasoning, planning, navigation, and VLA	Paper
Central Vision SSL	2026-02	arXiv	Ego4D-derived gaze-centered crops and temporal-slowness SSL for object representation learning from human-like visual experience	GitHub
PhysBrain	2025-12	arXiv	Uses human egocentric data to bridge vision-language models toward physical intelligence and embodied control	Paper
EgoM2P	2025-06	ICCV 2025	Egocentric multimodal multitask pretraining over RGB, depth, gaze, and camera pose	Paper
Exo2Ego / Ego-ExoClip	2025-03	AAAI 2026	Transfers exocentric MLLM knowledge into egocentric video understanding with 1.1M synchronized ego-exo clip-text pairs and EgoIT instruction tuning	Paper
GazeLLM	2025-03	Augmented Humans 2025	Uses gaze-focused first-person regions to reduce MLLM video processing while preserving task comprehension	Paper
EgoHOD	2025-03	ICLR 2025	Fine-grained hand-object-dynamics pretraining for egocentric representations (ICLR 2025)	GitHub
EgoSpeak	2025-02	NAACL 2025 Findings	Real-time speech-initiation prediction for egocentric conversational agents over first-person RGB streaming video	Paper
PRVQL	2025-02	ICCV 2025	Progressive knowledge-guided refinement for egocentric visual query localization under appearance change and clutter	Paper
Human Gaze Object-Centered Learning	2025-01	arXiv	Gaze-inspired central visual amplification for object-centered representation learning from Ego4D-scale egocentric experience	Paper
GEM Ego-Vision World Model	2024-12	CVPR 2025	Generalizable ego-vision world model trained on 4,000+ hours for RGB/depth generation and ego-trajectory control	Paper
Vinci	2024-12	IMWUT 2025	Real-time, always-on egocentric assistant with historical-context QA, planning, and visual demos	GitHub
Ego-VPA	2024-07	WACV 2025	Parameter-efficient adaptation for egocentric video-language foundation models with sparse basis prompts	Paper
EgoVideo	2024-06	CVPR 2024	Egocentric video foundation model with slow-fast adaptation; multi-track Ego4D / EPIC challenge winner	GitHub
AlanaVLM	2024-06	EMNLP 2024	Embodied-AI foundation model for egocentric video understanding	Paper
Missing Modality Token / MMT	2024-01	CVPRW 2025	Missing-modality modeling for multimodal egocentric action recognition and moment localization on Ego4D, EPIC-KITCHENS, and EPIC-Sounds	Paper
EgoDistill	2023-01	tech report	Distills heavy egocentric clip features into efficient models using head-motion signals	Project
EgoT2 / Egocentric Video Task Translation	2022-12	CVPR 2023 Highlight	Task-translation framework that transfers supervision between egocentric video tasks	Project

Video-Language and Long-Video Models

Resource	Released	Venue	What it contributes	Link
MERIT	2026-08	ECCV 2026 Oral	Simple multi-key episodic memory and query-time neighbor filtering for high-recall ultra-long video retrieval; reports 71.2 EgoLifeQA accuracy with a GPT-5 backbone	Project
SCOUT	2026-08	ACM MM 2026	Recovery-aware tool-thought agent that self-checks retrieval observations, switches temporal regions, and trains with uncertainty-prioritized policy optimization	Paper
R4DSG	2026-08	ACM MM 2026	Relative 4D scene-graph memory for persistent objects, anchor-relative state changes, and object-centric questions in long EgoLife video	Project
EgoCITE	2026-08	arXiv	Context-augmented multi-view memory indices plus question-conditioned temporal retrieval over EgoLifeQA, EgoMem, and EgoR1-Bench	Paper
SmartRes	2026-08	arXiv	Dynamic pixel-space resolution routing cuts visual tokens by up to 67% on Ego4D/EgoIntention grounding while retaining 86.4% of full-resolution performance	Paper
EgoPlay	2026-07	SIGGRAPH Asia 2026	Event-triggered egocentric video editing trained on 106K clip-prompt pairs, jointly learning trigger recognition, temporal restraint, and post-event editing	Project
VideoTreeSearch	2026-07	arXiv	Self-correcting temporal-tree agent for grounded long-video QA, with MIT-licensed inference/SFT/RL code, an 8B checkpoint, and released Haystack-Ego4D trajectories/evaluation data	GitHub
OmniView-Space	2026-07	arXiv	Tool-guided egocentric spatial reasoning with query-aligned visual cognitive maps, textual spatial graphs, and ego-frame rewards	Paper
CGGS	2026-07	IEEE TIP 2026	Text-to-3D egocentric scene generation with consistency-augmented 2D priors, point-track/depth layout decoration, and geometric Gaussian refinement	GitHub
Egocentric Scene Graphs / EgoSG	2026-06	arXiv	Converts long-form egocentric videos into temporally grounded scene graphs for compact HD-EPIC VQA reasoning	Paper
Low-Latency Doubly-Correct VLMs	2026-06	arXiv	Rationale-informed pruning for efficient egocentric VLMs while preserving both answer correctness and evidence grounding	Paper
DR-MV3D	2026-06	arXiv	Dense-reward multi-view 3D VQA pipeline with global-map construction, view planning, and egocentric grounding	Paper
Temporal Action Graphs for Ego VLMs	2026-06	arXiv	Converts egocentric videos into narratives and temporal action graphs for in-context action recognition with open-weight VLMs	Paper
ReRe Cross-View Revisiting	2026-06	ICML 2026	Training-free spatial reasoning that revisits egocentric conclusions through synthesized complementary novel-view videos	Project
CASTLE KG Retrieval	2026-06	CVPR 2026 EgoVis	Agentic long-context video understanding with video knowledge graphs and hierarchical retrieval for the CASTLE challenge	Paper
AnchorWorld	2026-06	arXiv	Embodied egocentric world simulation with view-based evolution customization	Paper
Understanding-Enhanced Ego Mistake Detection	2026-06	arXiv	Small/large-model collaboration for detecting incorrect procedural actions in egocentric video, with a Qwen3-VL Embedding reasoning branch	Paper
FlexLAM	2026-06	arXiv	Variable-length latent actions for action-free video interfaces, improving scarce-label alignment and Ego4D transition reconstruction	Paper
VisualClaw	2026-06	arXiv	Real-time personalized multimodal agent with streaming-frame filtering, skill evolution, and VisualClawArena workspace evaluation	Paper
FlowNar	2026-05	ICML 2026	Scalable streaming narration with bounded visual memory, evaluated on long-form Ego4D, Ego-Exo4D, and EPIC-KITCHENS-100-style narration	GitHub
E3C Video Generation	2026-05	arXiv	Controllable egocentric video diffusion with 3D environmental memory and ego-exo human-pose control	Project
CASTLE2026 Team WDL	2026-05	CVPR 2026 EgoVis	Evidence-aware multimodal reasoning pipeline for long-form CASTLE egocentric QA	Paper
CuriosAI CASTLE	2026-05	CVPR 2026 EgoVis	Search-verify-answer CASTLE challenge pipeline using timelines, transcripts, and VLM captions	Paper
MARS CASTLE	2026-05	CVPR 2026 EgoVis	Multimodal agentic reasoning with source selection for CASTLE challenge QA	Paper
OSGNet + MLLM Reranking	2026-05	CVPR 2026 EgoVis	Champion Ego4D Episodic Memory Challenge solution for NLQ and GoalStep using MLLM reranking over OSGNet candidates	GitHub
OmniEgo-R2	2026-05	CVPR 2026 EgoVis	Routed reasoning framework for EgoCross; second place in both Source-Limited and Open-Source tracks	GitHub
Being-H0.7	2026-05	arXiv	Latent world-action model from egocentric videos for future-aware reasoning and VLA policy learning	Paper
Reflective Dialogue EgoCross	2026-05	CVPR 2026 EgoVis	Inference-time Teacher/Solver reflective dialogue for EgoCross support-set adaptation without fine-tuning	Paper
EgoCross Domain-Wise Inference	2026-05	CVPR 2026 EgoVis	Nearly training-free source-limited inference strategy for EgoCross domain shift	Paper
HD-EPIC Semantic-Visual Evidence	2026-05	CVPR 2026 EgoVis	HD-EPIC VQA challenge solution separating semantic and visual evidence	Paper
HiERO-StepG	2026-05	CVPR 2026 EgoVis	Hierarchical activity-understanding solution for the Ego4D Step Grounding Challenge	Paper
TempRet	2026-05	CVPR 2026 EgoVis	Temporal enhancement and reranking for EPIC-KITCHENS-100 multi-instance retrieval	Paper
Trajectory-Conditioned Egocentric Prediction	2026-05	arXiv	Future ego-view prediction conditioned on camera trajectory to disambiguate action outcomes	Paper
EventPrune	2026-05	arXiv	Event-camera-guided visual-token pruning for efficient first-person dynamic spatial reasoning	Paper
SpatioRoute	2026-05	arXiv	Question-aware prompt routing for zero-shot egocentric spatial QA without fine-tuning or 3D input	Paper
MAGIC-Video	2026-05	arXiv	Multimodal memory graph plus narrative-chain retrieval for ultra-long egocentric video reasoning	GitHub
SpaceMind++	2026-05	arXiv	Builds allocentric cognitive maps from RGB video to support spatially consistent reasoning over fragmented egocentric observations	Paper
EgoSim	2026-04	ECCV 2026	Closed-loop egocentric world simulator with 3D grounding and dynamic state updates; inference code and weights are public while training code and a license remain pending	Project
World2VLM	2026-04	arXiv	Distills world-model imagination into VLMs for dynamic spatial reasoning under egocentric motion	Paper
EgoMotion	2026-04	arXiv	Hierarchical reasoning + diffusion for egocentric vision-language motion generation	Paper
Ego-InBetween	2026-04	CVPR 2026	Generates object state transitions in ego-centric videos from action instructions	Paper
EgoTSR	2026-04	arXiv	Curriculum-based task-oriented egocentric spatiotemporal reasoning with a reported 46M-sample EgoTSR-Data training set	Paper
Syn2Seq Exo-to-Ego	2026-04	arXiv	Sequential exo-to-ego video generation from synchronized third-person views and camera poses	Paper
UniversalVTG	2026-04	arXiv	Lightweight cross-dataset foundation model for video temporal grounding	Paper
V-Nutri	2026-04	CVPR 2026 MetaFood Workshop	Dish-level nutrition estimation from egocentric cooking videos	Paper
LOME	2026-03	arXiv	Action-conditioned egocentric world model generating photorealistic human-object interactions from image, text, and per-frame actions	Paper
Temporal-Aware Ego VLM	2026-03	arXiv	Training scheme that incentivizes temporal awareness in egocentric video-understanding models	Paper
EgoForge	2026-03	arXiv	Goal-directed egocentric world simulator that rolls out first-person video from a single image and a high-level instruction	Paper
EgoReasoner	2026-03	arXiv	Task-adaptive structured thinking for egocentric 4D spatial and object reasoning	Paper
Gaze-Regularized VLMs	2026-03	arXiv	Gaze-conditioned VLM training for ego-centric behavior understanding	Paper
Recurrent Reasoning VLM	2026-03	CVPR 2026	Recurrent VLM reasoning for long-horizon embodied task-progress estimation	Paper
Ropedia Xperience-10M Task Baselines	2026-03	Hugging Face	Public Xperience-10M sample task definitions, baselines, metrics, and scale-up evaluation notes	Hugging Face
EgoMAS	2026-03	arXiv	Shared-memory baseline for multi-agent egocentric video QA	Project
DreamDojo	2026-02	ICML 2026	Generalist robot world model pretrained on 44K hours of egocentric human video with continuous latent actions, distilled to real time	GitHub
Hand2World	2026-02	arXiv	Autoregressive egocentric world model generating first-person interaction video from free-space hand gestures	Paper
Edge Episodic Memory QA	2026-02	VISAPP 2026	On-device dual-thread MLLM for real-time egocentric episodic-memory QA (QAEgo4D-Closed) on the edge	Paper
EgoGraph	2026-02	arXiv	Training-free temporal knowledge graph for ultra-long egocentric video understanding	Paper
EGAgent	2026-01	arXiv	Entity-scene-graph agent for very-long egocentric video understanding over all-day wearable streams	GitHub
Walk through Paintings	2026-01	arXiv	Egocentric world models from internet priors that generate first-person scene walkthroughs	Paper
Event-VStream	2026-01	arXiv	Event-driven long-video stream understanding for real-time video-language systems	Paper
EgoHandICL	2026-01	ICLR 2026	First-person hand-object in-context learning from multimodal egocentric context and synthetic exemplars	Paper
HD-EPIC VQA T-CoT	2026-01	CVPR 2025 EgoVis	HD-EPIC VQA solution with temporal chain-of-thought prompting and Qwen2.5-VL adaptation	Paper
Robust Egocentric Visual Attention	2026-01	arXiv	Language-guided scene-context model for egocentric visual attention prediction	Paper
The N-Body Problem	2025-12	ECCV 2026 Workshop	Predicts feasible multi-person parallel execution plans from a single EPIC-KITCHENS or HD-EPIC egocentric video	Project
EgoVITA	2025-11	ECCV 2026	Plan-then-verify egocentric video reasoning with visual-grounding and cross-perspective consistency rewards	Paper
EgoControl	2025-11	arXiv	Pose-controllable egocentric video diffusion conditioned on sequences of 3D full-body poses	Paper
EAGLE VQL	2025-11	AAAI 2026	Episodic appearance- and geometry-aware memory for unified 2D/3D visual query localization in egocentric video	Paper
TimeSearch-R	2025-11	arXiv	Reinforcement-learned temporal search with self-verification for Haystack-Ego4D and long-video understanding	GitHub
DMC3 Ego VideoQA	2025-10	ACM MM 2025	Counterfactual contrastive construction for egocentric VideoQA across event descriptions and hand-object interaction cues	Paper
EgoThinker	2025-10	NeurIPS 2025	Egocentric reasoning model with spatio-temporal chain-of-thought and RL fine-tuning	Paper
HieraMamba	2025-10	CVPR 2026	Hierarchical Anchor-Mamba pooling for long-video temporal grounding, preserving fine temporal structure on Ego4D NLQ-style queries	Project
EgoPrompt	2025-08	ACM MM 2025	Prompt-learning framework for egocentric action recognition that models verb-noun semantic and contextual relationships	Paper
EgoTwin	2025-08	arXiv	Joint first-person video and human-motion generation with head-centric motion alignment	Paper
Ego-PM	2025-08	arXiv	Egocentric predictive model that jointly forecasts future actions and video frames conditioned on hand trajectories	Paper
GazeNLQ	2025-06	CVPR 2025 Ego4D Challenge	Gaze-augmented Ego4D Natural Language Queries localization from estimated wearer attention	Paper
EgoWorld	2025-06	ICLR 2026	Reconstructs egocentric views from exocentric point clouds, 3D hand poses, and text for cross-view world generation	Project
EgoVLM	2025-06	arXiv	GRPO-style policy optimization over Qwen2.5-VL-3B for egocentric video reasoning	Paper
Ego-R1	2025-06	arXiv	Chain-of-tool-thought framework and training data for ultra-long egocentric video QA	Paper
HiERO	2025-05	ICCV 2025	Weakly supervised hierarchical activity-thread representations for EgoMCQ, EgoNLQ, and procedure learning	Paper
OSGNet	2025-05	CVPR 2025	Object-shot enhanced grounding for egocentric temporal grounding / moment queries (CVPR 2025)	GitHub
EgoExo-Gen	2025-04	ICLR 2025	Predicts future ego-centric video from exocentric video, first ego frame, text instructions, and hand-object masks	Paper
EgoDTM	2025-03	arXiv	Depth- and text-aware egocentric VLP for 3D-aware representations	GitHub
EgoLife / EgoButler / EgoGPT / EgoRAG	2025-03	arXiv	Omni-modal egocentric assistant system with retrieval over long life recordings	Paper
EgoAgent	2025-02	ICCV 2025	Joint predictive agent model for perception, future-state prediction, and action in egocentric worlds	Paper
Embodied VideoAgent	2024-12	ICCV 2025	Builds persistent 3D scene memory from egocentric video plus depth/pose sensors for dynamic scene reasoning and planning	Project
ESOM / OVQ2D	2024-11	WACV 2026	Online visual query localization with compact egocentric streaming object memory	Paper
MM-Ego	2024-10	ICLR 2025	Egocentric multimodal LLM with Memory Pointer Prompting and a 7M-sample egocentric QA data engine (EgoMemoria benchmark)	Paper
AMEGO	2024-09	ECCV 2024	Active-memory representation from a single long egocentric video for fast multi-query answering (ECCV 2024)	Project
EgoNCE++	2024-05	ICLR 2025	Open-vocabulary HOI benchmark and asymmetric contrastive objective	GitHub
SoundingActions	2024-04	CVPR 2024	Self-supervised audio-language-vision embedding that learns how actions sound from narrated egocentric videos	Project
Video ReCap	2024-02	CVPR 2024	Recursive captioning for hour-long videos plus Ego4D-HCap long-range summaries	Project
EgoInstructor	2024-01	CVPR 2024	Retrieval-augmented egocentric captioning via cross-view retrieval of third-person clips (CVPR 2024)	Project
GroundVQA	2023-12	CVPR 2024	Unified temporal grounding and open/closed QA for long egocentric videos	GitHub
LifelongMemory	2023-12	arXiv	LLM-based long-form egocentric memory system for answering queries over a camera wearer's past	Paper
LEGO	2023-12	ECCV 2024	Visual-instruction-tuned egocentric action-frame generation	Paper
Action Scene Graphs	2023-12	CVPR 2024	Temporally evolving action scene graphs for long-form egocentric video understanding	Paper
LEAP	2023-11	arXiv	LLM generation of egocentric action programs with sub-actions, preconditions, postconditions, and object references	Paper
PALM	2023-11	ECCV 2024	Predicts future actions through language models for long-term action anticipation	Paper
Exo2EgoDVC	2023-11	WACV 2025	Dense video captioning for egocentric procedural activities using web instructional videos	Project
CliMer Temporal Grounding	2023-10	BMVC 2023	Learns temporal sentence grounding from narrated Ego4D/EPIC videos using clip merging and rough narration timestamps	GitHub
Encode-Store-Retrieve	2023-08	ISMAR 2024	Language-encoded egocentric memory pipeline for storing and retrieving human-perception events	Paper
EgoVLPv2	2023-07	ICCV 2023	Egocentric video-language pretraining with fusion in the backbone	Project
VQLoC	2023-06	NeurIPS 2023	Single-stage visual query localization with joint query-frame and frame-frame correspondences for long egocentric video	Project
EgoCOL	2023-06	CVPR 2023 Ego4D Challenge	Camera-pose estimation for Ego4D open-world 3D object localization using sparse video/scan pose reconstruction	GitHub
SpotEM	2023-06	ICML 2023	Efficient episodic-memory search that keeps most NLQ accuracy while computing only a small fraction of clip features	Project
GroundNLQ	2023-06	CVPR 2023	Two-stage multi-scale grounding for Ego4D Natural Language Queries; CVPR 2023 NLQ champion	GitHub
MINOTAUR	2023-02	arXiv	Unified video grounding model for multimodal queries over Ego4D-style grounding tasks	Paper
NaQ	2023-01	CVPR 2023	Uses narrations as implicit queries to supervise episodic-memory retrieval in long egocentric video	Paper
LaViLa	2022-12	CVPR 2023	Learns video-language representations from Ego4D narrations and LLM-generated narrations	Paper
CocoFormer VQL	2022-11	Ego4D Challenge 2022	Proposal-set transformer for Ego4D visual query localization, improving VQ2D/VQ3D object localization	GitHub
Negative Frames Matter	2022-08	Ego4D Challenge 2022	Trains Ego4D VQ2D with noisy negative/background frames and a more efficient object-query training loop	GitHub
EgoEnv	2022-07	NeurIPS 2023 Oral	Human-centric environment representations linking egocentric video to the wearer's local surroundings	Project
EgoVLP / EgoNCE	2022-06	NeurIPS 2022	EgoClip, EgoNCE objective, EgoMCQ, and transfer to Ego4D/EPIC/Charades-Ego tasks	GitHub

Action, Tracking, Pose, and HOI Baselines

Resource	Released	Venue	What it contributes	Link
DreamHand	2026-08	arXiv	Deterministic diffusion geometry encoder for continuous metric bimanual trajectories under occlusion and out-of-sight gaps, with large gains on ARCTIC and HOT3D	Project
G3Ego	2026-08	ECCV 2026 CONTEXTUS Workshop	Gaze-guided pruning of action-scene graphs for efficient and interpretable recognition and anticipation on EGTEA Gaze+ and MECCANO	Paper
EgoHRV	2026-08	ECCV 2026	Heart-rate and HRV estimation from headset gaze cameras; reported HRV features improve EgoExo4D proficiency classification by 17.8%	Paper
EgoTac	2026-08	arXiv	Predicts continuous force and contact from egocentric video after training on more than 5.7M image-tactile pairs	Paper
EgoTrack3D	2026-08	arXiv	Dynamic 3D object tracking from egocentric RGB through global-frame mask lifting, motion scoring, and voxel association; reports 11% higher PCL on Aria Digital Twin	Paper
EgoHieraLoc	2026-08	arXiv	Unified 2D/3D visual-query localization with segmentation-guided representations and geometry-semantic confidence across viewpoints	Paper
EgoPHI	2026-08	ECCV 2026	Dense hand/object contact maps and 3D force distributions from one ego RGB image and object geometry, with simulation supervision and an eight-participant physical evaluation	Paper
Clinical Freezing-of-Gait with Egocentric Vision	2026-08	ECCV 2026 Workshop	At-home comparison of ego-video and IMU representations using synchronized recordings and expert labels from 13 people with Parkinson's disease	Paper
HiResNets	2026-08	arXiv	Native Full-HD foveal residual streams preserve fine egocentric objects without quadratic feature-grid growth	Paper
Trustworthy Visual Predicates	2026-06	arXiv	Reliability framework for contact, grasp, release, support, and other manipulation predicates under visual degradation, evaluated on VISOR/EPIC-KITCHENS, H2O, and ARCTIC	Paper
FactCheck LTA	2026-06	arXiv	Feasibility-aware long-term action anticipation with a multi-agent Observe-Plan-Verify loop over EPIC-KITCHENS-55 and EGTEA Gaze+	Paper
TrAction	2026-06	arXiv	Sparse 2.5D point-trajectory transformer for action recognition that reduces appearance/background shortcuts on EPIC-KITCHENS-100	GitHub
HumanScale	2026-06	arXiv	Shows filtered, labeled egocentric human video can outperform teleoperated real-robot data for embodied pretraining	Paper
Do as I Do	2026-06	arXiv	Retargets human hand-object interactions from in-the-wild monocular video into executable dexterous robot trajectories	Paper
Motion-Focused Latent Action VLA	2026-06	IROS 2026	Extracts motion-focused latent actions from unlabeled human EgoVideos for cross-embodiment VLA pretraining and adaptation	Paper
EgoPriMo	2026-06	arXiv	Egocentric motion prior for interactive humanoid control from human demonstrations	Paper
EgoPhys	2026-06	arXiv	Learns deformable-object physics digital twins from egocentric RGB interaction video for robot planning	Project
EDITH	2026-06	arXiv	Streams first-person view, gaze, and speech from smart glasses into hierarchical robot policies for natural HRI	Project
Divide Deliberate Decide	2026-06	arXiv	Local zero-shot multi-agent VLM framework for fine-grained egocentric action recognition	Paper
Ego-Nav Co-training	2026-06	arXiv	Converts egocentric walking videos into robot-action datasets and co-trains a VLA with robot demos for mobile navigation	Paper
EgoGuide	2026-06	arXiv	Synchronized wrist/head egocentric demonstration collection with online guidance and a gated egocentric residual policy	Project
Ego-Pi	2026-06	arXiv	VLA fine-tuning study across egocentric human and robot data using pi0.5 and dexterous five-finger embodiments	Project
HALOMI	2026-06	arXiv	Extends UMI-style egocentric collection for humanoid loco-manipulation with active perception from human demonstrations	Paper
EgoTactile	2026-06	ICML 2026 spotlight	Egocentric video to full-hand grasp-pressure benchmark and diffusion/baseline models for everyday-object interactions	Project
EgoPressDiff	2026-06	ICASSP 2026	Conditional video diffusion for UV-domain egocentric hand-pressure maps using pose, mesh, and depth conditioning	Project
Hand Trajectory Fusion for Ego NLQ	2026-06	CVPR 2026 EgoVis	Hand-trajectory encoder and cross-attention fusion for Ego4D Natural Language Query grounding	Paper
PROSE	2026-06	arXiv	Training-free RGB-only egocentric scene registration using VLM-derived object-level 3D scene graphs	Project
ACE-Ego-0	2026-06	arXiv	Unifies egocentric human video with robot/sim data via reliability-aware weighting for VLA pretraining (RoboCasa GR1, RoboTwin 2.0)	Paper
HumanNet	2026-05	arXiv	1M-hour mixed first-/third-person human-centric corpus with interaction annotations and a 1K-hour egocentric subset used for VLA validation	Paper
EgoAction	2026-05	CVPR 2026	CVPR 2026 EPIC-KITCHENS action detection challenge pipeline	Paper
EgoAdapt	2026-05	CVPR 2026	CVPR 2026 HD-EPIC VQA challenge inference-time adaptation pipeline	Paper
FROST-STA	2026-05	CVPR 2026 EgoVis	Frozen dense-feature Ego4D short-term object-interaction anticipation submission	Paper
JFAA	2026-05	CVPR 2026 EgoVis	JEPA-based EPIC-KITCHENS-100 action-anticipation challenge submission	Paper
Mamba Ego Action Recognition	2026-05	CVPR 2026 EgoVis	Cross-modal egocentric action recognition using RGB and hand-skeleton streams with Mamba	Paper
TAP-JEPA	2026-05	CVPR 2026 EgoVis	EPIC-KITCHENS-100 action-anticipation runner-up using frozen V-JEPA 2.1 features	Paper
VISTA	2026-05	CVPR 2026 EgoVis	V-JEPA plus StillFast temporal anticipator for Ego4D STA at EgoVis 2026	Paper
WristCompass	2026-05	arXiv	Learns ego-camera orientation from hand/camera kinematic coupling in manipulation video	Paper
Zero-Shot Ego Object ReID	2026-05	arXiv	SAM3-feature fusion for zero-shot object re-identification in egocentric kitchen videos	Paper
EgoRelight	2026-05	arXiv	HMD-based egocentric human capture and illumination recovery for relightable avatars	Paper
StableHand	2026-05	arXiv	Quality-aware flow-matching baseline for world-space dual-hand motion estimation from egocentric video	Project
HumanEgo	2026-05	arXiv	Zero-shot robot learning from minutes of human egocentric video; public code and 122 Aria recordings include raw VRS/MPS plus processed entity-level hand-object supervision	Project
EgoSPT / SPOT	2026-05	arXiv	Spatially prompted egocentric manipulation trajectory prediction from first-frame object/target grounding	Paper
EggHand	2026-05	CVPR 2026 Findings	Multimodal egocentric hand-pose forecasting using video-language and VLA-style action decoding	Project
Map-Mono-Ego	2026-05	ICIP 2026	Map-grounded global human-pose estimation from monocular egocentric video and scanned environments	Paper
EgoForce Hand Pose	2026-05	SIGGRAPH 2026	Monocular egocentric 3D hand pose and shape reconstruction across fisheye, perspective, and wide-FOV camera models	Project
EARL	2026-05	ICML 2026	Analysis-guided RL framework for egocentric interaction reasoning and pixel grounding with coarse-to-fine parsing	GitHub
EgoExo-WM	2026-05	arXiv	Converts exocentric video into egocentric world-model training data using body-pose priors	Project
MotionGRPO	2026-05	ICML 2026	GRPO-based post-training for full-body 3D motion recovery from head-mounted device signals	Paper
ActiveGlasses	2026-04	arXiv	Learns robot manipulation from smart-glasses ego-centric human demonstrations and transfers active vision to a robot perception arm	Paper
GazeVLA	2026-04	arXiv	Pretrains on egocentric gaze, intention, and action signals before robot fine-tuning for manipulation	Project
WARPED	2026-04	arXiv	Wrist-aligned rendering turns monocular egocentric human demonstrations into robot policy observations	Paper
DP-DeGauss	2026-04	ICASSP 2026	Dynamic probabilistic Gaussian decomposition for egocentric 4D scene reconstruction of hands, objects, and background	Paper
Personal Point of View 3DGS	2026-04	CVPR 2026 EgoVis	Evaluation of dynamic 3D Gaussian splatting for egocentric scene reconstruction	Paper
VGGT-Segmentor	2026-04	arXiv	Geometry-enhanced segmentation across egocentric and exocentric views	Paper
Gaze-SoM HOI Anticipation	2026-04	ICPR 2026	Gaze and set-of-mark prompting in VLLMs for hand-object-interaction anticipation from egocentric video	Paper
EgoFlow	2026-04	CVPR 2026	Gradient-guided flow matching for physically plausible 6DoF object-motion generation from egocentric video	Paper
EgoAdapt Speaker Detection	2026-03	arXiv	Robust egocentric Talking-to-Me speaker detection under missing modalities, head-orientation ambiguity, and background noise	Paper
UniDex	2026-03	CVPR 2026	Robot foundation suite for universal dexterous hand control learned from egocentric human videos	Paper
PAWS	2026-03	arXiv	Articulation extraction from large-scale hand-object interactions in egocentric video	Paper
AG-EgoPose	2026-03	arXiv	Attention-guided egocentric 3D human-pose estimation from fisheye camera input	Paper
Static Scene Reconstruction from Dynamic Egocentric Videos	2026-03	arXiv	Mask-aware 3D reconstruction pipeline for long-form dynamic egocentric video	Paper
EgoHOI World Model	2026-03	arXiv	Physics-informed egocentric world model that synthesizes contact-consistent hand-object interactions from action signals alone	Paper
STAformer++ Affordance-Aware Anticipation	2026-02	IEEE TPAMI	Integrates temporal attention, scene affordance memory, and interaction hotspots for short-term object-interaction anticipation on Ego4D and EPIC-KITCHENS	Paper
Neck-Mounted Gaze (GLC)	2026-02	arXiv	Transformer gaze estimator for a shoulder-level neck-mounted camera with out-of-bound classification and multi-view co-learning	Paper
EgoPush	2026-02	arXiv	End-to-end egocentric multi-object rearrangement for mobile robots from a single first-person camera, with zero-shot sim-to-real	Paper
DeltaDorsal	2026-01	CHI 2026	Dorsal hand-skin deformation features for robust egocentric hand pose under self-occlusion (~18% lower joint-angle error)	GitHub
DexWM	2025-12	CVPR 2026	Dexterous interaction world model from finger keypoints in 900+ hours of egocentric human and robot video	Paper
WholeBodyVLA	2025-12	ICLR 2026	Unified latent VLA learning latent actions from action-free egocentric videos for whole-body humanoid loco-manipulation	GitHub
GateFusion	2025-12	WACV 2026	Hierarchical gated cross-modal fusion for active speaker detection, including unconstrained and egocentric ASD settings such as Ego4D-ASD	Paper
EgoSpanLift	2025-11	NeurIPS 2025 spotlight	Lifts 2D egocentric gaze forecasting into 3D visual-span prediction (SLAM keypoints + 3D U-Net) over a 364.6K-sample benchmark	Paper
Uni-Hand	2025-11	T-PAMI 2026	Universal egocentric hand motion forecasting for 2D/3D wrist and finger waypoints with code and data	GitHub
SFHand / EgoHaFL	2025-11	arXiv	Streaming language-guided 3D hand forecasting plus synchronized 3D hand-pose and language-instruction data	Paper
EgoCogNav	2025-11	arXiv	Cognition-aware egocentric navigation forecasting trajectory and head motion from perceived path uncertainty (CEN dataset)	Paper
In-N-On	2025-11	arXiv	Scales egocentric manipulation with 1,000+ hours in-the-wild human video (PHSD) to train the Human0 flow-matching policy	Paper
NS-iHOS / WISH	2025-09	arXiv	Weakly supervised in-hand object segmentation from egocentric narrations, distilled for inference without narration	Paper
INSIGHT	2025-08	AAAI 2026	Intention-guided cognitive reasoning for long-term action anticipation using hand-object cues and verb-noun dependencies	Paper
ECHO	2025-08	arXiv	Recovers human pose, object motion, and contact dynamics from sparse smart-glasses and wrist-tracker signals	Paper
Ego 6DoF Object Trajectories	2025-06	CVPR 2025	Generates 6DoF object manipulation trajectories from action descriptions in egocentric vision, evaluated with HOT3D	Paper
EgoAdapt Efficient Perception	2025-06	arXiv	Adaptive multisensory distillation for efficient action, speaker, and behavior anticipation across EPIC-KITCHENS, EasyCom, and AEA	Paper
EVA02-AT	2025-06	arXiv	Egocentric video-language model family with spatial-temporal rotary positions and symmetric multi-similarity optimization	GitHub
H2R	2025-05	arXiv	Human-to-robot data augmentation that converts egocentric human videos into robot-centric pre-training data	Paper
MEgoHand	2025-05	arXiv	Multimodal egocentric hand-object motion generator (VLM + flow-matching) over 3.35M RGB-D frames, 24K interactions, 1.2K objects	Paper
EgoZero	2025-05	arXiv	Trains robot manipulation policies from Project Aria smart-glasses human demos with zero robot training data	Paper
Ego4o	2025-04	CVPR 2025	Omni-modal egocentric human motion capture from headset/glasses/phone/watch inputs, sparse IMUs, and motion-language descriptions	Paper
EgoH4	2025-04	arXiv	Diffusion transformer forecasting both-hand 3D trajectories and poses from egocentric video, including out-of-frame hands (Ego-Exo4D)	Paper
Helios 2.0	2025-03	ECCV 2026 Event-Based Multimodal Vision Workshop	Synthetic-only event-camera smart-glasses gesture recognition at 6-8 mW, with more than 80% F1 for its six-channel model	Paper
Fish2Mesh	2025-03	ICCV 2025	Fisheye-aware transformer for 3D human mesh recovery from egocentric vision with weak supervision from third-person cameras	Paper
EgoSplat	2025-03	arXiv	Open-vocabulary egocentric scene understanding with language-embedded 3D Gaussian splatting (Aria Digital Twin)	Paper
mmEgoHand	2025-01	arXiv	Head-mounted millimeter-wave radar and IMU system for egocentric hand pose estimation and gesture recognition	Paper
ObjectRelator	2024-11	ICCV 2025 Highlight	Cross-view ego-exo object relation and correspondence model with multimodal fusion and self-supervised alignment	Paper
TouchInsight	2024-10	UIST 2024	Detects touch moment, finger, and location on arbitrary physical surfaces from egocentric hand tracking	Paper
EgoZAR	2024-09	Pattern Recognition Letters	Zone-aware egocentric action-recognition method improving cross-domain transfer with activity-centric zone priors	Project
3D-Aware Ego Instance Tracking	2024-08	ACCV 2024	3D-aware instance segmentation and tracking that lifts 2D masks with scene geometry to survive motion and occlusion (EPIC Fields)	Paper
EgoChoir	2024-05	NeurIPS 2024	Infers 3D contact and object affordance from egocentric video, head motion, and 3D object cues	Project
Diff-IP2D	2024-05	IROS 2025	Non-autoregressive diffusion forecasting of 2D hand trajectories and object affordances with camera-egomotion conditioning	GitHub
Spatial Cognition / LMK	2024-04	arXiv	Tracks active objects out of sight by lifting, matching, and preserving 3D object tracks across long EPIC-KITCHENS videos	Paper
EffHandEgoNet	2024-04	FG 2024	Egocentric 2D hand pose and action-recognition method for smart-glasses RGB input on H2O and FPHA	Paper
EgoLifter	2024-03	ECCV 2024	Open-world 3D segmentation decomposing natural egocentric video into individual 3D objects via 3D Gaussians and SAM	GitHub
X-MIC	2024-03	ECCV 2024	Cross-modal instance conditioning for egocentric action generalization across EPIC-KITCHENS, Ego4D, and EGTEA	GitHub
EgoPoseFormer	2024-03	ECCV 2024	Transformer baseline for stereo egocentric 3D human pose estimation	GitHub
GPT4Ego	2024-01	arXiv	Fine-grained concept-description prompting for zero-shot egocentric action recognition	Paper
Get a Grip	2023-12	arXiv	Reconstructs stable hand-object grasps from egocentric video	Project
AV-CONV	2023-12	CVPR 2024	Audio-visual conversational graph prediction from ego/exo conversation	Project
Aria-NeRF	2023-11	arXiv	Multimodal egocentric view synthesis for Project Aria-style capture	Paper
Egocentric Whole-Body MoCap	2023-11	CVPR 2024	FisheyeViT plus diffusion refinement for egocentric whole-body motion capture	Paper
Object-Centric LTA	2023-10	WACV 2024	Uses object prompts and visual-language pretrained features for long-term action anticipation on Ego4D and EGTEA Gaze+	Paper
Symbolic Active Object Localization	2023-10	EMNLP 2023	Uses symbolic world knowledge to localize active objects from egocentric vision and task instructions	Paper
EgoPCA	2023-09	ICCV 2023	Framework for egocentric hand-object interaction understanding using hand/object-centric cues	Paper
Ego3DPose	2023-09	SIGGRAPH Asia 2023	Captures 3D body cues from binocular egocentric views for pose estimation	Paper
Open-Vocabulary Egocentric Actions	2023-08	NeurIPS 2023	Open-vocabulary egocentric action recognition for verb-noun action labels	Project
Helping Hands	2023-08	ICCV 2023	Object-aware egocentric video-recognition model using hand-object cues	Paper
EgoPoser	2023-08	ECCV 2024	Real-time egocentric pose estimation from sparse and intermittent HMD/controller observations	Project
EgoHandTrajPred / USST	2023-07	ICCV 2023	Egocentric 3D hand trajectory forecasting over H2O and EgoPAT3D-style settings	Project
AntGPT	2023-07	ICLR 2024	Uses large language models to improve long-term action anticipation from video context	Paper
AV-EgoGaze Anticipation	2023-05	ECCV 2024	Audio-visual model for forecasting future gaze in egocentric video	Paper
StillFast	2023-04	CVPRW 2023	End-to-end short-term object interaction anticipation for Ego4D	Project
EgoViT	2023-03	arXiv	Pyramid video transformer with dynamic class-token generation for egocentric action recognition	Paper
Next Active Object Anticipation	2023-02	IEEE Access 2024	Anticipates next-active-object locations before contact in egocentric video	Paper
TransFusion	2023-01	IEEE TCSVT 2024	Summarizes past egocentric context in language to improve multimodal object-interaction anticipation	Paper
Ego-Only	2023-01	ICCV 2023	Egocentric action detection without exocentric pretraining transfer	Paper
EgoSTARK	2023-01	NeurIPS 2023	Adapted long-term tracker baseline for EgoTracks	Paper
EgoLoc	2022-12	ICCV 2023	Stronger Ego4D visual-query 3D object localization with camera-pose and localization baselines	Paper
CONE	2022-11	ECCV 2022 Workshop	Coarse-to-fine alignment framework for Ego4D Natural Language Queries	Paper
InternVideo-Ego4D	2022-11	Ego4D Workshop 2022	Pack of champion Ego4D challenge solutions spanning episodic memory, forecasting, hand-object, and audio/social tracks	Paper
UmeTrack	2022-10	SIGGRAPH Asia 2022	Unified multi-view end-to-end 3D hand tracking for VR headsets and hand-interaction systems	Paper
EgoHOS model	2022-08	GitHub	Context-aware hand-object segmentation and augmentation pipeline	GitHub
I-CVAE LTA	2022-07	WACV 2023	Intention-conditioned variational model for long-term human egocentric action forecasting	Project
SOS Handled Objects	2022-04	ECCV 2022	Self-supervised learning over handled-object sets for egocentric action recognition	Paper
HOI Forecast / Interaction Hotspots	2022-04	CVPR 2022	Forecasts future hand trajectories and interaction hotspots on next-active objects	Project
EgoGAN	2022-03	ECCV 2022	Generative future hand-mask forecasting from egocentric video	Paper
Untrimmed Action Anticipation	2022-02	ICIAP 2022	Reframes egocentric anticipation for untrimmed first-person streams	Paper
OWL	2022-02	CVPRW 2023	Audiovisual temporal context model for egocentric action localization on EPIC-KITCHENS and HOMAGE	Paper
Hand-Object Interaction Reasoning	2022-01	AVSS 2022	Transformer interaction unit for reasoning over acting hands, the other hand, and interacted objects in egocentric video	Paper
E2(GO)MOTION	2021-12	CVPR 2022	Motion-augmented event-stream representation for egocentric action recognition	Paper
Human Hands as Probes	2021-12	CVPR 2022	Uses hand observations in EPIC-KITCHENS to learn interactive object states, interaction regions, and grasp affordances	Paper
Temporal-Context Ego Action Recognition	2021-11	BMVC 2021	Multimodal transformer that uses surrounding temporal context for egocentric action recognition	Paper
NeuralDiff	2021-10	3DV 2021	Neural rendering factorization for segmenting moving 3D objects from dynamic egocentric video	Paper
Head-Motion SSL	2021-10	ICCV 2021 EPIC Workshop	Self-supervised representation learning by matching egocentric video clips with AR/VR head-motion IMU signals	Paper
Exo-to-Ego Video Synthesis	2021-07	ACM MM 2021	Cross-view synthesis model that generates egocentric video from exocentric video	Paper
KinPoly / Dynamics-Regulated Kinematic Policy	2021-06	NeurIPS 2021	Object-aware kinematic policy for egocentric pose estimation with scene dynamics	Project
RNA Cross-Domain FPV AV Recognition	2021-06	arXiv	Relative Norm Alignment for cross-domain first-person audio-visual action recognition	Paper
Ego-Topo 3D Map Localization	2021-05	ECCV 2022	Joint egocentric activity recognition and localization on a 3D environment map	Paper
Ego-Exo representation transfer	2021-04	CVPR 2021	Distillation from third-person video using ego-specific latent signals	Paper
Egocentric Pose from Human Vision Span	2021-04	ICCV 2021	Egocentric body-pose estimation from a wider glasses-like human vision span	Paper
HPS	2021-03	CVPR 2021	3D human pose and self-localization in large scenes from body-mounted sensors and a head-mounted camera	Paper
Indoor Future Person Localization	2021-03	IROS 2021	Future person-location and trajectory prediction from egocentric wearable-camera video	Paper
Contact Representations for Action Forecasting	2021-02	T-PAMI 2021	Forecasts first-person actions through making/breaking hand-object contact representations	Paper
Imagination-Based Ego Action Anticipation	2021-01	IEEE TIP 2021	Anticipates egocentric actions by imagining future visual representations	Paper
RHOI	2020-12	ICCV 2021	Reconstructs hand-object interactions in the wild without direct 3D supervision	Project
Egocentric 4D Human Body Capture	2020-11	3DV 2021	Reconstructs second-person 3D human meshes from monocular egocentric video and scene grounding	Paper
SelfPose	2020-11	T-PAMI 2020	3D egocentric body-pose estimation from downward-looking headset-mounted fisheye cameras	Paper
Ego-OMG	2020-06	arXiv	Object Manipulation Graph representation for activity modeling and near-future action anticipation	Paper
In the Eye of the Beholder	2020-05	IEEE TPAMI	Studies how gaze and wearer attention support action understanding in first-person video	Paper
Unifying Few- and Zero-Shot Ego Action Recognition	2020-05	CVPR 2020 EPIC Workshop	Compositional few- and zero-shot egocentric action recognition for EPIC-style verb-noun labels	Paper
First-Person Motion-Appearance SSL	2020-02	ICPR 2020	Self-supervised joint motion and appearance encoding for first-person action recognition	Paper
Symbiotic Attention	2020-02	AAAI 2020 Oral	Uses privileged hand/object information through symbiotic attention for egocentric action recognition	Paper
EGO-TOPO Environment Affordances	2020-01	CVPR 2020	Builds topological maps of interaction opportunities from egocentric video	Paper
Hands in Egocentric Vision Survey	2019-12	IEEE TPAMI	Focused survey of hand localization, interpretation, applications, and major hand-annotated egocentric datasets	Paper
Motor Attention HOI Forecasting	2019-11	arXiv	Joint prediction of motor attention and future actions in first-person human-object interaction	Paper
Seeing and Hearing Egocentric Actions	2019-10	ICCV 2019 EPIC Workshop	Audio-visual study and baseline for egocentric human-object action understanding	Paper
EPIC-Fusion	2019-08	ICCV 2019	Audio-visual temporal-binding model for egocentric action recognition	Paper
Few-Shot First-Person Action Recognition	2019-07	T-PAMI	Domain-specific priors and meta-learning for few-shot first-person action recognition	Paper
Rolling-Unrolling LSTMs	2019-05	ICCV 2019 Oral	Multi-scale LSTM with modality attention for anticipating egocentric actions and interacted objects	Paper
EPIC-KITCHENS action models	2019-01	GitHub	Public baseline models for EPIC-KITCHENS action recognition	GitHub
Visual Context Event Segmentation	2018-08	ACM MM 2018	Unsupervised segmentation of egocentric photostreams into contextual events, with EDUB-Seg labels	Paper
Task-Dependent Gaze Transition	2018-03	ECCV 2018	Learns task-dependent attention transitions for gaze prediction in egocentric video	Paper
Future Person Localization	2017-11	CVPR 2018	Predicts where people will appear in future frames from a first-person wearable camera	Paper
First-Person Activity Forecasting	2016-12	ICCV 2017 Oral	DARKO online inverse-reinforcement-learning system for forecasting actions and active objects	Paper
Unsupervised Important Objects	2016-11	ICCV 2017	Detects important first-person objects without wearer labels using cross-pathway visual/spatial supervision	Paper
Ego-Surfing	2016-06	IEEE TPAMI	Localizes a person in first-person videos using ego-motion signature correlation across wearable-camera views	Paper
First-Person Pose Recognition	2015-06	CVPR 2015	Uses egocentric workspaces and camera geometry for first-person human pose recognition	Paper
Egocentric FOV Localization	2015-01	WACV 2015	Localizes the camera wearer's first-person field of view in overhead/surveillance video	Paper

Practical Tooling

Tool	Released	Venue	Use
Ego-OSCAR	2026-08	arXiv	Sub-$200 stereo-inertial head-mounted capture design with recording, synchronization, and watchdog software; the paper also claims a 550-hour-per-camera annotated corpus, but no artifact URL is currently linked.
LightMem-Ego	2026-07	arXiv	MIT-licensed streaming visual-audio memory for smartphones and Rokid AI glasses, with hierarchical retrieval for object finding, conversation recall, summaries, routines, and evidence-grounded answers.
OpenGlass Visual Assistance	2026-07	ACL 2026 System Demonstrations	Local-first ESP32 smart-glasses sensing with nearby-device MLLM inference, speech output, and public evaluation assets.
OpenGlass Event Eyewear	2026-06	arXiv	Low-power open AI eyewear platform with event-based vision support for on-device smart-glasses research.
EgoKit	2026-05	arXiv	Low-cost synchronized ego/wrist recording workflow across phones, smart glasses, and XR hosts.
GBAT	2026-05	arXiv	Annotating egocentric eye-tracking and video in child-caregiver interaction studies.
MobileEgo Anywhere	2026-05	arXiv	Commodity-phone STERA tooling plus a gated 200-hour, 584-session release with 10M frames, pose, IMU, MANO hands, language, and about 1.6 TB of data.
VisionClaw	2026-04	arXiv	Always-on wearable AI-agent system on Ray-Ban Meta smart glasses, coupling live egocentric perception with speech-driven task execution.
HOMIE-toolkit	2026-03	GitHub	Loading and visualizing Xperience-10M HDF5 annotations, calibration, SLAM, hand/body mocap, depth, IMU, and point clouds.
RaycastGrasp	2025-10	ICIR 2025	Egocentric gaze-guided MR headset interface for robotic object retrieval and manipulation.
AiGet	2025-01	CHI 2025	Smart-glasses assistant for gaze/context/profile-driven informal learning during everyday moments.
HUX	2024-07	arXiv	Always-on smart-glasses / XR companion concept with gaze, environment context, verbal context, and memory storage.
HOT3D tooling	2024-06	project page	Loading HOT3D hand/object/camera pose annotations and models.
Ego-Exo4D CLI and docs	2023-11	project page	Downloading synchronized ego-exo data and annotations.
EgoObjects API	2023-09	ICCV 2023	Working with category and instance-level egocentric object labels.
EgoBlur	2023-08	arXiv	Privacy-preserving blur pipeline and responsible-innovation analysis for Project Aria egocentric capture.
Project Aria Tools	2023-08	arXiv	Reading Aria VRS, calibration, MPS, trajectory, gaze, and dataset artifacts.
VISOR API	2022-09	GitHub	Loading dense EPIC-KITCHENS hand/object masks and relations.
HOI4D tooling	2022-03	project page	Loading RGB-D frames, point clouds, object meshes, and pose/segmentation annotations.
Ego4D CLI and docs	2021-10	project page	Downloading and working with Ego4D data after license approval.
PAL	2021-05	CVPR 2021 EPIC Workshop	Wearable personalized visual-context detection for privacy-preserving intelligence augmentation.

Adjacent and Related Resources

These resources sit outside the atlas's human-worn egocentric core, but provide useful comparisons and transfer paths: robot-mounted or simulated first-person views, general human video, multi-view robotics, autonomous driving, and long-video reasoning. They are tagged scope: adjacent in `data/resources.yml` so public counts and filters keep the distinction explicit.

Resource	Released	Venue	Scale / signal	Why it is adjacent (not egocentric)	Status
AlloEgo-VLM	2026-08	arXiv	Public AlloEgo-View triplets and SFT code disambiguate allocentric versus egocentric spatial references, including an Isaac Sim object-search test	General images and simulation rather than wearable capture; repository-wide license is absent	partial
WorldRover-10M	2026-08	arXiv	Synthetic minute-scale exploration from first-, third-, and 360-degree views with RGB, depth, poses, actions, flow, and long-range tracks	Artist-built synthetic worlds; code and data links are not yet live	watch
EATR-Stereo	2026-08	arXiv	Embodiment-conditioned routing of paired head-stereo evidence reports 60% full-task and 100% grasp success on a 33-DoF humanoid	Robot head cameras rather than human wearable video	watch
Hydra-0	2026-08	arXiv	Action-flow world model trained after filtering 2,202 hours of multi-embodiment video, supporting replay evaluation and human-demo transfer	General robot/human video; code and models are still coming soon	watch
DECOWAM	2026-08	arXiv	Separates camera ego-motion, base, and arm action in a whole-body world-action model and introduces the ARMDOG real-robot dataset	Moving robot-camera perception; artifacts are not yet released	watch
RoboEdit	2026-08	arXiv	Turns human manipulation video into aligned robot video and 3D states; reports RoboEdit-14M with 174K pairs across seven embodiments	Human video need not be wearable first-person; data and code remain unreleased	watch
HiPHI	2026-08	Hugging Face	Gated 617.5-hour optical-motion corpus with 200.1M frames, 132 performers, and 245.7 hours of hand-object interaction	Full-body and HOI mocap without a central first-person camera	request
NVIDIA Video-to-Data Robot Dexterity Dataset	2026-08	Hugging Face	1,215 TACO-derived motions retargeted into 121,500 bimanual robot episodes totaling 924.8 GB, with CC-BY-4.0 data and public tools	Synthetic robot executions derived from existing human demonstrations	open
RoboLab-EgoX	2026-08	Hugging Face	4,000 simulation episodes over 28 tasks and 99 scenes, including success/failure labels, wrist RGB-D, exterior RGB-D, telemetry, and actions	Robot wrist-view simulation benchmark rather than human wearable capture	open
OccPlanner	2026-08	arXiv	Occupancy-conditioned pixel-goal diffusion planner with L3ROcc supervision from monocular robot videos; reports 71.55% average success at 5-8 m versus 20.81% for NavDP	Robot-view navigation rather than human wearable capture	watch
AdvDex	2026-08	arXiv	Aligns human and robot dexterous demonstrations through an SE(3) wrist plus 15-joint hand action space, tactile/kinematic OmniShare data, and adversarial embodiment learning	OmniShare is not established as a wearable first-person corpus	watch
ContactGuard	2026-08	arXiv	Predicts action-conditioned visual latents and aborts likely manipulation failures before contact from wrist and multi-view robot cameras	Robot execution monitoring rather than human wearable capture	watch
HumanoidVLN	2026-08	arXiv	Isaac Sim benchmark with four humanoids, 933 collision-aware routes, four instructions per route, and a 20-episode Unitree G1 sim-to-real pilot	Simulated humanoid first-person observations rather than human wearable capture	watch
Proxemic Risk from Robot Ego Images	2026-08	ECCV 2026 EMR Workshop	Four-level danger classification study of InternVL, Qwen-VL, and SmolVLM with prompting, QLoRA, and person-localization analysis	Robot viewpoint and safety labels rather than human wearable capture	watch
Fast-Slow ReAct ObjectNav	2026-08	arXiv	Bounded VLM deliberation over coordinate-anchored semantic memory and pose-tagged first-person keyframes; reports 68.75% HM3D and 47.29% MP3D success	Navigation-robot observations rather than human wearable capture	watch
VideoNIG	2026-08	arXiv	60K simulated ego-view tours and 37K multimodal prompts for navigation-instruction generation, spatial diagnosis, and downstream execution	Simulator-generated agent views rather than human wearable capture	watch
WNM-3D	2026-08	arXiv	Conditions joint future-view and action generation on geometry tokens reconstructed from monocular navigation history	Navigation-agent camera history rather than human wearable capture	watch
AtlasVLA	2026-08	arXiv	Wrist-camera VLA with persistent voxel-hashed 4D world memory and task-progress-aware ego working memory	Robot wrist-camera manipulation rather than human wearable capture	watch
CrossTracer	2026-08	arXiv	Cross-embodiment image-plane trace prediction and residual adaptation, evaluated on NaviTrace and wheeled/legged robots	Robot-mounted egocentric sensing rather than human wearable capture	watch
Omega-0	2026-08	arXiv	Whole-body latent world-action model supporting robot ego RGB; reports Omega-HOME with 40+ hours of synchronized household humanoid observations and actions	Robot-mounted egocentric sensing rather than human wearable capture	watch
LAWM-3D	2026-08	arXiv	Learns 3D-aware latent actions from multiview human video for generalizable robot world models	Source human video is not confirmed as wearable first-person capture	watch
HOST	2026-07	arXiv	One-video skill acquisition in 29 seconds on average with 62% reported success; training code and a 15.9 GB policy checkpoint are public	Human video is central but not required to be wearable first-person	partial
Data Pyramid for Embodied Manipulation	2026-07	arXiv	Open survey and living catalog organizing real-robot, UMI, human ego/ego-exo, simulation, and general video data into five levels	Broad embodied-manipulation taxonomy in which egocentric data is one major layer	open
ContactFlow	2026-07	arXiv	Embodiment-agnostic world-model conditioning through trajectories of 3D actor-object contact points, trained with human and robot interaction video	General human/robot video; wearable capture is not required	watch
Robot-Factored World Models	2026-07	arXiv	Factors actions into nominal robot trajectories and rendered geometry across DROID, egocentric RoboCasa-GR1, unseen robots, and retargeted human demos	Robot-camera world model rather than human wearable capture	watch
AWM Navigation Pretraining Corpus	2026-07	Hugging Face	Public 198.1 GB corpus with 23,076 indoor/outdoor robot-view clips and paired Gemini-generated spatial/action labels; source-derived reuse terms are incomplete	Robot navigation observations rather than human wearable capture	partial
Think at 5 Hz, Act at 20 Hz	2026-07	arXiv	Asynchronous driving VLA with a frozen 7B reasoner and fast action expert, raising CARLA route completion from 37.0 to 94.0 at fresh 20 Hz control	Autonomous-driving camera input rather than human wearable capture	watch
IMBench	2026-07	RSS 2026 SemRob Workshop	Open benchmark with 35 tasks, seven categories, and 14K trajectories integrating physical reasoning with executable manipulation	Robot-manipulation benchmark rather than human wearable capture	open
AC-VLA	2026-07	arXiv	Compositional VLA training with instruction decomposition, trajectory alignment, and wrist-view masking; reports about 28% absolute LIBERO-OOD improvement	Robot VLA method rather than a human first-person resource	watch
SkillNav	2026-07	arXiv	Training-free object-goal navigation skills written into a curiosity map, reaching 43.2 SPL and 75.9% success on HM3D v0.2	Robot navigation observations rather than human wearable capture	watch
Orbis 2	2026-07	arXiv	Hierarchical driving world model that separates long-horizon scene structure from detailed generation; code and checkpoints remain coming soon	Autonomous-driving world model rather than human wearable capture	watch
Data and Learning Where it Matters	2026-07	arXiv	Targeted collection and offline RL for contact-critical segments averages 96% success across four real tasks from 2-2.5 hours of data	Robot contact-rich manipulation rather than human wearable capture	watch
MotionForesight	2026-07	arXiv	Forecasts future object-centered 3D scene flow by repurposing frozen video and tracking priors trained on 40K monocular human-object videos	General monocular interaction video without a wearable-capture requirement	watch
VTM-Nav	2026-07	arXiv	Training-free object-goal navigation with persistent visual-topological room/object memory, coarse-to-fine retrieval, and a conservative execution guard	Robot navigation memory rather than human wearable capture	watch
SafeRelBench	2026-07	arXiv	507 household-agent evaluations testing support, containment, and proximity constraints before risk-prone actions	Robot-agent spatial-safety evaluation rather than human wearable capture	watch
SoftNav	2026-07	IROS 2026	Continuous 3D soft tokens ground a frozen VLM in detected objects and frontiers using about 1,200 samples and 17M trainable parameters	Robot object-goal navigation rather than human wearable capture	watch
Representation-Aligned Tactile Grounding	2026-07	arXiv	Latent tactile predictor aligns intermediate VLA features with future contact consequences without reconstructing noisy raw tactile signals	Robot tactile-VLA learning rather than human wearable capture	watch
Action QFormer	2026-07	arXiv	Instruction-conditioned query interface reorganizes inherited multimodal features before action generation, with large reported sim-to-real gains	Robot navigation VLA architecture rather than human wearable capture	watch
Reflex	2026-07	ICML 2026	Streaming flow-matching VLA runtime reports 2.58x speedup, stable 50 Hz control, and up to 54% lower reaction latency	Robot VLA inference framework rather than human wearable capture	watch
FoMoVLA	2026-07	arXiv	Future-feature foresight and sparse 2D trajectories jointly supervise target states and motion paths for continuous policies	Robot VLA foresight method rather than human wearable capture	watch
LifelongVLA	2026-07	arXiv	Continual VLA learning separates reusable shared knowledge from task-specific experts while controlling forgetting	Robot continual-learning method rather than human wearable capture	watch
Steering Robustness into World Action Models	2026-07	arXiv	Robustness-oriented WAM training uses uncertainty-aware perturbations and corrective trajectories to improve recovery under distribution shift	Robot world-action-model training rather than human wearable capture	watch
AeroAct	2026-07	arXiv	Vision-language-action policy couples aerial perception, language-conditioned planning, and low-level flight control	Aerial-robot VLA rather than human wearable capture	watch
DriftWorld	2026-07	arXiv	Driving world model explicitly represents scene dynamics and controllable ego motion for long-horizon generation	Autonomous-driving world modeling rather than human wearable capture	watch
BadWAM	2026-07	arXiv	Benchmark and attack framework studies backdoors in world-action models across visual triggers and action outcomes	Robot WAM security rather than human wearable capture	watch
RoboTTT	2026-07	arXiv	One-shot human-video imitation combines online test-time training with world-model supervision, reporting five-minute learning of a 10-stage assembly task	Human-video robot adaptation without a released wearable-video corpus	watch
PhysClaw-0	2026-07	arXiv	Open collect-verify-reset system that stores and reuses language corrections, reducing human working time to 16% and raising single-attempt success from 12.5% to 47.5%	Robot autonomous-data-collection toolkit rather than human wearable capture	open
Industrial Dexterity Benchmark	2026-07	arXiv	Industrial hardware boards plus DAG-ROS and AG-iDP3 for cable, harness, and gearbox tasks; best cable setup reaches 78% versus 36% for single-camera RGB	Industrial robot benchmark rather than human wearable capture	watch
GigaWorld-Policy-0.5	2026-07	arXiv	Open action-centered WAM with mixed world/action training, MoT experts, AutoResearch, and 85 ms RTX 4090 inference without future-video decoding	Robot world-action model rather than human wearable video	open
VSI-Super-Wild	2026-07	ECCV 2026	6,980 human-verified spatial QA pairs over 442 continuous panoramic videos totaling 284.52 hours; public 2.67 TB data lacks explicit reuse terms	Panoramic YouTube-derived video rather than wearer capture	partial
REAL / REAL-Bench	2026-07	ECCV 2026	Open mobile-manipulation framework and 241-task benchmark reporting 56.9% interactive and 78.3% physical-robot success, with code and 3.92 GB of data	Robot-only open-world manipulation rather than human wearable capture	open
UESF-Bench / SeekFollow-VLA	2026-07	arXiv	Unified language-guided human seeking and following with semantic exploration, delayed identity grounding, phase switching, and recovery	Robot-agent human search/following rather than human wearable video	watch
JOP-VLN	2026-07	IROS 2026	Joint imitation, DAgger, and reinforcement learning reaches 69.9% success on R2R and 68.0% on RxR	Navigation-agent observations rather than human wearable capture	watch
Reverse to Advance	2026-07	arXiv	Trains difficult manipulation from easier reversed rollouts using automated collection, temporal inversion, kinematic filtering, critic filtering, and iterative refinement	Robot manipulation data collection rather than human first-person data	watch
Anchor-Align	2026-07	arXiv	Representation anchoring plus language-action alignment improves physical xArm success from 28% to 54% and 37% to 60%; linked code currently returns 404	Robot-camera VLA finetuning rather than human wearable capture	watch
JITOMA / JITOMA-Bench	2026-07	arXiv	Just-in-time scene-graph growth activates task-relevant memory anchors to curb long-horizon perceptual saturation, graph size, and caption latency	Robot scene-graph memory rather than human wearable video	watch
WANDA / Worlds in One Demo	2026-07	arXiv	Public 16,922-episode, roughly 248-hour synthetic mobile-manipulation set expands one real demonstration per task across generated 3D worlds	Synthetic robot trajectories rather than human wearable capture	open
GPUSimBench	2026-07	IROS 2026	Tests Isaac Lab and Genesis for real-to-sim physical consistency, parallel throughput and memory, plus run-to-run and inter-environment nondeterminism	Robot-simulator infrastructure benchmark without human capture	watch
HRIBench (Interaction-Centric VLA)	2026-07	arXiv	13 role-conditioned tasks and 650+ episodes assess intent, synchronization, responsiveness, protocol, and safety across instructor, collaborator, and intruder roles	Robot-view collaboration benchmark rather than human wearable capture	watch
DenseReward	2026-07	arXiv	Dense frame-level vision-language rewards trained with automatically synthesized physical failures for manipulation, MPC, and RL; the claimed release page currently returns 404	Robot-camera reward learning rather than human wearable capture	watch
FlowWAM	2026-07	arXiv	Open optical-flow world-action model reporting 92.94%/92.14% RoboTwin Clean/Random success and a 63.71 WorldArena EWMScore, with code, checkpoints, and flow data	Robot-only manipulation and world modeling rather than human wearable capture	open
ChunkFlow	2026-07	arXiv	Continuity-consistent chunked-policy learning with editable overlap zones, seam and derivative losses, robust history training, and AWAC adaptation	Robot-policy stability method without a human first-person resource	watch
ExToken	2026-07	arXiv	Reinforcement fine-tuning that conditions VLA exploration on demonstration-derived behavioral tokens and learns a deployment-time token selector	Robot VLA exploration method rather than egocentric human data	watch
Hy-Embodied-VLM-1.0	2026-07	arXiv	Open approximately 30B MoE embodied VLM activating about 3B parameters per token; evaluated on 38 benchmarks with first-place results on 19 among compared similarly sized models	General physical-world agent model rather than a human wearable resource	open
Jetson-PI	2026-07	arXiv	Open foresight-aligned asynchronous VLA stack reporting 8.66x higher control frequency than naive PyTorch, 5.41x over vla.cpp, and +14.8% average success over VLASH	Onboard robot-control runtime rather than human first-person capture	open
TrustVLA	2026-07	arXiv	Retraining-free VLA backdoor defense using evidence-evolution monitoring, counterfactual trigger localization, and localized inpainting	Robot VLA security method rather than an egocentric data resource	watch
Self-in-Space / SIS-Bench	2026-07	ACM MM 2026	Open 4,856-question benchmark over 1,646 UAV videos and 13 tasks, plus 54,298 training examples, code, and a motion-aware adapter	Agent-centered UAV onboard video rather than human wearable capture	open
VistaVLA	2026-07	arXiv	3D-Gaussian-grounded VLA reporting 99% context-token reduction, +22.8% success across seven real tasks, and +30.0% OOD over VLA-Adapter	Multi-view robot manipulation rather than human wearable video	watch
Efficient VLA Inference via Temporal Redundancy Reduction	2026-07	arXiv	Dynamic visual-token updates plus two-step diffusion action sampling report more than 2x speedup and up to 98% success across LIBERO, RoboTwin, and real robots	Robot VLA inference optimization rather than a first-person resource	watch
REGRIND	2026-07	arXiv	Single-demonstration retargeting plus residual RL for scissors and screwdriver use on LEAP and WUJI hands; linked code repository is currently unavailable	Human hand-object references and robot control, but no wearable-video release	watch
E-VQA / ST-Evidence	2026-07	ECCV 2026	Open benchmark for answers with temporal segments and tracked object masklets, plus 160K instruction examples and released 3B/7B models	General video QA and evidence grounding rather than egocentric video	open
Xiaomi-Robotics-U0	2026-07	arXiv	38B embodied-synthesis model for multi-view scenes, embodiment transfer, editing, and video; the claimed release URL currently returns 404	Robot and embodied-scene generation rather than human wearable capture	watch
World Action Models to Embodied Brains Roadmap	2026-07	arXiv	Roadmap spanning WAM representations, standardization, physical harnesses, shared contracts, and closed-loop post-training	Broad physical-intelligence position paper rather than an egocentric resource	watch
DA-Nav / ReDA	2026-07	arXiv	Direction-aware city-scale VLN and recovery data, reporting 56.16% unseen-city success and kilometer-scale robot trials	Robot-agent egocentric navigation, not human wearable capture	watch
Robot-Centric Pointmaps	2026-07	arXiv	Robot-frame per-pixel XYZ improves pi0.5 by 7.6 points and SmolVLA by 4.2 on RoboCasa, with larger gains under unseen real-camera viewpoints	Robot-camera VLA representation rather than human first-person data	watch
TeleDexter	2026-07	arXiv	Hand-object co-tracking controller reporting 75.2% average success across seven tasks and two dexterous hands, with demonstrations used for behavior cloning	Robot teleoperation from human references, not a released wearable-video resource	watch
WALA	2026-07	arXiv	Executable latent actions from labeled demonstrations and action-free videos using DINOv3 feature deltas, depth, and latent world modeling	General robot-policy learning; code is still marked coming soon and no egocentric corpus is released	watch
SLVMBench	2026-07	arXiv	2,261 questions over 1,220 videos test learning a tutorial embedded in a 2-3-hour stream and applying it to an ongoing task	General long-video skill memory rather than first-person-centered data	watch
Lumo-2	2026-07	arXiv	Latent WAM with staged action-dynamics-vision-language alignment for scalable, OOD, long-horizon, and dexterous robot learning	Robot-centered WAM/VLA method without a wearable-video release	watch
AdvNav	2026-07	ACM MM 2026	Black-box adversarial attack on first-person VLN observations with 49.70%-87.30% reported attack success against HAMT and MapGPT	Navigation-agent views rather than human wearable video	watch
yyyyywv/egocentric	2026-07	Hugging Face	32.8 GB, 1,019 robot episodes, and 994,459 frames across seven task groups from eye/head and wrist cameras; downloadable but the card omits provenance and capture documentation	Robot-mounted cameras and action/state streams rather than human wearable capture	partial
TactiDex	2026-07	arXiv	Human-demonstration benchmark aligning whole-hand tactile pressure, hand kinematics, object 6D states, language, and task phases, plus tactile-guided single/bimanual robot transfer	Tactile-glove and motion-capture demonstrations, not human wearable video	watch
DemoBridge	2026-07	RSS 2026 RoboData Workshop	Apache-2.0 toolkit that converts single-view stereo hand demonstrations into collision-aware, physics-validated robot trajectories	Human-hand retargeting without explicitly egocentric capture	open
PhysV2A	2026-07	arXiv	Converts video-derived 6D object motion into feasible robot trajectories using grasp-conditioned reachability checks and semantic-mask-constrained refinement	Generic video-to-robot transfer rather than human wearable capture	watch
ACE-Brain-0.5	2026-07	arXiv	Public Qwen3-VL-derived 8B-class checkpoint unifying spatial perception, planning, robot action generation, progress monitoring, and self-improvement across 15 benchmarks; no model license is declared	Robot-centric embodied foundation model with egocentric spatial reasoning, not human wearable capture	partial
DexVerse	2026-07	arXiv	100-task dexterous-manipulation benchmark across three arms and six hands, with VR teleoperation and 3,180 reported demonstrations; core BSD-3-Clause suite is live while full data/tooling remain pending	Robot simulation and teleoperation benchmark, not a human wearable-video corpus	partial
FabriVLA	2026-07	arXiv	Compact 1B VLA with InternVL3.5 and a flow-matching action head, reporting 90.0% tier-average success on Meta-World MT50	Simulation-focused robot VLA, not human egocentric capture	watch
Harness VLA	2026-07	arXiv	Memory-guided agentic harness composing a frozen VLA with retryable contact primitives and fixed analytic primitives across LIBERO-Pro, RoboCasa365, and RoboTwin C2R	Robot VLA planning harness, not a first-person resource	watch
AnyDexRT	2026-07	arXiv	Calibration-free hand retargeting with self-supervised fingertip correspondence, few-shot human guidance, and contact-aware pinch refinement	Human-hand teleoperation method, not egocentric video data	watch
TFP	2026-07	RSS 2026 SemRob Workshop	Event-sensitive memory-action fusion for a 3.3B VLA, evaluated on LIBERO, LIBERO-plus, and MIKASA ShellGameTouch	Robot VLA memory method without wearable capture	watch
LingBot-VA 2.0	2026-07	arXiv	Native video-action model with a semantic tokenizer, causal pretraining, sparse MoE backbone, and asynchronous closed-loop inference	Robot video-action foundation model, not human first-person data	watch
LEEVLA	2026-07	arXiv	Task-aware VLA using drift-guided visual prioritization and structured latent feature-flow generation; the linked repository does not yet match the paper	Robot VLA architecture without human wearable capture	watch
Video-Action Generalization Gap / Temporal Ratio	2026-07	arXiv	Diagnostic Temporal Ratio and adaptive inference guidance for compositional generalization in video-action models on LIBERO and real robot tasks	Robot video-action analysis, not a human first-person resource	watch
GIRAF	2026-07	CVPR 2026 HuMoGen Workshop	Text-conditioned full-body interaction generation with articulated objects, object-centric contact representations, and contact augmentation	Generated third-person human-object motion, not wearable capture	watch
LingBot-World 2.0	2026-07	arXiv	Interactive world simulator with unbounded interaction horizon, 720p/60 FPS real-time distillation, richer action/event controls, and 14B/1.3B models	General interactive world simulator, not human first-person capture	watch
RoboDojo	2026-07	arXiv	Unified sim-and-real robot manipulation benchmark with 42 simulation tasks, 18 real tasks, remote evaluation, 30 integrated policies, and a leaderboard	Robot-policy benchmark, not human wearable capture	watch
LaMem-VLA	2026-07	arXiv	Latent-memory-native VLA framework that reconstructs short- and long-term history into latent memory tokens for long-horizon manipulation	Robot VLA memory architecture, not a human first-person resource	watch
TouchWorld	2026-07	arXiv	Predictive-and-reactive tactile foundation model separating vision-language planning, tactile world prediction, action generation, and residual refinement	Robot tactile manipulation method, not wearable capture	watch
WAM-TTT	2026-07	arXiv	Test-time training framework steering frozen WAMs from unlabeled human videos through adaptive memory and paired human-robot meta-training	Human-video-to-robot WAM method, not a human wearable dataset	watch
EAGOR	2026-07	arXiv	Training-free spherical-belief framework for 360-degree directional reasoning on HOS, OSR-Bench, and legged-robot navigation	Embodied 360-camera robot/agent reasoning, not human wearable capture	watch
VLA Models Review	2026-07	arXiv	Survey of 183 VLA contributions from 2017-2026 across architectures, training recipes, bimanual coordination, UAVs, memory, and world models	Robot/UAV VLA survey, not an egocentric resource	watch
NativeMEM	2026-07	arXiv	VLA memory compression that turns each historical camera frame into a single memory token for low-latency long-horizon manipulation	Robot VLA memory method, not human wearable capture	watch
Lift3D-VLA	2026-07	arXiv	3D geometry- and dynamics-aware VLA with point-cloud reasoning, geometry-centric masked autoencoding, and 22 sim / 8 real tasks	Robot-only 3D VLA method, not human first-person data	watch
Pelican-VLA 0.5	2026-07	arXiv	Unified VLA report combining vision-language understanding, future-frame generation, action prediction, and reasoning-slot attention	Robot VLA architecture, not human wearable capture	watch
ActionCache	2026-07	arXiv	Training-free cache/refinement method accelerating flow-matching VLA action heads by reusing intermediate actions with compact multimodal keys	Robot VLA inference method, not a first-person resource	watch
Smooth Operator / SBR	2026-07	arXiv	Sampling-based hand retargeter for low-jitter real-time teleoperation, evaluated in simulation and an 18-participant manipulation study	Teleoperation/retargeting method, not a first-person dataset	watch
LongVQUBench	2026-07	ECCV 2026	1,200+ long videos and 1,500 questions for video-quality reasoning, spanning movies, documentaries, surveillance, egocentric recordings, and animation	Broad long-video quality benchmark with egocentric recordings as one slice, not centered on wearable capture	watch
TAP VLA	2026-07	ICML 2026	Task-agnostic VLA pretraining learns motor priors from unlabeled robot interaction data before language grounding	Robot/VLA pretraining, not human wearable capture	watch
The Moving Eye	2026-07	IROS 2026	Hybrid dynamic data collection with one robot arm acting as a moving environmental camera while the other manipulates	Robot camera/viewpoint generalization, not human first-person capture	watch
Bridge-WA	2026-07	arXiv	World-action framework distilling future-change tokens, change maps, and motion-flow maps into a VLA action transformer	Robot world-action model, not wearable capture	watch
VLA-Corrector	2026-07	arXiv	Lightweight detect-and-correct inference for action-chunked VLA policies using visual feature evolution	Robot VLA correction method, not human wearable capture	watch
Embodied.cpp	2026-07	arXiv	Portable C++ runtime for VLA/WAM inference on heterogeneous robots with adapters, head plugins, and robot deployment layers	Embodied robot runtime, not a human first-person resource	watch
EVA-Client	2026-07	arXiv	Unified real-robot client for policy deployment, data collection, evaluation, rollout logging, and comparison views	Robot policy client and evaluator, not wearable capture	watch
InternVLA-A1.5	2026-07	arXiv	Robot VLA with native VLM backbone, continuous action expert, subtask/VQA supervision, and latent foresight tokens	Robot-only VLA architecture, not human first-person data	watch
SIEVE	2026-07	arXiv	Structure-aware selection of VLA demonstrations by reusable primitives, composition patterns, and medoid trajectories	Robot demonstration-selection method, not egocentric capture	watch
SVA / Look Before You Leap	2026-07	arXiv	Distills tree-search rollouts into an action evaluator for frozen VLA policies at test time	Robot VLA action-evaluation method, not wearable capture	watch
WorldOdysseyBench	2026-06	arXiv	600+ first/third-person interactive world-model test cases with 10-60s WASD interaction and action, vision, physics, and memory metrics	General interactive world-model benchmark, not human wearable capture	watch
Qwen-RobotNav	2026-06	arXiv	Qwen-branded navigation model trained on 15.6M samples with task modes and controllable visual-history observation parameters	Robot/agent navigation model, not human wearable capture	watch
Qwen-RobotWorld	2026-06	arXiv	Language-conditioned embodied video world model with an 8.6M video-text EWK corpus over 20+ embodiments and 500+ action categories	Robot, navigation, driving, and human-to-robot world modeling rather than human first-person capture	watch
FutureNav	2026-06	arXiv	Unified world-action modeling for vision-and-language navigation with action, dynamics, and future spatial-state objectives	Navigation-agent WAM, not human wearable capture	watch
ZR-0	2026-06	arXiv	2.6B VLA trained with dense embodied chain-of-thought supervision over ProcCorpus-60M for cross-embodiment manipulation	Robot VLA pretraining, not a human first-person dataset	watch
VLK	2026-06	arXiv	48K synthetic humanoid loco-manipulation trajectories with rendered egocentric observations, instructions, and whole-body kinematics	Synthetic humanoid supervision, not real human wearable capture	watch
HUMEMBR	2026-06	IROS 2026	Human-centered long-term memory for embodied robot QA and routine-conditioned navigation	Robot memory/navigation system, not an egocentric capture resource	watch
RhinoVLA	2026-06	arXiv	Edge-deployable VLA using a token-efficient Qwen3-VL backbone, Action Expert, View Registry, and hardware-aware execution	Robot-only VLA deployment system, not wearable capture	watch
OpenEAI-Platform	2026-06	arXiv	OpenEAI-Arm plus OpenEAI-VLA built on Qwen3-VL-4B and a Diffusion Transformer action head	Robot-arm hardware/software platform, not human first-person data	watch
SAGE-Nav	2026-06	arXiv	LLM-planned object-goal navigation with dynamic scene graphs and egocentric robot observations	Robot object-navigation method, not human wearable capture	watch
USS	2026-06	arXiv	Unified spatial-semantic prompting for embodied visual tracking with text, point, box, and mask prompts	Robot/agent tracking method, not human first-person capture	watch
RoboAtlas	2026-06	arXiv	Contextual Active SLAM with OpenRoboVox semantic mapping and egocentric VLM reasoning	Robot active-SLAM/navigation system, not human wearable capture	watch
iCrowdNav	2026-06	arXiv	Intention-aware crowd navigation from egocentric robot observations and human-pose interaction encoding	Robot crowd-navigation method, not human wearable capture	watch
Cloak	2026-06	arXiv	Masks the robot end-effector from wrist-camera VLA observations for zero-shot cross-embodiment transfer	Wrist-camera robot VLA method, not human first-person capture	watch
OpenHLM	2026-06	arXiv	Whole-body humanoid loco-manipulation recipe with teleoperation, VLA design, and HuMI co-training	Robot-centered humanoid VLA recipe, not human wearable capture	watch
UniviewVLA	2026-06	arXiv	Multiview VLA with world modeling for occlusion-aware robot manipulation from agent and wrist cameras	Robot VLA/world-modeling method, not human first-person capture	watch
ContactWorld	2026-06	arXiv	Vision-tactile world-model benchmark over 12 contact-rich manipulation tasks with wrist/front views, point clouds, and tactile force fields	Robot contact-rich manipulation benchmark, not human wearable capture	watch
3DG-VLN / UAV-VLN-FOV	2026-06	arXiv	2,717 target-visible UAV trajectories with high-resolution egocentric observations and 3D waypoint labels	Aerial-agent navigation benchmark, not human wearable capture	watch
NavWAM	2026-06	arXiv	Navigation world-action model predicting future egocentric visual observations and action sequences for goal-conditioned visual navigation	Robot visual-navigation world model, not human wearable capture	watch
IntentNav	2026-06	arXiv	Spatial-visual object-navigation policy learned from human demonstrations with frontier probing and spatial memory	Object-navigation work from human demonstrations, not a human wearable resource	watch
AlloSpatial	2026-06	arXiv	Agentic spatial-reasoning harness that converts local egocentric observations into allocentric spatial representations	General spatial-reasoning framework, not a wearable capture dataset	watch
World-Language-Action Model	2026-06	arXiv	Unified model predicting subtasks, subgoal images, and robot actions while learning from egocentric videos and robot trajectories	Broader embodied model using egocentric video among robot data, not centered on human wearable capture	watch
ManiSplat	2026-06	arXiv	Manipulation trajectory synthesis from monocular video using decoupled 3D Gaussian Splatting	Monocular manipulation synthesis; first-person human capture not confirmed	watch
DexFuture	2026-06	arXiv	Hierarchical future-state visuomotor targeting for bimanual dexterous tool use	Robot dexterity method, not human first-person data	watch
GRAIL	2026-06	arXiv	Digital-twin pipeline generating humanoid loco-manipulation demonstrations from 3D assets and video priors	Humanoid demonstration generation, not a human wearable dataset	watch
DataLadder	2026-06	arXiv	Simulation-enabled interconversion toolchain for robot trials, demonstrations, synthetic data, and evaluation assets	Embodied-data tooling rather than egocentric capture	watch
EgoInfinity	2026-06	arXiv	Public 46 GB preview with 104 Action100M clips and retargeted trajectories for four robot embodiments; the full web-scale engine is not released	Human-video-to-robot data engine, not centered on wearable first-person capture	partial
LUCID	2026-06	arXiv	Embodiment-agnostic intent model learned from unstructured human videos, paired with simulation-trained dexterous robot control	Unstructured human-video-to-robot transfer; source videos are not confirmed as egocentric wearable capture	watch
Dexterous Point Policy	2026-06	arXiv	Transfers raw human videos into dexterous hand policies through task-relevant 3D keypoints for hands and objects	Human demonstration video transfer, not confirmed first-person capture	watch
TopoRetarget	2026-06	arXiv	Interaction-preserving retargeting from human hand-object demonstrations to dexterous robot-hand references	Human hand-object retargeting method, not confirmed wearable first-person capture	watch
WARP Retarget	2026-06	arXiv	Whole-body-aware retargeting from offline human demonstrations into mobile-manipulator trajectories	Offline human-demonstration retargeting, not confirmed wearable first-person capture	watch
V2P-Manip	2026-06	arXiv	Learns dexterous manipulation from monocular human videos through 3D assets, trajectory estimation, physical refinement, and policy learning	Monocular human-video-to-robot work, not confirmed wearable capture	watch
SeeTraceAct	2026-06	arXiv	Demo-conditioned VLA with visibility-aware future end-effector traces and RoboCasa-DC cross-embodiment demonstration episodes	Robot VLA conditioned on demonstrations, not a human wearable dataset	watch
What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?	2026-06	arXiv	532 everyday human videos with 28 hours of high-quality hand labels for studying robot-policy cotraining under limited robot data	Everyday human-video transfer study, not scoped as egocentric capture	watch
LARA	2026-06	arXiv	Latent Action Representation Alignment for jointly optimizing latent action models and VLAs from unlabeled human videos	Human-video-to-VLA method, not itself an egocentric dataset	watch
Video2Sim2Real	2026-06	arXiv	Reconstructs a simulator-ready digital twin and robot/object motion priors from a single human manipulation video	Single-human-video robot skill acquisition, not confirmed first-person wearable data	watch
HOWTransfer	2026-06	arXiv	Localizes hand-object contact in human video demonstrations and retargets wrist trajectories into robot-executable motions	Hand-centric human-video-to-robot transfer, not confirmed egocentric wearable capture	watch
Retrieve, Don't Retrain	2026-06	arXiv	Retrieval-augmented VLA adaptation using pool-side demonstrations such as human-hand videos instead of per-task retraining	Robot VLA adaptation with human-hand video pools, not centered on first-person data	watch
SERF	2026-06	arXiv	Spatiotemporal environment and robot feature map updated from egocentric observations and proprioception for long-horizon mobile manipulation	Robot-centric policy work using egocentric observations, not human wearable capture	watch
ImageWAM	2026-06	arXiv	World-action model that uses image editing instead of dense video generation for robot action prediction	Robot WAM comparison point, not a human first-person resource	watch
CHORD	2026-06	arXiv	NVIDIA contact-wrench-guided dexterous RL with 4,739 simulation tasks, 82.12% success over 1,831 evaluated tasks, whole-body transfer, and real-hardware results	Motion-capture and third-person human demonstrations rather than a wearable-video release; code remains forthcoming	watch
JAXenstein	2026-05	arXiv	JAX Wolfenstein 3D renderer for fast reinforcement-learning experiments in visual first-person tasks	Virtual first-person RL environment, not real egocentric capture	watch
ACWM-Phys	2026-05	arXiv	Controllable action-conditioned world-model benchmark for rigid, kinematic, deformable, and particle dynamics	Simulator benchmark for physical interaction, not human first-person capture	watch
SpatialBench	2026-05	arXiv	Broad spatial foundation-model benchmark across viewpoints, scene domains, input densities, and hardware constraints	Useful context for egocentric spatial reasoning, but not centered on human wearable capture	watch
LEXI-SG	2026-05	arXiv	Monocular RGB open-vocabulary 3D scene-graph mapping validated on self-collected egocentric office sequences	Robot/agent mapping work, not centered on human wearable capture	watch
EgoTraj-Bench	2026-04	ICRA 2026	36,947 public sequences from 17 robot FPV sessions for converting noisy first-person trajectories into clean bird's-eye trajectories	Robot-mounted first-person views rather than human wearable capture	open
EgoDyn-Bench	2026-04	ECCV 2026	Public 1,000-clip physics-grounded driving benchmark with about 14K oracle-labelled QA pairs, dynamics arrays, and a 49-model reference leaderboard	Autonomous-driving ego-motion video rather than human or animal wearable capture	open
DenseStep2M	2026-04	arXiv	About 100K instructional videos and 2M dense procedural steps from automated long-video annotation	General instructional-video corpus with egocentric transfer tests, not first-person centered	watch
StarVLA	2026-04	arXiv	Modular VLA codebase supporting swappable VLM backbones such as Qwen-VL and world-model backbones such as Cosmos	VLA tooling for comparisons, not human first-person capture	watch
Point of View Robot Sociability Study	2026-03	arXiv	Immersive-VR study comparing allocentric, egocentric-proximal, and egocentric-distal ratings of identical robot trajectories and a head-nod signal	Simulated pedestrian viewpoints rather than recorded wearable video	watch
RoboMirror	2025-12	arXiv	Retargeting-free humanoid locomotion from raw egocentric or third-person video using VLM-derived motion intent and a diffusion policy	Supports egocentric input but targets humanoid locomotion	watch
VLA-Arena	2025-12	ICML 2026	Open-source VLA benchmark with 170 structured tasks and decoupled task, language, and visual perturbation axes	Robot VLA benchmark, not a human wearable/egocentric dataset	watch
VLSA / AEGIS	2025-12	IROS 2026	Plug-and-play safety constraint layer for VLA policies plus SafeLIBERO safety-critical manipulation benchmark	Robot VLA safety architecture, not human wearable capture	watch
UniWM	2025-10	ECCV 2026	Memory-augmented world model for navigation foresight and planning, with public code and a Hugging Face dataset	Robot navigation cameras rather than human wearable capture	partial
MM-Nav	2025-10	arXiv	Multi-view VLA navigation model with 360-degree observations and synthetic expert data for reaching, squeezing, and avoiding	Robot/synthetic visual navigation rather than human wearable capture	watch
Seeing Across Views / MV-RoboBench	2025-10	ICLR 2026	1.7K curated QA items over eight subtasks for multi-view spatial reasoning of VLMs in robotic manipulation (ICLR 2026)	Multi-camera robot scenes, not wearable capture	open
HUI360	2025-09	FG 2026	99 mobile-robot 360-degree HRI recordings, a 1M-annotation HUI360 open set, 6M SSUP-HRI annotations, and public anticipation baselines	Captured from a mobile robot, not a human camera wearer	partial
HomeSafeBench	2025-09	arXiv	Open VirtualHome benchmark with 1,000 test tasks, 3,400 training tasks, five hazard classes, and 3,158 CueBack trajectories	Rendered embodied-agent observations rather than wearable capture	open
RoboPearls	2025-06	arXiv	Editable 3D Gaussian video simulation for robot manipulation, evaluated on RLBench, COLOSSEUM, Ego4D, Open X-Embodiment, and real robots	Robot simulation/editing tool, not a human egocentric dataset	watch
NORA	2025-04	arXiv	3B generalist VLA using Qwen2.5-VL-3B and 970K real-world robot demonstrations	Robot-only demonstrations and observations, not human wearable capture	watch
SEED4D	2024-12	WACV 2025	Synthetic ego-exo dynamic 4D generator and autonomous-driving dataset (16.8M images, vehicle cameras, LiDAR; WACV 2025)	Vehicle-egocentric driving data, not human/wearable	open
Open X-Embodiment / RT-X	2023-10	ICRA 2024	1M+ real robot trajectories, 22 robot embodiments, 60 pooled robot datasets, standardized RLDS; unofficial Hugging Face mirror	Robot-mounted and wrist cameras, not human first-person; pairs with egocentric human-video VLA pretraining	open

Surveys, Papers, and Context

Foundational papers and recent surveys for background and citation.

Resource	Why it matters
Vision-Language Models for Egocentric Video	Critical survey connecting hand-object interaction, temporal and graph reasoning, efficient long-video processing, wearable assistance, and embodied AI.
Ego4D: Around the World in 3,000 Hours of Egocentric Video	Defines the modern large-scale egocentric benchmark suite.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives	Defines the major synchronized ego-exo activity resource.
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset and EPIC-KITCHENS-100	Core kitchen/action-recognition benchmark lineage.
Egocentric Video-Language Pretraining	Introduces EgoClip and EgoNCE, a central egocentric VLP recipe.
EgoSchema	A widely used diagnostic benchmark for very long-form video-language understanding.
HoloAssist	Important for human-AI assistance and instructor-performer interaction.
Nymeria	Major egocentric multimodal motion-language dataset from Project Aria.
HOT3D	Major 3D hand-object tracking dataset for AR/VR.
OpenEgo, EgoDex, EgoVerse	Recent large-scale egocentric manipulation data for robot learning.
Challenges and Trends in Egocentric Vision: A Survey	Recent comprehensive survey of egocentric tasks, datasets, and open problems.
Bridging Perspectives: Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision	Survey of ego-exo collaboration and paired-capture research.
World Action Models	Systematizes WAMs across VLA, world models, portable human demonstrations, simulation, and internet-scale egocentric video.

Workshops and Challenges

Recurring venues and challenge hubs where new egocentric tasks and leaderboards appear.

Event / hub	Focus
SLT 2026 SmartGlasses Challenge	Two-track evaluation of timestamped speaker-attributed ASR and spoken-language understanding over 106.98 hours and 714 four-channel smart-glasses audio sessions; data is distributed to registered participants.
EgoLink 2026	ACM MM 2026 grand challenge for egocentric social reasoning and interactive tool-using agents, with public labels, code, leaderboard, 4,030 MCQs, and 1,055 evaluation videos.
Ego4D Challenges	Episodic memory, forecasting, hand-object, social/audio, and other Ego4D tasks.
Ego-Exo4D Challenges	Cross-view, correspondence, pose, skill, and proficiency tasks.
EPIC-KITCHENS Challenges	Action recognition, action detection, anticipation, retrieval, and domain adaptation.
HOT3D Challenge	3D hand-object tracking and AR/VR pose estimation.
Project Aria	Aria datasets, tools, and AR sensing ecosystem.
Joint Egocentric Vision (EgoVis) Workshop	Cross-dataset egocentric vision workshop (CVPR) spanning Ego4D, Ego-Exo4D, EPIC-KITCHENS, HoloAssist, and more.
CVPR / ICCV / ECCV egocentric and embodied AI workshops	Search annually for "egocentric", "embodied", "ego-exo", "wearable", and "first-person".

Watchlist

These entries are promising but should be rechecked before treating them as stable public datasets in a paper or benchmark.

Resource	Why to track	Current note
EgoLive	2026-04	arXiv
WiYH	Large manipulation corpus with rich hand/wrist/depth signals	Release and license need verification.
OpenEgo	2025-09	arXiv
EgoVerse	2026-04	arXiv
EgoDex	2025-05	ICLR 2026
UMI family: FastUMI / MV-UMI / UMIGen / YUBI / Hoi!	Rapidly expanding handheld and wrist-view manipulation interfaces	Track public code, HF/GitHub datasets, licenses, and whether claimed releases are complete.
EgoIntrospect	2026-05	arXiv
EgoBench	2026-05	arXiv
EgoAERO / EgoEngine / HumanEgo / EgoGuide / Ego-Pi	Conversion of egocentric videos into robot demonstrations and policies	Track code, dataset artifacts, and robot-transfer evaluation protocols.
EgoEMG / EgoEVHands / TouchMoment / EgoFun3D / EgoTactile / EgoPressDiff / EgoForce	2026 hand/contact/event/3D resources	Track GitHub/HF data release, pressure/contact assets, and license.
UCS-Bench / StreamMemBench / ReFocus / Plan Watch Recover	Streaming memory, spatial reasoning, and proactive assistance	Track full data/code releases and raw-video dependencies.
EgoProx / BARISTA / TAVIS / EgoPoint-Bench / Ego-METAS	New 2026 evaluation suites with strong task definitions	Track stable leaderboards, splits, and license terms.
Causal-Plan-1M / HowToDIV / EgoThink-family / EgoCoT / NoRA / Ego2Web / Pause and Think	Reasoning, planning, safety, and agentic egocentric benchmarks	Confirm URLs, licenses, and raw-video dependencies.

Inclusion Rules

1.Prefer official project pages, GitHub repositories, Hugging Face datasets, arXiv pages, or university dataset pages.
2.Keep the public status conservative. If raw video is not clearly available, mark partial or watch.
3.Separate raw datasets from derived annotations and benchmarks.
4.Record modality, scale, task, license/access friction, and raw-video dependency when known.
5.Keep the main atlas strictly egocentric: human or animal first-person capture, or ego-exo resources where the ego view is central. List related but non-egocentric resources (robot-only datasets, multi-view robotic benchmarks, autonomous-driving data, general long-video reasoning) under Adjacent and Related Resources.
6.Keep pure dashcam/autonomous-driving and robot-embodiment data out of the main tables; surface it as context in Adjacent and Related Resources only when it is genuinely useful to egocentric work.
7.Keep figures text-light. Exact resource names and labels should remain in Markdown/SVG so readers can inspect them directly.

Contributing

Contributions are welcome through pull requests and issues. Please include an official source, a concise description of what the resource contributes, and any known access or license notes.

See `CONTRIBUTING.md` for the inclusion policy and style. Automated checks run in CI to keep the README, catalog data, figures, and public exports consistent; contributors do not need to run local scripts before opening a PR.

Cite This Atlas

If this atlas helps your research or project, a citation or a link back is appreciated. Machine-readable metadata lives in `CITATION.cff`.

bibtex

@misc{he_awesome_egocentric_atlas,
  author       = {He, Chaoyue},
  title        = {Awesome Egocentric Atlas: Egocentric AI Datasets, Benchmarks, Models, and Tools},
  year         = {2026},
  howpublished = {\url{https://github.com/ChaoYue0307/awesome-egocentric-atlas}}
}