Datasetpublic

Anonymous36/aria-surgical-hand-pose-v1

EgoSurgHands: Egocentric 3D Hand Pose for Open-Surgery Training (v1) A multi-rater IAA-validated dataset of 3D hand-pose annotations on egocentric Project Aria recordings of surgical suturing tasks. At a glance Train Validation Hand-instance rows 41,909 9,476 Procedure recordings (PRs) 55 13 Wearers (P1-P4) 4 4 Glove colors 3 3 Tasks 13 13 Recording days 2 2 Train and validation are PR-disjoint by construction (zero overlap). The 13… See the full description on the dataset page: https://huggingface.co/datasets/Anonymous36/aria-surgical-hand-pose-v1.

sourceHugging Facecc-by-nc-sa-4.0updated 5mo agoView on Hugging Face

0likes45downloads

Dataset Card

EgoSurgHands: Egocentric 3D Hand Pose for Open-Surgery Training (v1)

A multi-rater IAA-validated dataset of 3D hand-pose annotations on egocentric Project Aria recordings of surgical suturing tasks.

At a glance

	Train	Validation
Hand-instance rows	41,909	9,476
Procedure recordings (PRs)	55	13
Wearers (P1-P4)	4	4
Glove colors	3	3
Tasks	13	13
Recording days	2	2

Train and validation are PR-disjoint by construction (zero overlap). The 13 validation PRs are IAA-validated through a multi-rater workflow.

Quick start

python

from datasets import load_dataset
ds = load_dataset("Anonymous36/aria-surgical-hand-pose-v1")
print(ds)
print(ds["train"][0]["joints_3d"])  # 21×3 array, Aria MPS GT in camera frame (m)

Schema

Each row = one (frame, hand) pair. 21 columns:

Column	Type	Description
`frame_key`	string	unique key `PR{nnn}/{frame_id:05d}_{hand_side}`
`source_tag`	string	provenance tag
`image`	image (PNG)	1408×1408 fisheye RGB ego frame
`image_width`, `image_height`	int64	pixel dimensions
`intrinsics`	list<float>	4-vec `[fx, fy, cx, cy]` (Aria pinhole approximation)
`extrinsics`	list<list<float>>	4×4 SE(3) pose (identity for the released split)
`bbox`	list<float>	hand bounding box: 4-vec `[x0, y0, x1, y1]` derived from joint hull + 16.5% padding
`hand_side`	string	`left` or `right`
`joints_3d`	list<list<float>>	21×3 hand joints in Aria camera frame (m), Aria MPS sensor-fused
`joints_2d`	list<list<float>>	21×2 hand joints projected to image (px)
`pr`, `sequence_name`	string	procedure-recording id (e.g. `PR108`)
`frame_id`	int64	frame index within the PR
`frame_ts_us`	int64	device-uptime microseconds since recording start (NOT Unix epoch)
`wearer_id`	string	de-identified wearer label (`P1`, `P2`, `P3`, `P4`)
`experience`	string	training stage (`MS1`, `PGY0`, `PGY3`)
`handedness`	string	wearer's dominant hand
`glove_color`	string	`blue`, `white`, or `brown`
`task`	string	one of 13 surgical tasks (e.g. `figure of 8 + instrument tie`)
`recording_day`	string	`day_1` or `day_2` (anonymized; original ISO dates remapped)

The 21 joint indices follow the OpenPose-21 convention (wrist=0, thumb=1-4, index=5-8, middle=9-12, ring=13-16, pinky=17-20).

Task list (13)

simple interrupted + instrument tie (n=13,615), running sub-cuticular + aberdeen knot (5,956), horizontal mattress + one hand tie (5,450), needle loading (5,321), horizontal mattress + instrument tie (4,993), figure of 8 + instrument tie (4,498), figure of 8 + two handed tie (3,560), cutting (3,453), simple interrupted + two hand tie (1,469), figure of 8 + one hand tie (1,444), vertical mattress + instrument tie (952), horizontal mattress (826), two hand tie (293).

Wearer demographics (anonymized)

Wearer	Experience	Glove colors used	Handedness
P1	PGY3	blue	right
P2	MS1	white	left
P3	PGY0	white, brown	right
P4	PGY3	brown, blue	right

All 4 wearers are adults aged 22-35; none use vision correction during recording (Aria glasses fit constraints).

Ground truth

3D joint positions are derived from Project Aria Machine Perception Services (MPS), an on-device sensor-fusion pipeline that combines visual-inertial SLAM, multi-camera stereo, and online-calibration-aware undistortion. Because MPS uses inertial and stereo channels that monocular RGB models cannot access, MPS labels come from a fundamentally different sensing process than any vision-only model, supporting independent benchmark scoring.

Validation labels go through a multi-rater inter-annotator-agreement (IAA) workflow with adjudication of disagreements.

Splits and protocol

—Standard train→val protocol: train the head/adapter on the 41,909-row training partition; evaluate on the 9,476-row validation partition.
—Train and val PR sets are disjoint ({PR108..PR185} curated set on val side; remainder of {PR109..PR189} on train side).

License

CC BY-NC-SA 4.0 (non-commercial, share-alike, with attribution).

Anonymization & ethics

This dataset comprises ego-video of consenting adult medical trainees performing standard suturing exercises on synthetic-skin training pads. No patient data, third-party PII, faces, or clinical-systems content is captured.

Author names, institutional affiliations, contact emails, and absolute capture dates are intentionally omitted for double-blind review. The site of capture is described generically as a single medical-training simulation centre; recording days are remapped to day_1/day_2.

Maintenance

The dataset is committed to be available for at least 24 months from publication. Versioning follows semantic-versioning conventions; v1 freezes the rows reported in the accompanying paper. Issues / requests via the HF Discussions tab.