CoolFace
Datasetpublic

zeno-labs/egostation-smartphone-raw-v1

EgoStation Smartphone Raw Catalog v1 Pre-processing catalog of smartphone egocentric video recordings. Most episodes are exposed as metadata only; a small set of representative sample episodes are shipped with full raw video + derived data so partners can inspect data quality before signing up for the full corpus. Built: 2026-05-12T19:05Z Total episodes: 5962 Total recorded time: ~693.9 hours Format: Apache Parquet (catalog) + raw media for sample episodes Maintained by: ZenO… See the full description on the dataset page: https://huggingface.co/datasets/zeno-labs/egostation-smartphone-raw-v1.

sourceHugging Facecc-by-nc-4.0updated 5mo agoView on Hugging Face
0likes97downloads
Dataset Card

EgoStation Smartphone Raw Catalog v1

Pre-processing catalog of smartphone egocentric video recordings. Most episodes are exposed as metadata only; a small set of representative sample episodes are shipped with full raw video + derived data so partners can inspect data quality before signing up for the full corpus.

  • —Built: 2026-05-12T19:05Z
  • —Total episodes: 5962
  • —Total recorded time: ~693.9 hours
  • —Format: Apache Parquet (catalog) + raw media for sample episodes
  • —Maintained by: ZenO Labs — https://zen-o.xyz

Data exposure policy

There are two tiers of access in this repo:

1. Sample episodes (samples/<domain>/<upload_id>/ — full data)

For a curated subset, the actual files are shipped inside this HF repo:

FileSourcePurpose
video.mp4 (or .mov)ZenO Core / R2 originalRaw egocentric capture
pose_keypoints.jsonMediaPipe Hands (Zeno pipeline)Per-frame 2D hand keypoints (21 points × 2 hands)
depth_overlay.mp4Zeno depth pipelineColorized relative depth visualization
meta.jsonthis catalogEpisode metadata snapshot

These are the only files in this repo that can be downloaded directly. Mission domain (cooking, kitchen-cleanup, …) decides the sub-folder.

2. All other episodes (~5946+ — metadata only)

The remaining episodes are listed in catalog.parquet with full metadata, but the actual media files are not in this repo. Each row's modality flags (hand_pose, depth, hand_world, …) show what can be delivered on request, packaged in LeRobot v2.1 format.

To obtain raw video, derived data, or a full LeRobot bundle for any upload_id outside the samples/ folder, contact support@zen-o.xyz with the upload_id list and the modalities you need.

Why is no derived data available yet?

These recordings are smartphone first-person videos captured by ZenO contributors. They are queued for processing once the storage infrastructure migration completes. After processing, episodes will graduate to the main egostation-catalog-v1 catalog with full LeRobot v2.1 data layout.

Per-episode availability flags

Each row in catalog.parquet carries a status string per modality:

FieldMeaning
head_trajectorynot-supported — smartphone capture lacks synchronized inertial data, so mono-inertial SLAM cannot recover 6DoF camera pose.
hand_poseavailable — derivable on request (2D keypoints, both hands, 21 points each).
depthavailable — relative depth maps derivable on request.
hand_worldavailable — 3D hand keypoints in camera frame derivable on request (depends on depth).
action_segmentavailable — manual annotation possible on request.

After processing, the value updates to completed and the episode is moved to the main catalog.

catalog.parquet schema

FieldTypeDescription
episode_idstringep_NNNNNN, unique within this repo
upload_idstringZenO Core upload identifier — abstract handle (does not change across storage migrations)
sourcestringsmartphone
mission_namestringTask category (e.g. Cooking, Washing Dishes, Tidy Up Shoes)
mission_groupstringHigher-level grouping (Mission 5, Mission 7, …)
mission_idintInternal ZenO Core mission identifier
file_namestringOriginal device filename
file_size_mbintFile size in megabytes
file_typestring"Video"
duration_sfloatEpisode length in seconds
width, heightintOriginal video resolution
uploaded_attimestamp (UTC, ISO 8601)When the contributor uploaded
catalog_tierstringprivate (raw catalog — metadata only)
head_trajectory, hand_pose, depth, hand_world, action_segmentstringModality availability status
is_processedboolfalse for all rows in this raw catalog

Mission distribution

  • —Washing Dishes: 1216
  • —Cooking: 1131
  • —Wipe Objects and Furniture: 863
  • —Gripping a Door Handle: 629
  • —Kitchen Cleanup: 531
  • —Laundry Folding Challenge: 459
  • —Tidy Up Challenge: 260
  • —Home Cleaning Task: 219
  • —Dishwashing Task: 217
  • —Tidy Up Your Room: 188
  • —Laundry Task: 182
  • —Tidy Up Shoes: 67

Sample episodes (real data)

16 representative episodes with raw video + 2D hand keypoints, organized by mission domain.

egostation-smartphone-raw-v1/
└── samples/
    ├── cooking/
    │   ├── 89c5487e…/
    │   │       ├── video.mp4    (843 MB, 731s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   ├── e6966629…/
    │   │       ├── video.mp4    (718 MB, 625s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   ├── 630c7405…/
    │   │       ├── video.mp4    (690 MB, 1046s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   └── 72700d9c…/
    │           ├── video.mp4    (981 MB, 1003s)
    │           ├── pose_keypoints.json
    │           ├── depth_overlay.mp4
    │           └── meta.json
    ├── kitchen-cleanup/
    │   └── e0797265…/
    │           ├── video.mp4    (353 MB, 445s)
    │           ├── pose_keypoints.json
    │           ├── depth_overlay.mp4
    │           └── meta.json
    ├── laundry-folding-challenge/
    │   ├── 8e3cfad8…/
    │   │       ├── video.mp4    (567 MB, 587s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   ├── 38c6b391…/
    │   │       ├── video.mp4    (592 MB, 414s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   ├── 00c5a81c…/
    │   │       ├── video.mp4    (925 MB, 946s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   └── dc36f9b0…/
    │           ├── video.mp4    (653 MB, 372s)
    │           ├── pose_keypoints.json
    │           ├── depth_overlay.mp4
    │           └── meta.json
    ├── tidy-up-shoes/
    │   └── d311a32e…/
    │           ├── video.mp4    (902 MB, 923s)
    │           ├── pose_keypoints.json
    │           ├── depth_overlay.mp4
    │           └── meta.json
    ├── washing-dishes/
    │   ├── 7c98ebfb…/
    │   │       ├── video.mp4    (638 MB, 802s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   ├── 50b9fc4b…/
    │   │       ├── video.mp4    (943 MB, 964s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   ├── 61e65f9c…/
    │   │       ├── video.mp4    (704 MB, 619s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   ├── 71fcad78…/
    │   │       ├── video.mp4    (618 MB, 718s)
    │   │       ├── pose_keypoints.json
    │   │       ├── depth_overlay.mp4
    │   │       └── meta.json
    │   └── dfea9e2f…/
    │           ├── video.mp4    (587 MB, 695s)
    │           ├── pose_keypoints.json
    │           ├── depth_overlay.mp4
    │           └── meta.json
    └── wipe-objects-and-furniture/
        └── 7de39f3a…/
                ├── video.mp4    (910 MB, 930s)
                ├── pose_keypoints.json
                ├── depth_overlay.mp4
                └── meta.json

Sample list

MissionEpisodeDurationVideoHand poseDepth
Laundry Folding Challenge8e3cfad8…9:46567 MB✓✓
Cooking89c5487e…12:10843 MB✓✓
Laundry Folding Challenge38c6b391…6:54592 MB✓✓
Washing Dishes7c98ebfb…13:22638 MB✓✓
Kitchen Cleanupe0797265…7:24353 MB✓✓
Laundry Folding Challenge00c5a81c…15:45925 MB✓✓
Washing Dishes50b9fc4b…16:03943 MB✓✓
Cookinge6966629…10:24718 MB✓✓
Washing Dishes61e65f9c…10:18704 MB✓✓
Cooking630c7405…17:25690 MB✓✓
Washing Dishes71fcad78…11:58618 MB✓✓
Wipe Objects and Furniture7de39f3a…15:30910 MB✓✓
Washing Dishesdfea9e2f…11:35587 MB✓✓
Tidy Up Shoesd311a32e…15:22902 MB✓✓
Cooking72700d9c…16:42981 MB✓✓
Laundry Folding Challengedc36f9b0…6:12653 MB✓✓

Sample episodes are full quality — for the rest of the catalog, raw videos are available via R2 pre-signed URL under signed agreement.

Usage

python
from datasets import load_dataset

catalog = load_dataset("zeno-labs/egostation-smartphone-raw-v1", "catalog", split="train")
df = catalog.to_pandas()
print(df.head())

# Filter to a domain
cooking = df[df["mission_name"] == "Cooking"]
print(f"Cooking episodes: {len(cooking)}, total {cooking['duration_s'].sum()/3600:.1f}h")

Future structure (after processing)

Each episode, once processed, will be packaged in the LeRobot v2.1 format under the main catalog:

egostation-catalog-v1/
└── smartphone/
    └── <mission_name>/
        ├── data/chunk-NNN/episode_NNNNNN.parquet
        ├── videos/chunk-NNN/
        │   ├── observation.images.head/episode_NNNNNN.mp4
        │   └── observation.images.depth/episode_NNNNNN.mp4
        └── meta/
            ├── info.json
            ├── episodes.jsonl
            └── tasks.jsonl

Access to raw videos

Metadata in this catalog is freely browsable. For raw video access (needed for downstream processing or your own pipeline), contact support@zen-o.xyz with your upload_id list.

Licensing

Catalog metadata: CC-BY-NC 4.0. Raw videos: commercial license required.

Contact

  • —Data access & partnerships: support@zen-o.xyz
  • —Technical issues: support@zen-o.xyz
  • —Website: https://zen-o.xyz
  • —Pipeline (ZenO Studio): https://studio.zen-o.xyz