rlorlou/HUI360
HUI360 HUI360: A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation (IEEE FG 2026) Open-access skeleton annotations for HUI360, a large-scale 360° egocentric dataset for human-robot interaction anticipation in the wild. This repository provides the annotations as tabular CSV files (one row per detection), ready for training and evaluation with HUI360-Baselines. Related resources Resource Link Project… See the full description on the dataset page: https://huggingface.co/datasets/rlorlou/HUI360.
HUI360
HUI360: A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation (IEEE FG 2026)
<p align="center"> <a href="https://hucebot.github.io/hui360/"><img src="https://img.shields.io/badge/website-hui360-green" alt="Website"></a> <a href="https://huggingface.co/papers/2608.11051"><img src="https://img.shields.io/badge/paper-HuggingFace-yellow" alt="Paper"></a> <a href="https://hal.science/view/index/docid/5609928"><img src="https://img.shields.io/badge/paper-HAL%20Science-blue" alt="Paper"></a> <a href="https://doi.org/10.1109/FG67764.2026.11556969"><img src="https://img.shields.io/badge/IEEE-FG%202026-orange" alt="IEEE FG 2026"></a> <a href="https://github.com/RaphaelLorenzo/Interact360/"><img src="https://img.shields.io/badge/annotation%20tool-Interact360-black?logo=github" alt="Annotation tool"></a> <a href="https://github.com/hucebot/HUI360-Baselines/"><img src="https://img.shields.io/badge/baselines-GitHub-black?logo=github" alt="Baselines"></a> <a href="https://huggingface.co/datasets/rlorlou/HUI360-Videos"><img src="https://img.shields.io/badge/videos-HuggingFace-yellow" alt="Videos"></a> </p>
Open-access skeleton annotations for HUI360, a large-scale 360° egocentric dataset for human-robot interaction anticipation in the wild. This repository provides the annotations as tabular CSV files (one row per detection), ready for training and evaluation with HUI360-Baselines.
Related resources
Annotation tool. Annotations were produced with the Interact360 pipeline (automatic detection, tracking, pose estimation, and interaction labelling) and manually refined with itsvisualize.pyGUI. The original per-frame JSON format is available in theannotations/folder of rlorlou/HUI360-Videos.
Dataset overview
The dataset covers 99 recordings (70 from INRIA Shelfy, 29 from Cornell SSUP-HRI), subdivided into episodes. Each CSV file corresponds to one recording; each row corresponds to one detection of one track.
Tracks have a unique ID within a file. Extract all detections for a single track using unique_track_identifier:
track_data = df[df["unique_track_identifier"] == "2022_09_21_astor_place_landfill_0000_0"]Within each recording, episodes are contiguous in time (no detections were found between episodes, so intervening data was discarded).
Column reference
xmin,xmax,ymin,ymax— bounding box in pixel coordinates. Images are equirectangular (3840×1920); boxes may wrap around the panorama (xmin > 3840meansxmin = 4240is equivalent toxmin = 400).sapiens_308_[JOINTNAME]_[x,y,score]— pixel coordinates and confidence for detections using Sapiens with the Goliath 308-keypoint format.vitpose_[JOINTNAME]_[x,y,score]— pixel coordinates and confidence for detections using ViTPose with the COCO-17 format.mask_rle— RLE-encoded binary mask of the person in the image. Encoding / decoding functions:
import torch
def encode_RLE(mask):
"""
Encode a mask into a RLE.
Args:
mask: torch.bool [H, W]
Returns:
runs: torch.tensor [N] - run lengths
"""
flat = mask.flatten() # [H*W]
if flat.numel() == 0:
return torch.tensor([], device=flat.device), False
starts_with_true = flat[0].item()
# Find transitions between True/False
# Add dummy values at start and end to handle boundaries
padded = torch.cat([torch.tensor([not flat[0]], device=flat.device), flat, torch.tensor([not flat[-1]], device=flat.device)])
# Find where values change
transitions = torch.nonzero(padded[1:] != padded[:-1], as_tuple=False).flatten()
# Calculate run lengths
runs = torch.diff(transitions)
# append a 1 if starts_with_true else a 0 so that we don't have to return starts_with_true
if starts_with_true:
runs = torch.cat([torch.tensor([1], device=runs.device), runs])
else:
runs = torch.cat([torch.tensor([0], device=runs.device), runs])
return runs
def decode_RLE(runs, shape):
"""
Decode a RLE into a mask.
Args:
runs: torch.tensor [N] - run lengths
shape: tuple - shape to reshape result to
Returns:
mask: torch.bool [H, W]
"""
start_with_true = runs[0].item()
runs = runs[1:]
if runs.numel() == 0:
return torch.zeros(shape, dtype=torch.bool, device=runs.device)
# Create alternating pattern: start_with_true determines first value
start_val = 1 if start_with_true else 0
vals = (torch.arange(runs.numel(), device=runs.device) + start_val) % 2
# Expand runs into full sequence
expanded = torch.repeat_interleave(vals, runs).bool()
# Reshape to target shape
total_elements = shape[0] * shape[1] if len(shape) == 2 else shape[0]
if expanded.numel() != total_elements:
# Pad or truncate if needed
if expanded.numel() < total_elements:
padding = torch.zeros(total_elements - expanded.numel(), dtype=torch.bool, device=runs.device)
expanded = torch.cat([expanded, padding])
else:
expanded = expanded[:total_elements]
mask_dec = expanded.view(shape)
return mask_decCitation
If you use this dataset, please cite:
@INPROCEEDINGS{11556969,
author={Lorenzo-Louis, Raphael and Amadio, Fabio and Luvison, Bertrand and Ivaldi, Serena},
booktitle={2026 IEEE 20th International Conference on Automatic Face and Gesture Recognition (FG)},
title={HUI360 : A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation},
year={2026},
volume={},
number={},
pages={1-9},
doi={10.1109/FG67764.2026.11556969}
}