CoolFace
Datasetpublic

rlorlou/HUI360

HUI360 HUI360: A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation (IEEE FG 2026) Open-access skeleton annotations for HUI360, a large-scale 360° egocentric dataset for human-robot interaction anticipation in the wild. This repository provides the annotations as tabular CSV files (one row per detection), ready for training and evaluation with HUI360-Baselines. Related resources Resource Link Project… See the full description on the dataset page: https://huggingface.co/datasets/rlorlou/HUI360.

sourceHugging Faceupdated 1mo agoView on Hugging Face
1likes872downloads
Dataset Card

HUI360

HUI360: A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation (IEEE FG 2026)

<p align="center"> <a href="https://hucebot.github.io/hui360/"><img src="https://img.shields.io/badge/website-hui360-green" alt="Website"></a> <a href="https://huggingface.co/papers/2608.11051"><img src="https://img.shields.io/badge/paper-HuggingFace-yellow" alt="Paper"></a> <a href="https://hal.science/view/index/docid/5609928"><img src="https://img.shields.io/badge/paper-HAL%20Science-blue" alt="Paper"></a> <a href="https://doi.org/10.1109/FG67764.2026.11556969"><img src="https://img.shields.io/badge/IEEE-FG%202026-orange" alt="IEEE FG 2026"></a> <a href="https://github.com/RaphaelLorenzo/Interact360/"><img src="https://img.shields.io/badge/annotation%20tool-Interact360-black?logo=github" alt="Annotation tool"></a> <a href="https://github.com/hucebot/HUI360-Baselines/"><img src="https://img.shields.io/badge/baselines-GitHub-black?logo=github" alt="Baselines"></a> <a href="https://huggingface.co/datasets/rlorlou/HUI360-Videos"><img src="https://img.shields.io/badge/videos-HuggingFace-yellow" alt="Videos"></a> </p>

Open-access skeleton annotations for HUI360, a large-scale 360° egocentric dataset for human-robot interaction anticipation in the wild. This repository provides the annotations as tabular CSV files (one row per detection), ready for training and evaluation with HUI360-Baselines.

Related resources

ResourceLink
Project websitehucebot.github.io/hui360
Paper (Hugging Face)huggingface.co/papers/2608.11051
Paper (HAL)hal.science/view/index/docid/5609928
Videos & JSON annotations (gated)rlorlou/HUI360-Videos
SSUP processed videos (gated)rlorlou/HUI360-Videos-SSUP
Annotation & processing pipelineInteract360
Baselines codeHUI360-Baselines
Annotation tool. Annotations were produced with the Interact360 pipeline (automatic detection, tracking, pose estimation, and interaction labelling) and manually refined with its visualize.py GUI. The original per-frame JSON format is available in the annotations/ folder of rlorlou/HUI360-Videos.

Dataset overview

The dataset covers 99 recordings (70 from INRIA Shelfy, 29 from Cornell SSUP-HRI), subdivided into episodes. Each CSV file corresponds to one recording; each row corresponds to one detection of one track.

Tracks have a unique ID within a file. Extract all detections for a single track using unique_track_identifier:

python
track_data = df[df["unique_track_identifier"] == "2022_09_21_astor_place_landfill_0000_0"]

Within each recording, episodes are contiguous in time (no detections were found between episodes, so intervening data was discarded).

Column reference

  • xmin, xmax, ymin, ymax — bounding box in pixel coordinates. Images are equirectangular (3840×1920); boxes may wrap around the panorama (xmin > 3840 means xmin = 4240 is equivalent to xmin = 400).
  • sapiens_308_[JOINTNAME]_[x,y,score] — pixel coordinates and confidence for detections using Sapiens with the Goliath 308-keypoint format.
  • vitpose_[JOINTNAME]_[x,y,score] — pixel coordinates and confidence for detections using ViTPose with the COCO-17 format.
  • mask_rle — RLE-encoded binary mask of the person in the image. Encoding / decoding functions:
python
import torch

def encode_RLE(mask):
    """
    Encode a mask into a RLE.
    Args:
        mask: torch.bool [H, W]
    Returns:
        runs: torch.tensor [N] - run lengths
    """
    flat = mask.flatten() # [H*W]
    
    if flat.numel() == 0:
        return torch.tensor([], device=flat.device), False
    
    starts_with_true = flat[0].item()
    
    # Find transitions between True/False
    # Add dummy values at start and end to handle boundaries
    padded = torch.cat([torch.tensor([not flat[0]], device=flat.device), flat, torch.tensor([not flat[-1]], device=flat.device)])
    
    # Find where values change
    transitions = torch.nonzero(padded[1:] != padded[:-1], as_tuple=False).flatten()
    
    # Calculate run lengths
    runs = torch.diff(transitions)
    
    # append a 1 if starts_with_true else a 0 so that we don't have to return starts_with_true
    if starts_with_true:
        runs = torch.cat([torch.tensor([1], device=runs.device), runs])
    else:
        runs = torch.cat([torch.tensor([0], device=runs.device), runs])
        
    return runs

def decode_RLE(runs, shape):
    """
    Decode a RLE into a mask.
    Args:
        runs: torch.tensor [N] - run lengths
        shape: tuple - shape to reshape result to
    Returns:
        mask: torch.bool [H, W]
    """
    
    start_with_true = runs[0].item()
    runs = runs[1:]
    
    if runs.numel() == 0:
        return torch.zeros(shape, dtype=torch.bool, device=runs.device)
    
    # Create alternating pattern: start_with_true determines first value
    start_val = 1 if start_with_true else 0
    vals = (torch.arange(runs.numel(), device=runs.device) + start_val) % 2
    
    # Expand runs into full sequence
    expanded = torch.repeat_interleave(vals, runs).bool()
    
    # Reshape to target shape
    total_elements = shape[0] * shape[1] if len(shape) == 2 else shape[0]
    if expanded.numel() != total_elements:
        # Pad or truncate if needed
        if expanded.numel() < total_elements:
            padding = torch.zeros(total_elements - expanded.numel(), dtype=torch.bool, device=runs.device)
            expanded = torch.cat([expanded, padding])
        else:
            expanded = expanded[:total_elements]
    
    mask_dec = expanded.view(shape)
    return mask_dec

Citation

If you use this dataset, please cite:

bibtex
@INPROCEEDINGS{11556969,
  author={Lorenzo-Louis, Raphael and Amadio, Fabio and Luvison, Bertrand and Ivaldi, Serena},
  booktitle={2026 IEEE 20th International Conference on Automatic Face and Gesture Recognition (FG)},
  title={HUI360 : A 360° Egocentric Dataset and Baselines for Human-Robot Interaction Anticipation},
  year={2026},
  volume={},
  number={},
  pages={1-9},
  doi={10.1109/FG67764.2026.11556969}
}