TUM-ICS/Hide-and-Seek

Hide-and-Seek Dataset

The Hide-and-Seek (HAS) dataset is a large-scale multimodal tactile dataset for tactile-only object recognition and retrieval in vision-denied environments.

This dataset accompanies the ICRA 2026 paper:

Tactile Hide and Seek: Bimanual Object Blind Search and Retrieval Via Tactile-Only Feedback

The dataset was collected on the Tactile Omnidirectional Mobile Manipulator (TOMM) using hand-mounted tactile skin sensors, wrist force-torque sensing, and robot proprioception. It supports learning-based tactile perception for object classification, weight classification, and bimanual verification.

Project page: https://tactile-hide-and-seek.github.io/

Dataset Summary

HAS contains 1,120,116 frame-wise tactile examples, approximately 1.12M frames, grouped into approximately 4.3k trajectories.

The dataset covers:

• —33 physical objects
• —1 negative class
• —61 object-weight categories
• —4 weight labels: none, light, medium, heavy
• —Multimodal tactile observations from robot skin, force-torque sensing, and end-effector pose

The dataset is stored in Parquet format and provides official train / validation / test splits.

Data Splits

Features

Each row corresponds to one frame of synchronized tactile interaction data.

The tactile point cloud has shape 88 × 6, where the 88 points correspond to tactile skin cells across both robot hands. Each point contains spatial and sensor-derived channels.

Loading the Dataset

from datasets import load_dataset

dataset = load_dataset("TUM-ICS/Hide-and-Seek")

print(dataset)
print(dataset["train"][0].keys())

Example output:

DatasetDict({
    train: Dataset(...)
    validation: Dataset(...)
    test: Dataset(...)
})

Access one sample:

sample = dataset["train"][0]

point_cloud = sample["obs_point_cloud_skin_contact"]
mask = sample["mask_point_cloud_skin_contact"]
label = sample["classification_label"]

print(len(point_cloud))  # 88
print(label)

Task Description

The dataset supports tactile-only classification from robot interaction data.

Given a window of tactile observations, the learning task is to predict:

1. 1.Object identity
2. 2.Weight category

The ICRA 2026 paper uses a dual-head multimodal classifier that jointly predicts object class and weight class from synchronized tactile modalities.

Modalities

HAS includes five synchronized sensing streams:

1. 1.Tactile point cloud Contact geometry and sensor intensities from distributed tactile skin cells.

1. 1.Raw skin signals Per-cell proximity, force, and distance readings.

1. 1.Virtual skin wrenches Aggregated proximity- and force-based wrench representations.

1. 1.Proprioceptive pose End-effector pose from the robot kinematic state.

1. 1.Wrist force-torque wrench Six-axis force / torque measurements from the wrist sensor.

Dataset Collection

The dataset was collected in a vision-denied tabletop workspace using TOMM, a bimanual robot equipped with hand-mounted tactile skin.

The data collection protocol includes:

• —Single-hand pushing interactions for tactile object classification
• —Bimanual grasping interactions for verification
• —Randomized object placements
• —Multiple object-weight variants
• —Data collection across different days to reduce overfitting to a single sensor state

The official train / validation / test splits are provided for reproducible evaluation.

Baseline Results

The ICRA 2026 paper reports the following representative results on the HAS dataset and online robot trials:

The bimanual verification stage further corrects up to 17.6% of single-hand errors in online trials.

Label Mapping

The classification_label field encodes 61 object-weight categories as integer labels.

The full mapping is provided in `label_mapping.yaml`.

Each label follows the format:

<object_name>_<weight_class>

where the weight class is one of:

light, medium, heavy

The special label negative corresponds to non-target / negative examples.

Citation

If you use this dataset, please cite:

@inproceedings{fu2026tactilehideandseek,
  title     = {Tactile Hide and Seek: Bimanual Object Blind Search and Retrieval Via Tactile-Only Feedback},
  author    = {Fu, Xiangyu and Xing, Hao and Armleder, Simon and Shen, Wenlan and Wang, Fengyi and Guadarrama-Olvera, Julio Rogelio and Cheng, Gordon},
  booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
  year      = {2026}
}

Contact

For questions, please contact:

Xiangyu Fu Technical University of Munich Project page: https://tactile-hide-and-seek.github.io/