zw1213757576/CareManip

Dataset Card for CareManip (HDF5 Format)

CareManip is a real-world leader-follower robot teleoperation dataset for care-oriented tabletop manipulation. The release contains 15 task categories and 1,500 HDF5 episodes. Each HDF5 file records one complete demonstration trajectory and preserves the original action and robot-state arrays for reproducible use in robot learning research.

Dataset release: v1.0 Dataset DOI: To be generated after the final public release Associated paper: To be added License: CC BY 4.0

Dataset Description

CareManip was collected to support research on robot imitation learning, embodied AI, assistive robotics, and multimodal tabletop manipulation. The task suite includes care-oriented object pick-and-place and push operations designed around everyday object handling.

The control data follow a leader-follower configuration:

• —The action vector has 14 dimensions. It represents the left and right leader wrist poses - position and orientation - together with two leader gripper commands.
• —The robot state vector (`qpos`) has 16 dimensions. It represents 14 follower-arm joint positions - seven joints per arm - and two follower claw positions.

This separation makes it possible to study mappings from leader-space teleoperation commands to follower-robot joint states, as well as sequence-policy learning from visual, state, and action observations.

Release Statistics

Task Categories

The release is organized as one directory per task under data/. Each task directory contains the 100 HDF5 demonstrations belonging to that task.

File Structure

Each .hdf5 file represents one complete trajectory, also referred to as an episode.

.
├── README.md
├── LICENSE
├── CITATION.cff
└── data/
    ├── pick_block/
    │   ├── pick_block0000.hdf5
    │   ├── pick_block0001.hdf5
    │   ├── ...
    │   └── pick_block0099.hdf5
    ├── pick_bin_bag/
    │   ├── pick_bin_bag0000.hdf5
    │   ├── ...
    │   └── pick_bin_bag0099.hdf5
    ├── pick_brown_bottle/
    ├── pick_mask/
    ├── pick_medicine/
    ├── pick_medicine_bottle/
    ├── pick_one_medicine_bottle/
    ├── pick_toy/
    ├── pick_vitamin_bottle/
    ├── pick_white_bottle/
    ├── push_cotton_swab_holder/
    ├── push_cup/
    ├── push_glasses_box/
    ├── push_tissues_box/
    └── push_towel/

File naming convention:

<task_name><zero-padded_episode_index>.hdf5

For example:

pick_block0004.hdf5

denotes the fifth recorded episode of the pick_block task.

HDF5 Internal Structure

Each HDF5 episode contains an action sequence and a follower-robot joint-state sequence. Let T denote the number of synchronized time steps in one episode.

The exact HDF5 group path used for dimension-name metadata may differ between recording versions. The semantic order below is the authoritative interpretation of the 14-dimensional action and 16-dimensional qpos vectors.

Action Vector: /action

The action vector contains two six-degree-of-freedom leader-wrist pose representations and two leader-gripper values.

The coordinate frame, units, orientation convention, and gripper-value range are inherited from the original teleoperation logging system. Users should preserve these conventions when training, normalizing, or replaying policies.

Follower State Vector: /observations/qpos

The 16-dimensional qpos vector contains 14 follower-arm joint positions and two follower-claw states.

Visual Observations

Some CareManip recording versions may include image observations in the HDF5 file. The camera keys, resolutions, encodings, and frame-synchronization method must be documented from the final released HDF5 schema before DOI generation.

Use the schema-inspection code below to identify all visual-observation keys in a representative episode. If RGB images are stored as compressed byte buffers, they must be decoded before use.

Usage Example

Install the required packages:

pip install h5py numpy

Load and inspect one local HDF5 episode:

from pathlib import Path
import h5py

file_path = Path("data/pick_block/pick_block0000.hdf5")

with h5py.File(file_path, "r") as f:
    print("Top-level keys:", list(f.keys()))

    def show_tree(name, obj):
        if isinstance(obj, h5py.Dataset):
            print(f"{name}: shape={obj.shape}, dtype={obj.dtype}")

    f.visititems(show_tree)

    action = f["action"][:]
    qpos = f["observations/qpos"][:]

print("Action shape:", action.shape)  # expected: (T, 14)
print("qpos shape:", qpos.shape)      # expected: (T, 16)

Download a single episode from Hugging Face:

from huggingface_hub import hf_hub_download
import h5py

repo_id = "zw1213757576/CareManip"
filename = "data/pick_block/pick_block0000.hdf5"

local_path = hf_hub_download(
    repo_id=repo_id,
    repo_type="dataset",
    filename=filename,
)

with h5py.File(local_path, "r") as f:
    action = f["action"][:]
    qpos = f["observations/qpos"][:]

Recommended Evaluation Protocol

CareManip is released as raw demonstrations. To ensure fair comparisons:

1. 1.Split data by episode, never by individual frames.
2. 2.Keep all frames from one HDF5 trajectory within the same split.
3. 3.Report task-level performance and aggregate performance across the 15 task categories.
4. 4.State whether models use actions, follower state, visual observations, language instructions, or a combination of modalities.
5. 5.Report the preprocessing applied to action and qpos vectors, including normalization, resampling, filtering, clipping, and coordinate transformations.
6. 6.Clearly distinguish in-distribution task performance from evaluation on unseen objects, scene configurations, or tasks.

The official train/validation/test split files will be added in a future update.

Intended Uses

CareManip is intended for research and education in:

• —Behavior cloning and robot imitation learning;
• —Dual-arm manipulation and coordinated bimanual control;
• —Visual and multimodal robot learning;
• —Embodied AI and vision-language-action research;
• —Assistive and care-oriented service robotics;
• —Task-conditioned action prediction;
• —Teleoperation analysis and leader-follower control modeling;
• —Benchmark development for HDF5-based manipulation datasets.

Limitations

• —CareManip is collected in structured tabletop environments and does not by itself establish generalization to unseen homes, clinics, objects, users, robot platforms, or manipulation settings.
• —The action representation is expressed in leader-wrist and leader-gripper space, while qpos represents the follower robot. Users must account for this representation difference when designing learning targets.
• —The current release contains raw HDF5 files. Hugging Face's Dataset Viewer may not directly preview all HDF5 contents.
• —Dataset quality, task-success labels, scene annotations, camera calibrations, and official data splits should be interpreted only from files and documentation included in the final public release.
• —The dataset must not be used as the sole basis for safety-critical or clinical decision-making.

Ethical and Privacy Considerations

Before public release, every HDF5 episode and associated metadata must be reviewed for personally identifiable information. This includes faces, names, speech, computer-screen contents, laboratory credentials, and other sensitive information.

The final accompanying paper should state the applicable ethics-review status, consent procedure, and data-sharing restrictions, if any.

License

The CareManip dataset is released under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

When using the dataset, users must cite both the dataset DOI and the associated paper.

Citation

The Hugging Face DOI and paper citation will be inserted after the public archival release has been finalized.

@dataset{caremanip_2026,
  title     = {CareManip: A Teleoperation Dataset for Care-Oriented Tabletop Manipulation},
  author    = {REPLACE WITH AUTHOR LIST},
  year      = {2026},
  version   = {1.0},
  publisher = {Hugging Face},
  doi       = {REPLACE WITH HUGGING FACE DOI},
  url       = {https://huggingface.co/datasets/zw1213757576/CareManip}
}

Contact

For questions, corrections, or collaboration requests, please use the repository discussion page or contact:

REPLACE WITH CONTACT EMAIL