Datasetpublic

AnandMayank/QueST-PartNetMobility-SAPIEN

QueST: PartNet-Mobility SAPIEN Simulation Dataset This dataset accompanies the paper: QueST: Persistent Queries as Semantic Monitors for Drift Suppression in Long-Horizon TrackingMayank Anand, Mohammad Saqlain, Kyan Mahajan, Priya Shukla, G.C Nandi, Andrew MelnikCAO Workshop at ICLR 2026 What Is This Dataset? Synchronized RGB-D simulation sequences rendered in SAPIEN from PartNet-Mobility articulated objects, designed to stress-test long-horizon point… See the full description on the dataset page: https://huggingface.co/datasets/AnandMayank/QueST-PartNetMobility-SAPIEN.

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

10likes34kdownloads

Dataset Card

QueST: PartNet-Mobility SAPIEN Simulation Dataset

![Paper](https://arxiv.org/abs/2605.09513v1) ![License](https://creativecommons.org/licenses/by/4.0/) ![Downloads]() ![IIITA](https://www.iiita.ac.in)

This dataset accompanies the paper:

QueST: Persistent Queries as Semantic Monitors for Drift Suppression in Long-Horizon Tracking Mayank Anand, Mohammad Saqlain, Kyan Mahajan, Priya Shukla, G.C Nandi, Andrew Melnik CAO Workshop at ICLR 2026

What Is This Dataset?

Synchronized RGB-D simulation sequences rendered in SAPIEN from PartNet-Mobility articulated objects, designed to stress-test long-horizon point tracking under articulation, occlusion, and viewpoint change.

The dataset supports the QueST framework — which replaces frame-to-frame Markovian tracking with persistent semantic queries that attend globally across time, achieving a 67.7% APE reduction over TAP-Net on long-horizon articulated sequences.

Exact Folder Structure

Each sequence is stored as an individual take folder:

QueST-PartNetMobility-SAPIEN/
│
├── manipulation_1/              Level 1 — 1 joint actuated
│   ├── {object_id}/
│   │   ├── take_00/
│   │   │   ├── frames/          RGB-D frames (PNG sequence)
│   │   │   ├── affordance/      Pixel-level affordance maps
│   │   │   ├── video.mp4        Full sequence video (36.6 kB avg)
│   │   │   ├── affordance_vis_10frames.png   Visualization (455 kB)
│   │   │   └── metadata.json    Sequence metadata (20.5 kB)
│   │   ├── take_01/
│   │   └── ...
│   └── ...
│
├── manipulation_2/              Level 2 — 2 joints actuated
├── manipulation_3/              Level 3 — 3 joints actuated
└── manipulation_4/              Level 4 — 4 joints, 240 frames

What Each File Contains

File	Description
`frames/`	Individual RGB-D frames as PNG — use for frame-level tracking evaluation
`affordance/`	Pixel-level affordance annotations — interaction-relevant regions labeled
`video.mp4`	Full sequence as compressed video — use for temporal model training
`affordance_vis_10frames.png`	Visual summary of affordance labels across 10 evenly-spaced frames
`metadata.json`	Object ID, joint configuration, ground truth 3D trajectories, camera intrinsics

Complexity Levels

Level	Folder	Joints actuated	Max frames	Purpose
1	`manipulation_1`	1	~60	Short-horizon training baseline
2	`manipulation_2`	2	~120	Medium complexity
3	`manipulation_3`	3	~180	Hard multi-joint sequences
4	`manipulation_4`	4	240	Long-horizon stress test

Each level actuates joints sequentially — Level 4 is the cumulative long-horizon challenge designed to expose drift in Markovian trackers.

Key Statistics

Property	Value
Total images / rows	50,551
Total size	5.75 GB
Camera viewpoints	3 synchronized RGB-D views per sequence
Renderer	SAPIEN (physics-based)
Depth data	Included in frames/
Annotations	Pixel-level affordance + 3D ground-truth trajectories
Articulation types	Revolute, prismatic
Object categories	Storage furniture, appliances, hinged devices

Loading the Dataset

python

from datasets import load_dataset

# Load full dataset
ds = load_dataset("AnandMayank/QueST-PartNetMobility-SAPIEN")

# Load only long-horizon sequences (manipulation_4)
ds = load_dataset(
    "AnandMayank/QueST-PartNetMobility-SAPIEN",
    data_files={"train": "manipulation_4/**/*"}
)

Loading metadata for a specific take

python

import json
from huggingface_hub import hf_hub_download

# Download metadata for a specific take
meta_path = hf_hub_download(
    repo_id="AnandMayank/QueST-PartNetMobility-SAPIEN",
    filename="manipulation_1/35059/take_00/metadata.json",
    repo_type="dataset"
)

with open(meta_path) as f:
    meta = json.load(f)

print(meta.keys())
# dict_keys(['object_id', 'joint_config', 'trajectory_gt',
#            'camera_intrinsics', 'affordance_labels', ...])

Loading frames for tracking evaluation

python

from huggingface_hub import snapshot_download
import os
from PIL import Image

# Download a single take
path = snapshot_download(
    repo_id="AnandMayank/QueST-PartNetMobility-SAPIEN",
    repo_type="dataset",
    allow_patterns="manipulation_4/*/take_00/**"
)

# Load frames in order
frames_dir = os.path.join(path, "manipulation_4/35059/take_00/frames")
frames = sorted([
    Image.open(os.path.join(frames_dir, f))
    for f in os.listdir(frames_dir)
    if f.endswith(".png")
])
print(f"Loaded {len(frames)} frames")

Benchmark Results

Method	APE ↓	Drift@100 ↓	Identity Acc ↑
RAFT-3D	0.341	0.472	8.7%
CoTracker	0.276	0.398	19.2%
TAP-Net	0.251	0.372	21.4%
QueST (ours)	0.081	0.155	86.5%

QueST achieves 67.7% APE reduction over TAP-Net — the strongest prior method — while maintaining bounded error growth vs near-linear drift in baselines.

Reproducing Results

bash

git clone https://github.com/AnandMayank/QueST
cd QueST
pip install -r requirements.txt

# Download dataset
python scripts/download_dataset.py \
  --repo AnandMayank/QueST-PartNetMobility-SAPIEN \
  --output data/

# Evaluate on Level 4 long-horizon sequences
python evaluate.py \
  --data data/manipulation_4 \
  --checkpoint checkpoints/quest_full.ckpt \
  --level 4

Citation

If you use this dataset please cite:

bibtex

@inproceedings{anand2026quest,
  title     = {QueST: Persistent Queries as Semantic Monitors for 
               Drift Suppression in Long-Horizon Tracking},
  author    = {Anand, Mayank and Saqlain, Mohammad and Mahajan, Kyan 
               and Shukla, Priya and Nandi, G.C. and Melnik, Andrew},
  booktitle = {CAO Workshop at ICLR 2026},
  year      = {2026}
}

License

Creative Commons Attribution 4.0 International (CC-BY 4.0)

Free to use for any purpose including commercial use, with attribution.

Contact

IIIT Allahabad — Department of Information Technology Mayank Anand · iit2024036@iiita.ac.in G.C. Nandi · gcnandi@iiita.ac.in

University of Bremen Andrew Melnik · andrew.melnik.papers@gmail.com

Issues and questions: github.com/AnandMayank/QueST