Datasetpublic

anon-review-artifact-7k3p9/review-dataset-index

MANUS-Bench — anonymous release bundle (WACV 2027 submission #1065) MANUS-Bench (Multimodal Annotated Naturalistic Hand Understanding) is a benchmark for geometry-conditioned hand-gesture synthesis in natural scenes: aligned full-scene and localized hand conditions over an official condition-complete protocol of 88,294 samples (64,931 train / 23,363 test) drawn from an 88,312-record archive, with fixed in-domain, near-domain, and hand-object OOD split roles. This repository is… See the full description on the dataset page: https://huggingface.co/datasets/anon-review-artifact-7k3p9/review-dataset-index.

sourceHugging Faceotherupdated 29d agoView on Hugging Face

0likes154downloads

Dataset Card

MANUS-Bench — anonymous release bundle (WACV 2027 submission #1065)

MANUS-Bench (Multimodal Annotated Naturalistic Hand Understanding) is a benchmark for geometry-conditioned hand-gesture synthesis in natural scenes: aligned full-scene and localized hand conditions over an official condition-complete protocol of 88,294 samples (64,931 train / 23,363 test) drawn from an 88,312-record archive, with fixed in-domain, near-domain, and hand-object OOD split roles.

This repository is the index and metadata bundle: split lists, per-record manifests, schema docs, loaders, canonical evaluation code, checksums, license notes, and the (unexecuted) human-verification package. It hosts no image data. Everything here is anonymised for double-blind review.

Where the data is

The payload is hosted anonymously as WebDataset shards, sha256-verified, 80 shards / 66.9 GiB total, split by upstream license (see licenses/):

payload repository	shards	contents
`anon-review-artifact-7k3p9/review-dataset-subset-a`	73	HaGRID-derived: `hagrid_v1` train (18) + test (8), `hagrid_v2` train (33) + test (7), plus 7 supplementary `v2_test` vertices shards
`anon-review-artifact-7k3p9/review-dataset-subset-b`	7	DexYCB-derived: `dexycb` test

FLAG (staging): verify the namespace and repository names against the live anonymous repositories before any upload of this bundle (no upload happens from staging). Per-shard sha256 digests are in checksums/shards.sha256; checksums/BUNDLE_CHECKSUMS.sha256 covers this bundle.

Five-minute reviewer path

bash

python examples/smoke_test.py            # metadata-only: counts + uint8 assertion
# after downloading and unpacking the payload (or pointing at an unpacked copy):
python examples/smoke_test.py --root <DATA_ROOT>
python loaders/load_manus_bench.py --root <DATA_ROOT> --splits-dir splits \
    --config hagrid_v1 --split test --verify 5

Bundle map

item	contents
`dataset_card.md`, `datasheet.md`, `croissant.json`	dataset documentation (human- and machine-readable)
`PROTOCOL_VERSION`	the frozen protocol stamp every reported number is tied to
`splits/`	official condition-complete split lists (64,931 / 23,363) per config and combined
`manifests/`	88,312-row per-record manifest, the 18-record exclusion list, the fixed 1,008-sample diagnostic subset
`annotations/`	record schema and the released bbox / canonical-crop convention
`examples/`	one worked record + the smoke test
`loaders/`	reference loader (`--root`, no hard-coded paths) and a hub streaming example
`evaluation/`	canonical FID/KID code, crop rule, uint8 round-trip assertion, compliance-evaluator protocol
`checksums/`	bundle checksums (relative paths)
`licenses/`	source-specific license matrix and notes
`reconstruction_scripts/`	recover source imagery from upstream via per-record `source_id`
`human_study/`	prepared-but-unexecuted 450-sample verification package
`errata.md`	errata channel, including the round-1 metric-defect disclosure

Task and protocol in one paragraph

The task is the rendering stage of a planning-then-rendering pipeline: a model receives the released prompt, full-scene depth, gesture label, hand bbox, hand depth, and pseudo-MANO mesh, and must synthesise the scene with a correct, realistic hand. Training uses the HaGRIDv1 portion only; HaGRIDv1 test is in-domain, HaGRIDv2 test is near-domain transfer (its training portion is released but unused), DexYCB is a hand-object OOD diagnostic, test-only. Realism (FID/KID, whole and hand-region) and conditional compliance (gesture top-1, detection rate, keypoint distance) are separate axes; see evaluation/semantic_eval_protocol.md.

Reference baseline

Exactly one model checkpoint accompanies the benchmark: HanDrawer-R2, trained from scratch in a single stage (single command; no historical checkpoint dependencies; ControlNet initialised from_unet, fresh hand encoder; seed 42; effective batch 24; lr 1e-5; 120k steps with checkpoints every 5k). The released checkpoint is the pre-registered selection: the step with the lowest whole-image FID on the in-domain test split, with the full checkpoint curve reported. The selected step and its numbers are stamped at release: step TBD-S\* (placeholder until the frozen selection rule resolves; no number in this bundle is provisional in any other way). A legacy architecture path is kept behind a documented switch (HD_ARCH_V2 unset) for exact reproduction of the round-1 implementation.

Citation (anonymous during review)

Anonymous. MANUS-Bench. Under double-blind review, WACV 2027 submission #1065. Cite the upstream sources (HaGRID/HaGRIDv2, DexYCB) for any subset you use.

Use restrictions

Research use for hand understanding and hand-aware generation. No biometric identification, person recognition, or surveillance uses. Subset licenses in licenses/ apply on top; DexYCB-derived data is non-commercial only.