Datasetpublic

muset-ai/MPIE-Bench

MPIE-Bench Official 2,500-sample test set for multi-person interaction-aware image editing evaluation. GitHub (code + protocol): https://github.com/AnnLin0628/mpie-bench Org: muset-ai Dataset Viewer The default config (default / test) is one row per evaluation sample: Column Meaning cat Interaction category (filter / group by this) gt Held-out ground-truth image prompt Edit instruction ref_paths Reference image paths under images/ sample_id… See the full description on the dataset page: https://huggingface.co/datasets/muset-ai/MPIE-Bench.

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

0likes1.2kdownloads

Dataset Card

MPIE-Bench

Official 2,500-sample test set for multi-person interaction-aware image editing evaluation.

—GitHub (code + protocol): https://github.com/AnnLin0628/mpie-bench
—Org: muset-ai

Dataset Viewer

The default config (`default` / `test`) is one row per evaluation sample:

Column	Meaning
`cat`	Interaction category (filter / group by this)
`gt`	Held-out ground-truth image
`prompt`	Edit instruction
`ref_paths`	Reference image paths under `images/`
`sample_id`	Stable sample id

Use the `cat` column in the viewer to browse by large category (hug, dance, wrestle_grapple, …).

If you still see an ImageFolder subset in the dropdown, switch to `default`. That auto-view treats every file under images/<cat>/<scene>/ as a separate row and is not the benchmark unit.

Repository layout

text

data/test.parquet   # viewer table (2500 samples; GT paths resolve into images/)
manifest.jsonl      # original pack manifest
pack_meta.json
images/<cat>/<id>/  # R* reference crops + GT_*.jpg
instr_qa_v2/        # frozen Instruction QA bank
LICENSE             # multi-source data terms

Load in Python

python

from datasets import load_dataset

ds = load_dataset("muset-ai/MPIE-Bench", split="test")  # default config
print(ds)  # 2500 rows
# `cat` is a ClassLabel — compare via int2str, or filter by id
hug = ds.filter(lambda x: ds.features["cat"].int2str(x["cat"]) == "hug")

For the raw evaluation pack (paths relative to a local checkout):

bash

huggingface-cli download muset-ai/MPIE-Bench --repo-type dataset --local-dir ./MPIE-Bench
export MPIE_TEST_PACK=$PWD/MPIE-Bench

Scoring: see the GitHub repo (code/eval/run_eval_e2e.sh).

License

Multi-source data terms — see `LICENSE`. Not a single CC/Apache grant for the whole pack. Obey each source’s original terms.