Datasetpublic

EstherrrCheng/mea-benchmark

MEA-Benchmark A benchmark dataset for evaluating explainability of neural network models across three modalities (tabular, vision, text) with ten question types (Q1–Q10). Dataset Structure Each split (train, test) is organized by modality and then by {dataset}_{model}_{q_type}.json: {split}/ ├── tabular/ │ ├── adult_2layernn_q1.json │ ├── adult_tabnn_q1.json │ ├── cancer_2layernn_q1.json │ ├── cancer_tabnn_q1.json │ └── ... ├── text/ │ ├──… See the full description on the dataset page: https://huggingface.co/datasets/EstherrrCheng/mea-benchmark.

sourceHugging Facecc-by-4.0updated 1mo agoView on Hugging Face

0likes74downloads

Dataset Card

MEA-Benchmark

A benchmark dataset for evaluating explainability of neural network models across three modalities (tabular, vision, text) with ten question types (Q1–Q10).

Dataset Structure

Each split (train, test) is organized by modality and then by {dataset}_{model}_{q_type}.json:

{split}/
├── tabular/
│   ├── adult_2layernn_q1.json
│   ├── adult_tabnn_q1.json
│   ├── cancer_2layernn_q1.json
│   ├── cancer_tabnn_q1.json
│   └── ...
├── text/
│   ├── imdb_2layernn_q1.json
│   ├── imdb_cnn_q1.json
│   ├── snli_2layernn_q1.json
│   ├── snli_cnn_q1.json
│   └── ...
└── vision/
    ├── cub_densenet_q1.json
    ├── cub_resnet_q1.json
    ├── stl10_densenet_q1.json
    ├── stl10_resnet_q1.json
    └── ...

Fields

Field	Description
`row_no`	Sample index
`image_path`	Path to input image (vision only)
`modality`	`tabular`, `text`, or `vision`
`dataset`	Dataset name (e.g. `adult`, `cancer`, `imdb`, `snli`, `cub`, `stl10`)
`model`	Model architecture (e.g. `2layernn`, `tabnn`, `cnn`, `resnet`, `densenet`)
`features`	Input features or tokens
`target`	Ground truth label
`predicted`	Model prediction
`example`	Raw input example
`q_type`	Question type (`q1`–`q10`)
`q`	The XAI question prompt

Question Types

—Q1 Most responsible feature
—Q2 Least responsible feature
—Q3 Distinctive feature
—Q4 Contrastive instances
—Q5 Mask prediction
—Q6 Flip prediction
—Q7 Change prediction
—Q8–Q10 Additional reasoning questions