Datasetpublic

ceil2/ArenaBench

ArenaBench Dataset Description ArenaBench is a large-scale benchmark designed for comprehensive evaluation of deepfake detection methods. It contains approximately 45K testing samples and evaluates detectors across five levels, including in-domain, cross-model, cross-manipulation, commercial models, and real-world scenarios. Overview Overview of the ArenaBench benchmark. Benchmark Design ArenaBench evaluates deepfake… See the full description on the dataset page: https://huggingface.co/datasets/ceil2/ArenaBench.

sourceHugging Faceunknownupdated 29d agoView on Hugging Face

2likes996downloads

Dataset Card

ArenaBench

Dataset Description

ArenaBench is a large-scale benchmark designed for comprehensive evaluation of deepfake detection methods. It contains approximately 45K testing samples and evaluates detectors across five levels, including in-domain, cross-model, cross-manipulation, commercial models, and real-world scenarios.

Overview

Overview of the ArenaBench benchmark.

Benchmark Design

ArenaBench evaluates deepfake detectors under five complementary settings:

Evaluation Track	Description
In-Domain	Evaluates performance on manipulation or generation domains seen during training.
Cross-Model	Evaluates generalization to unseen generative models.
Cross-Manipulation	Evaluates generalization to unseen facial manipulation techniques.
Commercial-Model	Evaluates detectors on images generated by recent commercial or closed-source generative models.
Real-world Scenarios	Evaluates robustness under realistic data distributions and practical deployment scenarios.

Dataset Structure

text

ArenaBench/
├── images/
│   ├── train/
|   |   ├── real/
|   |   |   └── *.png/
|   |   └── fake/
|   |   |   └── *.png/
│   └── test/
|   |   ├── In-Domain/
|   |   |   ├── FaceForensics++/
|   |   |   |   ├── 0_real/
|   |   |   |   |   └── *.png/
|   |   |   |   ├── 1_fake/
|   |   |   |   |   └── *.png/
|   |   |   ├── stylegan3/
|   |   |   |   ├── ...
|   |   |   ├── SDXL/
|   |   |   |   ├── ...
|   |   ├── Cross-Model/
|   |   |   ├── FLUX.2/
|   |   |   |   ├── 0_real/
|   |   |   |   |   └── *.png/
|   |   |   |   ├── 1_fake/
|   |   |   |   |   └── *.png/
|   |   |   ├── GRN/
|   |   |   |   ├── .../
|   |   |   ├── ...
|   |   ├── Cross-Manipulation/
|   |   |   ├── EchoMimic/
|   |   |   |   ├── 0_real/
|   |   |   |   |   └── *.png/
|   |   |   |   ├── 1_fake/
|   |   |   |   |   └── *.png/
|   |   |   ├── OSDFace/
|   |   |   |   ├── .../
|   |   |   ├── ...
|   |   ├── Commercial-Model/
|   |   |   ├── Gpt-Image-2/
|   |   |   |   ├── 0_real/
|   |   |   |   |   └── *.png/
|   |   |   |   ├── 1_fake/
|   |   |   |   |   └── *.png/
|   |   |   ├── NanoBananaPro/
|   |   |   |   ├── .../
|   |   |   ├── ...
|   |   ├── Real-world-Scenarios/
|   |   |   ├── ScaleDF/
|   |   |   |   ├── 0_real/
|   |   |   |   |   └── *.png/
|   |   |   |   ├── 1_fake/
|   |   |   |   |   └── *.png/
|   |   |   ├── MFFI/
|   |   |   |   ├── .../
|   |   |   ├── ...
├── annotations/
|   ├── train/
|   |   └── *.json/
|   └── test/
|   |   └── *.json/

Annotation Format

Each sample is represented as a JSON object:

json

{
  "images": [
    "ArenaBench/images/test/Cross_Model/FLUX.2/1_fake/image_0103.png"
  ],
  "type": "entire face generation",
  "label": 1
}

Download

The complete ArenaBench dataset can be downloaded using the Hugging Face CLI:

bash

hf download ceil2/ArenaBench --repo-type dataset --local-dir ArenaBench

Limitations and Responsible Use

ArenaBench is intended for academic research on deepfake detection and multimedia forensics. All commercial use is strictly prohibited. Users are responsible for complying with the licenses and terms of use of the original data sources.