buildborderless/CommunityForensics-DeepfakeDet-ViT

CommunityForensics DeepfakeDet-ViT

Vision Transformer (ViT-Small) trained on 2.7M samples across 4,803 generators for detecting AI-generated images. Presented in Community Forensics: Using Thousands of Generators to Train Fake Image Detectors (CVPR 2025).

Uploaded for community validation as part of OpenSight — An upcoming open-source framework for adaptive deepfake detection.

Project OpenSight HF Spaces coming soon with an eval playground and eventually a leaderboard. Preview:

IMPORTANT — Configuration Fix (July 2026)

If you downloaded this model before July 22, 2026, your local copy has incorrect config and weights. Apologies for the mess — this model was originally hastily put together as an internal proof-of-concept for a hackathon, and we never imagined it would quietly become one of the top image classification models on Hugging Face. This update is long overdue.

The model.safetensors has been regenerated from the correct training checkpoint and all metadata has been fixed. For a detailed breakdown of every change, see CHANGELOG.md. If you use LLM-based coding agents (Claude Code, Cursor, GitHub Copilot, etc.), the repo includes an AGENTS.md to help your agent ramp up quickly.

⚠️ Breaking change for older transformers versions

This model now requires transformers >= 5.4.0 for correct image preprocessing. Versions older than 5.4.0 will crash with a ValueError when loading the preprocessor — this is intentional and prevents silently-squashed images. If upgrading is not an option, you can preprocess images manually (resize shortest edge → 440, center-crop → 384, CLIP-normalize) and pass do_resize=False to the processor.

How to verify you have the fix

import json
with open("path/to/config.json") as f:
    cfg = json.load(f)
assert cfg["num_labels"] == 1, "Still broken — re-download the model"
assert cfg["num_attention_heads"] == 6, "Still broken — re-download the model"
assert cfg["intermediate_size"] == 1536, "Still broken — re-download the model"

If you were using the old custom wrapper (modeling_vit_classifier.py)

It has been moved to scripts/ and marked deprecated. Switch to the standard HuggingFace path:

from transformers import ViTForImageClassification, ViTImageProcessor
model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")

If you were using the ONNX repo

The separate `buildborderless/CommunityForensics-DeepfakeDet-ViT-ONNX` repo is now deprecated. All ONNX models are included here in onnx/ with corrected weights. Old exports are archived in onnx_legacy/.

Archived files

• —model_legacy.safetensors — previous (incorrect) weights, frozen for reference
• —model_fixed.safetensors — identical copy of the current model.safetensors
• —onnx_legacy/ — previous ONNX exports from the incorrect weights

Quick Start

from transformers import ViTForImageClassification, ViTImageProcessor
from PIL import Image
import torch

model = ViTForImageClassification.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")
processor = ViTImageProcessor.from_pretrained("buildborderless/CommunityForensics-DeepfakeDet-ViT")

image = Image.open("suspicious_image.jpg")
inputs = processor(image, return_tensors="pt")
outputs = model(**inputs)

fake_prob = torch.sigmoid(outputs.logits).item()
print(f"fake: {fake_prob:.4f}, real: {1 - fake_prob:.4f}")
print(f"verdict: {'fake' if fake_prob > 0.5 else 'real'}")

Dependencies

• —transformers >= 5.4.0 (required — older versions lack shortest_edge resize and will crash. Do not downgrade below 5.4.0 or images will be silently squashed.)
• —torch, torchvision, Pillow
• —onnxruntime >= 1.27 (for ONNX models — install onnxruntime for CPU or onnxruntime-gpu for GPU)

ONNX Variants (v1.1)

Five pre-exported ONNX models with different size/speed trade-offs. All use the corrected config (single-class sigmoid output).

Which variant should I use?

Quantization note: Dynamic per-tensor quantization without calibration causes quantized variants to diverge from FP32 on certain inputs (up to 10–70 percentage points) — particularly images from generators outside the training set. Significant disagreement between FP32 and INT8/Q4 indicates the input is near the model's decision boundary or out-of-distribution. For maximum single-model consistency, use model.onnx (FP32).

import onnxruntime as ort, numpy as np
from PIL import Image

session = ort.InferenceSession("onnx/model_int8.onnx")

# Preprocess: shortest edge → 440 (maintain aspect ratio), center-crop → 384, CLIP normalize
image = Image.open("image.jpg")
w, h = image.size
scale = 440 / min(w, h)
img = image.resize((int(w * scale), int(h * scale)))
left = (img.size[0] - 384) // 2
top = (img.size[1] - 384) // 2
img = img.crop((left, top, left + 384, top + 384))
arr = np.array(img, dtype=np.float32) / 255.0
arr = (arr - np.array([0.4815, 0.4578, 0.4082])) / np.array([0.2686, 0.2613, 0.2758])
arr = np.expand_dims(arr.transpose(2, 0, 1), 0)

logit = session.run(None, {"pixel_values": arr})[0][0, 0]
fake_prob = 1 / (1 + np.exp(-logit))

Benchmark & Comparison Space

A companion Gradio Space lets you test every variant side by side — upload your own images and compare PyTorch vs ONNX performance in real time.

What it does:

Use it to:

• —See how quantization affects prediction confidence on your own images
• —Measure real-world inference speed across variants (CPU/GPU)
• —Verify the corrected model produces results consistent with the original timm pipeline

Link coming soon — deploying as a separate Space. Follow the repo for updates.

Model Details

• —Developed by: Jeongsoo Park and Andrew Owens, University of Michigan
• —HF integration + ONNX: Han Yoon, Borderless / Ethix R&D
• —Model type: Vision Transformer (ViT-Small)
• —License: MIT
• —Input: RGB image, shortest edge resized to 440 (aspect ratio preserved), center-cropped to 384×384, CLIP-normalized
• —Output: single logit → sigmoid → fake probability
• —Architecture: hiddensize=384, 6 attention heads, 12 layers, patchsize=16, intermediate_size=1536

Links

• —Original paper: arXiv:2411.04125
• —Original repository: JeongsooP/Community-Forensics
• —Project page: https://jespark.net/projects/2024/community_forensics
• —Datasets: Full (1.1TB), Small (278GB), Eval (206GB)

Coming Soon — v2

We're actively working on a significantly stronger model with an expanded dataset and novel detection concepts. Follow the repo for updates in the coming months.

Citation

@InProceedings{Park_2025_CVPR,
    author    = {Park, Jeongsoo and Owens, Andrew},
    title     = {Community Forensics: Using Thousands of Generators to Train Fake Image Detectors},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
    month     = {June},
    year      = {2025},
    pages     = {8245-8257}
}