Bagus-Sutha/fsof-deepfake-detection-ui

FSOF Deepfake Detection Dashboard

A Streamlit-based web application for detecting deepfake videos using Facial Sparse Optical Flow (FSOF) features and Mini-Xception classifiers. Developed as a Tugas Akhir (undergraduate thesis) project.

How It Works

The system tracks 68 facial landmarks across video frames and computes sparse optical flow vectors between consecutive frames. These flow signals are organized into fixed-length windows and fed into five binary classifiers — one per deepfake manipulation type — trained on the FaceForensics++ dataset.

Pipeline:

1. 1.Face detection and 68-point landmark tracking (face-alignment)
2. 2.Optical flow feature extraction (7 channels: dx, dy, magnitude, cos θ, sin θ, acc\x, acc\y)
3. 3.Windowed classification with 5 Mini-Xception models
4. 4.Aggregated REAL / MANIPULATED verdict with per-type confidence scores

Detectable manipulation types:

• —Deepfakes
• —Face2Face
• —FaceSwap
• —FaceShifter
• —NeuralTextures

Local Installation

Requirements

• —Python 3.9 – 3.11
• —Windows / Linux / macOS
• —CPU is sufficient (GPU optional but speeds up inference)

Steps

# 1. Clone the repository
git clone https://github.com/YOUR_USERNAME/YOUR_REPO_NAME.git
cd YOUR_REPO_NAME

# 2. Create and activate a virtual environment (recommended)
python -m venv venv

# Windows
venv\Scripts\activate

# Linux / macOS
source venv/bin/activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Run the app
streamlit run app.py

The app will open at http://localhost:8501.

First run note: face-alignment will automatically download its pretrained models (~175 MB) from the internet on first launch. This is a one-time download.

Windows-specific note

If you see a torch.compile crash on Windows (missing cl.exe), set this environment variable before running:

set TORCHDYNAMO_DISABLE=1
streamlit run app.py

This is already handled automatically inside app.py.

Project Structure

fsof_ui_app/
├── app.py                 # Streamlit entry point
├── fsof_extract.py        # Landmark tracking + optical flow extraction
├── inference.py           # Video inference pipeline
├── aggregate.py           # Multi-model verdict aggregation
├── history.py             # Classification history (saved as JSON)
├── models_config.py       # Model paths, thresholds, and channel configs
├── config.py              # App-level constants and paths
├── requirements.txt
├── models/                # Pretrained Mini-Xception models (.keras)
│   ├── deepfakes.keras
│   ├── face2face.keras
│   ├── faceshifter.keras
│   ├── faceswap.keras
│   └── neuraltextures.keras
└── src/                   # Supporting modules
    ├── schemas.py
    ├── state.py
    ├── ui_layout.py
    ├── video_io.py
    ├── visualization.py
    └── mock_inference.py

Usage

1. 1.Upload a video file (MP4, AVI, MOV, MKV)
2. 2.Adjust the sensitivity slider if needed
3. 3.Click Run Detection
4. 4.View the verdict (REAL / MANIPULATED), per-detector confidence scores, and frame-level timeline

Previously analyzed videos are cached and can be revisited from the History tab without re-running inference.

Dependencies

Notes

• —GPU acceleration: when a CUDA GPU is visible, face detection + 68-landmark extraction (PyTorch / face-alignment) runs per frame on the GPU, and the 5 Mini-Xception classifiers (TensorFlow) run as chunked batched predict calls. The active device is shown under the title. CPU fallback is automatic — local Windows runs unchanged. (TensorFlow is CPU-only here: tensorflow[and-cuda] clashes with torch over bundled CUDA libs, so the GPU is given to torch, which carries the real bottleneck.)
• —Inference time: CPU ≈ 3–10 minutes per video; on an HF T4-small GPU it drops substantially (per-frame GPU detection + GPU FAN landmarks).
• —HF Space auto-pause: the GPU Space uses a 600-second idle sleep (deploy_hf.py → request_space_hardware(..., sleep_time=600)), so GPU billing stops after ~10 min idle. It must exceed the longest single run — a too-short timer pauses the Space mid-prediction (one long request makes no new HTTP traffic), which shows as a restart. Use the Pause button in Settings to stop billing immediately.
• —The classification history is stored locally in results_history.json. On HuggingFace Spaces, history is persisted to a private HF Dataset repo so it survives container restarts. Recap frame images are ephemeral on HF Spaces (they exist only during the current session).
• —Mock mode is available for UI demonstration without running actual inference.