MUmairAB/Spatiotemporal-DeepFake-Detection

Deepfake Detection - Hybrid CNN-Transformer

Live demo for our Deep Machine Learning research project on video-level deepfake detection. Upload a short face video and the proposed Hybrid CNN-Transformer will predict whether it is real or manipulated, and show a Grad-CAM heatmap of which regions of the face the model is looking at.

The model combines an EfficientNet-B3 spatial backbone with a 4-layer Transformer encoder over a sequence of 24 frame features. It was trained on FaceForensics++ (C23) on five manipulation classes (original, Deepfakes, Face2Face, FaceSwap, NeuralTextures). FaceShifter was held out for the cross-manipulation generalization test.

Notes for use

• —The Space runs on a free CPU tier, so a single video takes about 30 - 60 seconds end to end.
• —Short clips work best (under 10 seconds). The pipeline samples 24 evenly spaced frames regardless of clip length.
• —The video must contain a clearly visible face. If MTCNN cannot find any face in the sampled frames the demo will error out.

Model weights

Weights are downloaded automatically from the `MUmairAB/deepfake-detection-ff-cn-transformer` repository the first time the Space starts.