MrDevCoder01/DSBackend

1---2title: DSBackend3emoji: 🦀4colorFrom: red5colorTo: pink6sdk: docker7app_port: 78608license: mit9language:10- en11metrics:12- accuracy13library_name: tf-keras14pipeline_tag: image-classification15---16 17# Deepfake Detection Backend & Model (V1)18 19This repository contains a Convolutional Neural Network (CNN)-based model fine-tuned for deepfake classification, now wrapped in a high-performance **FastAPI** backend that natively supports processing both images and frame-by-frame videos.20 21## Core Advancements22To drastically improve real-world accuracy (especially on webcams and scaling distortions), we implemented **Ultralytics YOLO11-Pose** (`yolo11n-pose.pt`) for facial extraction. 23 24The underlying CNN (`model.h5`) excels only when evaluated on *tight facial crops* matching its training data. Generative YOLO bounding boxes are too loose and capture background noise. By extracting tracking keypoints (eyes, nose, ears) and explicitly drawing bounding configurations around them via YOLO11, we mathematically generate tight facial configurations, ensuring that the CNN captures exactly what it was trained to see, regardless of camera distance.25 26### Key Features:27- **Model Architecture:** Convolutional Neural Network (CNN)28- **Input Size:** 128x128 pixels (Tight facial crop)29- **Face Extractor:** Ultralytics YOLO11-Pose (`yolo11n-pose.pt`)30- **Video Processing:** Extracts and analyzes 1 in every 5 frames (~6 fps) for robust temporal spoof detection. Deepfake videos are flagged as "Fake" if *any* evaluated frame's prediction score exceeds 50%.31- **Number of Classes:** 2 (Real, Fake)32- **API Framework:** FastAPI, Uvicorn, Python-Multipart33 34## Processing Flow & Algorithm35 36The system natively processes both images and videos using a unified core prediction pipeline. The following describes the step-by-step logic.37 38### 1. Media Handling Flow39 40**For Images:**411. The image is parsed and decoded directly from the HTTP request.422. The image is passed to the **Core Prediction Pipeline**.433. A confidence score is returned, classifying the image as "Real" or "Fake".44 45**For Videos:**461. The video is saved to a temporary file and read using OpenCV.472. Frames are iteratively extracted.483. To optimize performance without sacrificing temporal accuracy, **1 in every 5 frames** (~6 FPS for a 30 FPS video) is analyzed.494. Each selected frame is individually passed to the **Core Prediction Pipeline**.505. The backend collects a list of `confidence_scores` from the analyzed frames.516. The video is flagged as "Fake" if the **maximum** confidence score among all frames (i.e., the most manipulated frame) exceeds 0.5.52 53### 2. Core Prediction Pipeline (Pseudocode)54 55To definitively locate and strictly frame the face, the YOLO11-Pose pipeline extracts 5 specific facial keypoints: **Nose, Left Eye, Right Eye, Left Ear, and Right Ear**.56 57```python58function process_frame(frame):59    # Step 1: Detect Face & Extract Keypoints (YOLO11-Pose)60    results = yolo_pose_model.predict(frame)61    62    if face_keypoints_found(results):63        # Eyes, nose, and ears detected64        bounding_box = calculate_tight_box_from_keypoints()65        face_crop = crop_image(frame, bounding_box)66    elif person_bounding_box_found(results): 67        # Fallback to standard object detection box if keypoints fail68        bounding_box = shrink_box_to_approximate_face()69        face_crop = crop_image(frame, bounding_box)70    else: 71        # Extreme fallback if no person is detected72        face_crop = frame73 74    # Step 2: Preprocessing75    resized_face = resize_image(face_crop, width=128, height=128)76    normalized_face = resized_face / 255.077    model_input = expand_dimensions(normalized_face)78 79    # Step 3: CNN Model Inference80    confidence_score = cnn_model.predict(model_input)81    82    return confidence_score83```84 85## Training Performance86 87Below are the graphs illustrating the training and validation accuracy and loss for the model:88 89![Model Training/Validation Graph 1](Unknown.png)90 91![Model Training/Validation Graph 2](Unknown-2.png)92 93## Installation94 951. Create a Python 3.11 virtual environment and activate it:96```bash97python3.11 -m venv venv98source venv/bin/activate99```1002. Install the required dependencies:101```bash102pip install -r requirements.txt103```104 105## Running the API Server106 107We provide a convenient startup script to launch the FastAPI backend:108```bash109chmod +x start_server.sh110./start_server.sh111```112The server will bind to `0.0.0.0:8000`, making the `/predict` endpoint available.113 114## Usage (API)115 116You can send a `POST` request with an image or video to the `/predict` endpoint using `multipart/form-data`:117 118```python119import requests120 121url = "http://localhost:8000/predict"122file_path = "sample_video.mp4" # Or an image.jpg123 124with open(file_path, "rb") as file:125    files = {"file": file}126    response = requests.post(url, files=files)127 128print(response.json())129```130 131**JSON Output Structure (Video):**132```json133{134  "filename": "sample_video.mp4",135  "type": "video",136  "prediction": "Fake",137  "confidence_score": 0.8921, 138  "frames_analyzed": 120,139  "fake_frames_count": 14,140  "max_fake_score": 0.8921,141  "avg_score": 0.3102 142}143```144*Note: A score closer to `1.0` is recognized as heavily manipulated. A score closer to `0.0` is authentic. An inference resulting in `max_fake_score` ≥ 0.5 triggers a "Fake" prediction limit.*145 146## Usage (Direct Python Inference)147 148If you'd like to use the YOLO11 inference pipeline directly in your Python code without the API server, feel free to adapt this minimal inference script:149 150```python151import cv2152import numpy as np153import warnings154from tensorflow.keras.preprocessing import image155from tensorflow.keras.models import load_model156from ultralytics import YOLO157 158warnings.filterwarnings('ignore', category=UserWarning)159 160# Load Models161model = load_model('model.h5', compile=False)162detector = YOLO('yolo11n-pose.pt')163 164def detect_and_predict(img_path):165    img = cv2.imread(img_path)166    167    # 1. Detect Face using YOLO11-Pose Keypoints168    results = detector.predict(img, verbose=False)169    if len(results) > 0 and results[0].keypoints is not None and len(results[0].keypoints.xy[0]) > 0:170        kpts = results[0].keypoints.xy[0].cpu().numpy()171        valid_kpts = np.array([k for k in kpts[0:5] if k[0] > 0 and k[1] > 0]) # Eyes, nose, ears172        173        if len(valid_kpts) > 0:174            x_min, y_min = np.min(valid_kpts, axis=0)175            x_max, y_max = np.max(valid_kpts, axis=0)176            177            # Expand tight box to capture full face (forehead to jaw)178            w, h = x_max - x_min, y_max - y_min179            if w > 0 and h > 0:180                x1 = max(0, int(x_min - w * 0.3))181                y1 = max(0, int(y_min - h * 0.5))182                x2 = min(img.shape[1], int(x_max + w * 0.3))183                y2 = min(img.shape[0], int(y_max + h * 0.8))184                185                face = img[y1:y2, x1:x2]186                if face.size > 0:187                    face = cv2.resize(face, (128, 128))188                    189                    # 2. Preprocess & Predict190                    img_array = np.expand_dims(image.img_to_array(face), axis=0) / 255.0191                    score = float(model.predict(img_array, verbose=0)[0][0])192                    193                    prediction = 'Fake' if score >= 0.5 else 'Real'194                    print(f"Prediction: {prediction} (Score: {score:.4f})")195                    return196                    197    print("Could not detect a clear face.")198 199# Try it out200detect_and_predict('path_to_your_image.jpg')201```202