jli43/drone-detection-frames

UAV Drone Detection and Tracking

This project implements a UAV drone detection and tracking pipeline using a deep learning object detector and a Kalman filter. The system detects drones in video frames and tracks their motion across time while visualizing the drone’s trajectory.

The goal of the project is to demonstrate multi-object tracking concepts using probabilistic filtering and modern computer vision models.

Detector Configuration

The detector used in this project is YOLOv8 (Ultralytics).

YOLOv8 was chosen because it provides:

• —strong real-time object detection performance
• —easy fine-tuning using custom datasets

Training Settings

• —model: yolov8n
• —image size: 640
• —epochs: 10
• —batch size: 16

The detector processes frames extracted from video at 5 FPS, which balances detection performance with computational efficiency.

Each frame is passed through the model and any drone detections are recorded.

Frames containing at least one detection are saved to the detections/ directory.

Kalman Filter Tracking

Drone tracking is implemented using the FilterPy library. A Kalman filter is used to estimate the drone’s position across frames.

State Vector

[x, y, vx, vy]

Where:

• —x, y = center position of the drone bounding box
• —vx, vy = estimated velocity of the drone

This allows the tracker to predict motion between frames.

Tracking Process

For each frame:

1. 1.The Kalman filter predicts the next state.
2. 2.If a detection is present, the filter updates using the detected bounding box center.
3. 3.If the detector misses the drone, the filter continues predicting its location for a limited number of frames.

This approach provides smoother trajectories and allows the tracker to handle temporary missed detections.

Trajectory Visualization

The tracker records the estimated center of the drone across frames. A 2D trajectory polyline is drawn on the output video showing the path of the drone through time.

Each output frame overlays:

• —the detector bounding box
• —the Kalman-filter estimated center
• —the trajectory polyline connecting previous positions

Output Videos and Dataset

Video 1 https://youtu.be/yWBAPU6xFlw

Video 2 https://youtu.be/YebJyiWLUCs

Dataset https://huggingface.co/datasets/jli43/drone-detection-frames

Code

The processing pipeline is designed to work on any directory of .mp4 files.

The script automatically loops through all videos in the specified folder and applies:

1. 1.frame extraction
2. 2.drone detection
3. 3.Kalman filter tracking
4. 4.trajectory visualization
5. 5.output video generation

This ensures the system can be easily applied to additional drone footage.

Failure Cases

Some challenges remain in drone detection and tracking:

• —Small drones at long distances may be difficult to detect.
• —Motion blur or fast movement can reduce detector confidence.
• —Background clutter may occasionally cause false detections.
• —Temporary missed detections occur when the drone moves quickly or leaves the frame.

The Kalman filter helps mitigate these issues by continuing to predict the drone’s position during short detection gaps.