joyadevu/fallarmyworm_automated_detection

AgriShield: Edge-AI Fall Armyworm Detection Portal

Live demo: joyadevu/fallarmyworm_automated_detection

How to use the system: see `USAGE.md` (open the app, wait for Ready, drop a photo or video, read the sliders and the "What this means" panel).

A client-side computer vision tool for detecting Fall Armyworm (FAW) insects in maize crop photos. The entire inference pipeline - preprocessing, ONNX model execution, and Non-Maximum Suppression - runs in the browser. Crop photos are not uploaded to a server.

What the model can and cannot see. The ONNX graph is a single-class detector trained on the larva. It does not detect infested or chewed leaves, windowed patches, frass, or a damaged whorl. A photo of damage with no insect in frame will usually return zero boxes. The UI states this next to every result.

How It Works

Drop a photo or a video of a maize crop (file picker or drag-and-drop - there is no clipboard paste handler). Video is scored frame by frame with the same insect detector; WASM will skip frames while a run is in flight. Two sliders let you tune:

• —Confidence cutoff - minimum detection score (default 35%)
• —Overlap limit (IoU) - NMS threshold to suppress duplicate boxes (default 45%)

Threshold changes redraw immediately - no re-inference. A plain-language panel explains the count for a non-specialist and repeats the leaf-damage limit.

AI Pipeline

Input / Output

Preprocessing

Images are letterboxed (centered, 114-gray fill) to 640×640 preserving aspect ratio. Pixel values are divided by 255.0 and transposed from interleaved RGBA to planar NCHW float32.

Inference

ONNX Runtime Web 1.18.0 is vendored under vendor/onnxruntime-web/ (JS + WASM, including the WebGPU JSEP binary). Execution provider priority: WebGPU, then WASM. The model buffer is cached in IndexedDB after the first verified load - keyed by SHA-256 so a model update auto-invalidates the cache. Images themselves are never written to disk.

Offline behaviour (honest)

• —Served locally (python3 -m http.server): fonts, ORT, and model/model.onnx are all on disk. No third-party request after the page is served.
• —Hosted on Hugging Face Spaces: first visit downloads model.onnx and the WebGPU runtime from the Space itself (same origin). After SHA-256, the weights and runtime live in IndexedDB.
• —Opening the HTML as file:/// is not supported (WASM + fetch).

NMS

1. 1.Discard boxes below confidence threshold
2. 2.Sort remaining boxes by score descending
3. 3.Suppress boxes whose IoU with a higher-scoring box exceeds the overlap threshold
4. 4.Invert letterbox transform to map coordinates back to original image space

Running Locally

Must be served - browsers block file:/// for ONNX/WASM:

python3 -m http.server 8000
# open http://localhost:8000

Python / Gradio debug UI (loopback only, no public share link):

pip install -r requirements.txt
python app.py
# http://127.0.0.1:7860

Contract tests (stdlib only):

python -m unittest tests.test_pipeline

Files

Privacy & Security

• —Crop photos are not uploaded. The Gradio app binds 127.0.0.1 with share=False.
• —IndexedDB stores only the verified model bytes, never the farmer's image.
• —SHA-256 integrity check on every model load (or cache hit verification).
• —Full threat model: see SECURITY.md

Dissertation Note

"The trained YOLOv8 model was exported to ONNX format with input dimensions [1 × 3 × 640 × 640]. For privacy-preserving inference, the model was deployed client-side using a vendored ONNX Runtime Web 1.18.0. The runtime selects WebGPU where session creation succeeds, falling back to multithreaded WebAssembly. Preprocessing applies centered letterboxing before transposing the canvas pixel array to NCHW float32. Postprocessing applies confidence filtering and IoU-based Non-Maximum Suppression, then inverts the letterbox transform. The detector has one class - the Fall Armyworm insect - and does not score leaf damage. A separate plain-language panel translates the box count for a non-specialist without running a second model."