The task is to create a web app that takes image as an input and predict the objects present in that image.
It's nothing but an Object Detection.
First let's start working on the UI designs
Am going with Streamlit as it is super easy and have multiple pre defined components that I can use like buttons, File uploads, loaders etc.
Let's first add a title & subtitle to the web page which gives a basic info about our web page. Like, Object Detection (Detect objects from an image using simple lightweight models).
Then we add a file uploader, which takes the images only as input. We can restrict the formats (PNG only) but let's just allow all types of images for time being.
Once an image is uploaded in the file uploader, the user can start calling the backend to detect the objects from the uploaded image.
Thus we need a detection model, as mentioned in the technical assessment let's not just use any one of the Lightweight detection models like YOLO or MobileNet SSD, instead let's use the model selected by the user.
Thus the user must be provided a list of options to choose the model to detect the objects present in the image. Let's add buttons in streamlit for choosing the model
Once the model is selected, we can display the detect button. On clicking which the API call will happen.
Now let's dive into the designing of backend
Let's first create an endpoint named /detect which will be called from frontend.
Now use the model that the user selected from frontend and initialize the model.
We are clear that the payload that we send to this endpoint is nothing but an image from which we need to detect the objects. Thus let's read this image first.
Let's use the user selected model to predict the classes or objects in the image with confidence scores & bounding boxes.
Let the response object/JSON have the following keys or fields.
CLASS, CONFIDENCE, BOUNDING_BOX
Let's keep appending these 3 fields for all the classes that are identified from the image and send it as a final json object
OBJECTS (Final JSON response)
Now let's come back to UI
Where were we? Oh yeah! We finished our File Upload, models buttons and detect button. Now, let's handle our API response.
So we don't get the image back from the response and we really don't need it as well as we already are storing or having it when the user uploads.
Thus, let's just use the same image to display the user of what are all classes identified from the uploaded image. Remembed that we got the bounding box locations as well as confidences along with the classes.
Nice! Let's start using the response. Assuming that the API is working and you have a proper Internet, you received the response back from the detect API that we have created.
We first draw a rectangle around the clasified object and then display the confidence scores. We can use opencv as well, but let's just keep it simple and use PIL / Pillow (Not the one you use to sleep on)
Once we get the bounding box coordinates, we can draw the rectangle with it's class name as text and confidence score.
Now return that image to the user along with the response as well if clicked on View Response button.
