Yousefxp8/ml-inference-api

1---2title: ML Inference API3emoji: 🎭4colorFrom: blue5colorTo: purple6sdk: docker7app_port: 78608pinned: false9---10 11# ML Inference API12 13**[Live Demo / Swagger Docs: https://Yousefxp8-ml-inference-api.hf.space/docs](https://Yousefxp8-ml-inference-api.hf.space/docs)**14 15A Dockerized REST API for celebrity face recognition, built with FastAPI, Celery, Redis, and a transfer-learned ResNet50 model. Image predictions are queued as background tasks, results can be polled by task ID, and Redis-backed metrics report request and inference activity.16 17## Features18 19- Celebrity recognition using a fine-tuned ResNet50 model20- Asynchronous prediction jobs backed by Celery and Redis21- Single-image and batch prediction endpoints22- Pollable job status and result endpoints23- Shared Redis-backed inference metrics24- Configurable thread-based inference concurrency through Docker Compose25 26## Architecture27 28```text29Client30  |31  | POST /predict or /predict-batch32  v33FastAPI API  ---- enqueue task ---->  Redis broker/result backend34  ^                                      |35  | GET /job/{task_id}                  | task delivery/result storage36  | GET /metrics                         v37  +-------------------------------  Celery worker38                                      |39                                      v40                              ResNet50 inference41```42 43The API returns a Celery `task_id` after submission. The client polls the job endpoint until Celery reports `SUCCESS` or `FAILURE`. The worker uses a thread pool because PyTorch inference runs correctly with the loaded model in that configuration, while a prefork worker caused tasks to remain pending in this environment.44 45## Endpoints46 47| Method | Endpoint         | Description                                                      |48| ------ | ---------------- | ---------------------------------------------------------------- |49| `POST` | `/predict`       | Queue one uploaded image and return a `task_id`                  |50| `POST` | `/predict-batch` | Queue multiple images and return their task IDs                  |51| `GET`  | `/job/{task_id}` | Return Celery status and a completed prediction result           |52| `POST` | `/jobs-status`   | Return status/result data for multiple task IDs                  |53| `GET`  | `/metrics`       | Return total requests, average inference latency, and queue size |54 55Example completed result:56 57```json58{59  "task_id": "task-uuid",60  "status": "SUCCESS",61  "result": {62    "class_id": 0,63    "label": "pins_Adriana Lima",64    "latency": 0.09965  }66}67```68 69## Model70 71| Property          | Value                                                 |72| ----------------- | ----------------------------------------------------- |73| Architecture      | ResNet50                                              |74| Training method   | Transfer learning with a replaced classification head |75| Output classes    | 105 celebrities                                       |76| Runtime artifacts | `weights/celeb.pth`, `weights/labels.json`            |77| Inference mode    | `model.eval()` with `torch.no_grad()`                 |78 79Training and validation data are expected under `trianing/train/` and `trianing/val/`. Running `trianing/train.py` writes the model and label artifacts used by the API into `weights/`.80 81## Run With Docker82 83Start the API, Redis broker/backend, and Celery worker:84 85```bash86docker compose up -d --build87```88 89Open the API documentation at:90 91```text92http://127.0.0.1:8000/docs93```94 95Check the running services:96 97```bash98docker compose ps99docker compose logs --tail=30 worker100```101 102The worker configuration is defined in `docker-compose.yml`:103 104```yaml105command: >106  celery -A app.workers.inference_worker worker107  --pool=threads108  --concurrency=8109  --loglevel=info110```111 112To test another concurrency level, change `--concurrency`, recreate the worker, and run the concurrent benchmark:113 114```bash115docker compose up -d --force-recreate worker116python -u tests/test_concurrent.py117```118 119Keep `--pool=threads` for the current model-loading approach. A `prefork` pool was tested and caused queued tasks to hang in this environment.120 121## Hugging Face Space122 123This repository is also configured as a Docker Space. Hugging Face builds the `Dockerfile` and exposes the FastAPI application on port `7860`; `start-space.sh` launches an internal Redis instance and Celery thread worker before starting Uvicorn.124 125The Space endpoint is:126 127```text128https://Yousefxp8-ml-inference-api.hf.space/docs129```130 131Local Docker Compose remains separate from the Space entrypoint: Compose runs API, Redis, and worker as individual services, while the Space runs them inside its single container.132 133## Training134 135Install Python dependencies and prepare data inside `trianing/train/`, with one directory per celebrity. Then run:136 137```bash138python trianing/split_dataset.py139python trianing/train.py140```141 142The scripts create/use `trianing/val/` and write updated serving artifacts to `weights/`.143 144## Benchmarks145 146The concurrent benchmark submitted 20 images through `/predict` and polled `/job/{task_id}` until completion. It was measured with one Celery worker container using the thread pool and different `--concurrency` values.147 148| Worker Threads | Total Wall Time | Average Latency | Fastest User | Slowest User |149| -------------: | --------------: | --------------: | -----------: | -----------: |150|              1 |           6.38s |           3.24s |        0.20s |        6.33s |151|              2 |           1.94s |           1.07s |        0.31s |        1.94s |152|              4 |           1.07s |           0.68s |        0.32s |        1.03s |153|              8 |           0.95s |           0.64s |        0.43s |        0.89s |154 155The largest gains occur from 1 to 4 worker threads. Increasing from 4 to 8 reduced total wall time by `0.12s`, showing diminishing returns for this 20-request workload.156 157## Project Structure158 159```text160.161|-- main.py162|-- docker-compose.yml163|-- Dockerfile164|-- requirements.txt165|-- app/166|   |-- core/167|   |   |-- celery_app.py168|   |   `-- state.py169|   |-- models/170|   |   `-- celeb_model.py171|   |-- routes/172|   |   |-- jobs.py173|   |   |-- metrics.py174|   |   `-- predict.py175|   |-- services/176|   |   |-- inference_service.py177|   |   `-- metrics_service.py178|   `-- workers/179|       `-- inference_worker.py180|-- tests/181|   |-- test_batch.py182|   `-- test_concurrent.py183|-- trianing/184|   |-- split_dataset.py185|   `-- train.py186`-- weights/187    |-- celeb.pth188    `-- labels.json189```190