ShahanMalik/lip_reading

Lip Reading API (JSON)

This project is now API-only (no Gradio UI).

It runs a Visual Speech Recognition pipeline:

Video -> Face/Lip processing -> Auto-AVSR model -> Predicted text

Endpoints

• —GET / : service info
• —GET /health : health check
• —GET /post-format : JSON describing POST format
• —POST /predict : run lip reading from a base64 video payload

Required JSON POST Format

POST /predict

Content-Type: application/json

Do not send multipart/form-data to this endpoint.

{
  "video_base64": "<base64_encoded_video_bytes>",
  "filename": "sample.mp4"
}

If you send the wrong body type, the API returns HTTP 422 with a helpful error message.

Sample Response

{
  "ok": true,
  "text": "hello how are you",
  "status": "Done in 2.1s | Frames: 96 | Device: CPU"
}

Python Example

import base64
import requests

api_url = "https://YOUR_SPACE_URL/predict"

with open("video.mp4", "rb") as f:
    payload = {
        "video_base64": base64.b64encode(f.read()).decode("utf-8"),
        "filename": "video.mp4",
    }

response = requests.post(api_url, json=payload, timeout=300)
print(response.status_code)
print(response.json())

Model

Powered by Auto-AVSR: https://github.com/mpc001/auto_avsr

• —Model: vsrtrlrs3vox2base.pth
• —Trained on: LRS3 + VoxCeleb2
• —Architecture: ResNet-18 frontend + Conformer backend