anslem19/llm-fastapi-boilerplate

⚡ LLM FastAPI Boilerplate

A robust backend service for deploying GGUF language models using FastAPI and Docker.

🎯 Overview

This boilerplate provides a production-ready API for serving Large Language Models, with focus on type safety, easy deployment, and standardized interfaces.

✨ Key Features

• —Type-Safe Backend: FastAPI with comprehensive request/response typing
• —Model Support: GGUF format via llama-cpp-python
• —Production Ready: CORS support, health checks, and Docker deployment
• —Flexible: Configurable inference parameters (temperature, topp, topk)
• —Developer Friendly: OpenAPI documentation included

📂 Project Structure

.
├── app/
│   ├── main.py        # FastAPI application and routes
│   ├── schemas.py     # Pydantic data models
│   ├── llm_utils.py   # LLM inference engine
│   └── __init__.py
├── requirements.txt   # Dependencies
├── Dockerfile        # Container configuration
└── README.md

🚀 Getting Started

Local Development

git clone https://github.com/Anslemakadu/llm-fastapi-boilerplate
cd llm-fastapi-boilerplate
pip install -r requirements.txt
uvicorn app.main:app --reload --host 0.0.0.0 --port 7860

API documentation available at: http://localhost:7860/docs

Docker Deployment

docker build -t llm-fastapi-boilerplate .
docker run -p 7860:7860 llm-fastapi-boilerplate

🔗 API Endpoints

⚙️ Configuration

📝 Example Usage

curl -X POST "https://anslem19-llm-fastapi-boilerplate.hf.space/chat" \
-H "Content-Type: application/json" \
-d '{
  "session_id": "demo",
  "message": "Hello LLM Boilerplate!"
}'

📄 License

MIT License © 2025 Anslem Akadu