Nephalem/llama-cpp-telegram_bot

Telegram Bot LLM

This project provides a simple Telegram bot using llama-cpp-python. It runs as a FastAPI service so it can be deployed easily to Hugging Face Spaces.

Directory structure

/app
├─ llamabot/
│   ├─ engine.py
│   ├─ telegram_ui.py
│   ├─ i18n.py
│   └─ settings.py
├─ main.py
├─ presets/
├─ characters/

Environment variables

• —TELEGRAM_TOKEN – bot token.
• —HISTORY_DIR_PATH – history directory (/tmp/history by default).
• —MODEL_CACHE_DIR – model cache directory (/tmp/model_cache).
• —LLAMA_THREADS – number of inference threads (defaults to CPU count).
• —LLAMA_BATCH – batch size (512 by default).
• —MLLOCK_DISABLED – disable mlock when true.

Running locally

Install dependencies and run the FastAPI app:

pip install -r requirements.txt
python app/main.py

For deployment on HF Spaces use the provided Dockerfile.