loleg/fastapi-apertus

Apertus transformer on FastAPI ------------------------------

A FastAPI-based Python application that provides an API to interface with the Apertus LLM from the Swiss AI Initiative.

The OpenAI-compatible API is inspired by openai-compatible-fastapi.

The goal is to use the Apertus Format API in a lightweight service with an efficient, paged attention mechanism.

If you're debugging locally, it may help you to use tiny-random/apertus instead of the full model.

Terms of use

For more information on Apertus, go to https://huggingface.co/swiss-ai

This is an open source project under the Apache 2.0 License

If you have further suggestions please leave them on Codeberg Issues