NathanParas26/LocalAI

<h1 align="center"> <br> <img height="300" src="https://github.com/go-skynet/LocalAI/assets/2420543/0966aa2a-166e-4f99-a3e5-6c915fc997dd"> <br> LocalAI <br> </h1>

<p align="center"> <a href="https://github.com/go-skynet/LocalAI/fork" target="blank"> <img src="https://img.shields.io/github/forks/go-skynet/LocalAI?style=for-the-badge" alt="LocalAI forks"/> </a> <a href="https://github.com/go-skynet/LocalAI/stargazers" target="blank"> <img src="https://img.shields.io/github/stars/go-skynet/LocalAI?style=for-the-badge" alt="LocalAI stars"/> </a> <a href="https://github.com/go-skynet/LocalAI/pulls" target="blank"> <img src="https://img.shields.io/github/issues-pr/go-skynet/LocalAI?style=for-the-badge" alt="LocalAI pull-requests"/> </a> <a href='https://github.com/go-skynet/LocalAI/releases'> <img src='https://img.shields.io/github/release/go-skynet/LocalAI?&label=Latest&style=for-the-badge'> </a> </p>

:bulb: Get help - ❓FAQ 💭Discussions :speech_balloon: Discord :book: Documentation website 💻 Quickstart 📣 News 🛫 Examples 🖼️ Models


LocalAI is a drop-in replacement REST API that's compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families that are compatible with the ggml format. Does not require GPU.

<p align="center"><b>Follow LocalAI </b></p>

<p align="center"> <a href="https://twitter.com/LocalAIAPI" target="blank"> <img src="https://img.shields.io/twitter/follow/LocalAIAPI?label=Follow: LocalAIAPI&style=social" alt="Follow LocalAIAPI"/> </a> <a href="https://discord.gg/uJAeKSAGDy" target="blank"> <img src="https://dcbadge.vercel.app/api/server/uJAeKSAGDy?style=flat-square&theme=default-inverted" alt="Join LocalAI Discord Community"/> </a>

<p align="center"><b>Connect with the Creator </b></p>

<p align="center"> <a href="https://twitter.com/mudlerit" target="blank"> <img src="https://img.shields.io/twitter/follow/mudlerit?label=Follow: mudlerit&style=social" alt="Follow mudlerit"/> </a> <a href='https://github.com/mudler'> <img alt="Follow on Github" src="https://img.shields.io/badge/Follow-mudler-black?logo=github&link=https%3A%2F%2Fgithub.com%2Fmudler"> </a> </p>

<p align="center"><b>Share LocalAI Repository</b></p>

<p align="center">

<a href="https://twitter.com/intent/tweet?text=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.&url=https://github.com/go-skynet/LocalAI&hashtags=LocalAI,AI" target="blank"> <img src="https://img.shields.io/twitter/follow/LocalAI?label=Share Repo on Twitter&style=social" alt="Follow LocalAI"/></a> <a href="https://t.me/share/url?text=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.&url=https://github.com/go-skynet/LocalAI" target="blank"><img src="https://img.shields.io/twitter/url?label=Telegram&logo=Telegram&style=social&url=https://github.com/go-skynet/LocalAI" alt="Share on Telegram"/></a> <a href="https://api.whatsapp.com/send?text=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.%20https://github.com/go-skynet/LocalAI"><img src="https://img.shields.io/twitter/url?label=whatsapp&logo=whatsapp&style=social&url=https://github.com/go-skynet/LocalAI" /></a> <a href="https://www.reddit.com/submit?url=https://github.com/go-skynet/LocalAI&title=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally. " target="blank"> <img src="https://img.shields.io/twitter/url?label=Reddit&logo=Reddit&style=social&url=https://github.com/go-skynet/LocalAI" alt="Share on Reddit"/> </a> <a href="mailto:?subject=Check%20this%20GitHub%20repository%20out.%20LocalAI%20-%20Let%27s%20you%20easily%20run%20LLM%20locally.%3A%0Ahttps://github.com/go-skynet/LocalAI" target="blank"><img src="https://img.shields.io/twitter/url?label=Gmail&logo=Gmail&style=social&url=https://github.com/go-skynet/LocalAI"/></a> <a href="https://www.buymeacoffee.com/mudler" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="23" width="100" style="border-radius:1px"></a>

</p>

<hr>

In a nutshell:

• —Local, OpenAI drop-in alternative REST API. You own your data.
• —NO GPU required. NO Internet access is required either
• —Optional, GPU Acceleration is available in llama.cpp-compatible LLMs. See also the build section.
• —Supports multiple models
• —🏃 Once loaded the first time, it keep models loaded in memory for faster inference
• —⚡ Doesn't shell-out, but uses C++ bindings for a faster inference and better performance.

LocalAI was created by Ettore Di Giacinto and is a community-driven project, focused on making the AI accessible to anyone. Any contribution, feedback and PR is welcome!

Note that this started just as a fun weekend project in order to try to create the necessary pieces for a full AI assistant like ChatGPT: the community is growing fast and we are working hard to make it better and more stable. If you want to help, please consider contributing (see below)!

🔥🔥 Hot topics / Roadmap

🚀 Features

• —📖 Text generation with GPTs (llama.cpp, gpt4all.cpp, ... :book: and more)
• —🗣 Text to Audio
• —🔈 Audio to Text (Audio transcription with whisper.cpp)
• —🎨 Image generation with stable diffusion
• —🔥 OpenAI functions 🆕
• —🧠 Embeddings generation for vector databases
• —✍️ Constrained grammars
• —🖼️ Download Models directly from Huggingface

:book: 🎥 Media, Blogs, Social

• —Create a slackbot for teams and OSS projects that answer to documentation
• —LocalAI meets k8sgpt
• —Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All
• —Tutorial to use k8sgpt with LocalAI

💻 Usage

Check out the Getting started section in our documentation.

💡 Example: Use GPT4ALL-J model

See the documentation

🔗 Resources

• —How to build locally
• —How to install in Kubernetes
• —Projects integrating LocalAI

❤️ Sponsors

Do you find LocalAI useful?

Support the project by becoming a backer or sponsor. Your logo will show up here with a link to your website.

A huge thank you to our generous sponsors who support this project:

🌟 Star history


📖 License

LocalAI is a community-driven project created by Ettore Di Giacinto.

MIT - Author Ettore Di Giacinto

🙇 Acknowledgements

LocalAI couldn't have been built without the help of great software already available from the community. Thank you!

• —llama.cpp
• —https://github.com/tatsu-lab/stanford_alpaca
• —https://github.com/cornelk/llama-go for the initial ideas
• —https://github.com/antimatter15/alpaca.cpp
• —https://github.com/EdVince/Stable-Diffusion-NCNN
• —https://github.com/ggerganov/whisper.cpp
• —https://github.com/saharNooby/rwkv.cpp
• —https://github.com/rhasspy/piper
• —https://github.com/cmp-nct/ggllm.cpp

🤗 Contributors

This is a community project, a special thanks to our contributors! 🤗 <a href="https://github.com/go-skynet/LocalAI/graphs/contributors"> <img src="https://contrib.rocks/image?repo=go-skynet/LocalAI" /> </a>