lalanikarim/ai-chatbot-conversational

Streamlit + Langchain + LLama.cpp w/ Mistral + Conversational Memory

Run your own AI Chatbot locally without a GPU.

To make that possible, we use the Mistral 7b model. However, you can use any quantized model that is supported by llama.cpp.

This model will chatbot will allow you to define it's personality and respond to the questions accordingly. This example remembers the chat history allowing you to ask follow up questions.

TL;DR instructions

1. 1.Install llama-cpp-python
2. 2.Install langchain
3. 3.Install streamlit
4. 4.Run streamlit

Step by Step instructions

The setup assumes you have python already installed and venv module available.

1. 1.Download the code or clone the repository.
2. 2.Inside the root folder of the repository, initialize a python virtual environment:

python -m venv venv

1. 1.Activate the python envitonment:

source venv/bin/activate

1. 1.Install required packages (langchain, llama.cpp, and streamlit):

pip install -r requirements.txt

1. 1.Start streamlit:

streamlit run main.py

1. 1.The models directory will be created and the app will download the Mistral7b quantized model from huggingface from the following link: mistral-7b-instruct-v0.1.Q4_0.gguf

Screenshot