Technologic101/AIE5-RAG

1# Building a Chainlit App2 3What if we want to take our Week 1 Day 2 assignment - [Pythonic RAG](https://github.com/AI-Maker-Space/AIE4/tree/main/Week%201/Day%202) - and bring it out of the notebook?4 5Well - we'll cover exactly that here!6 7## Anatomy of a Chainlit Application8 9[Chainlit](https://docs.chainlit.io/get-started/overview) is a Python package similar to Streamlit that lets users write a backend and a front end in a single (or multiple) Python file(s). It is mainly used for prototyping LLM-based Chat Style Applications - though it is used in production in some settings with 1,000,000s of MAUs (Monthly Active Users).10 11The primary method of customizing and interacting with the Chainlit UI is through a few critical [decorators](https://blog.hubspot.com/website/decorators-in-python).12 13> NOTE: Simply put, the decorators (in Chainlit) are just ways we can "plug-in" to the functionality in Chainlit. 14 15We'll be concerning ourselves with three main scopes:16 171. On application start - when we start the Chainlit application with a command like `chainlit run app.py`182. On chat start - when a chat session starts (a user opens the web browser to the address hosting the application)193. On message - when the users sends a message through the input text box in the Chainlit UI20 21Let's dig into each scope and see what we're doing!22 23## On Application Start:24 25The first thing you'll notice is that we have the traditional "wall of imports" this is to ensure we have everything we need to run our application. 26 27```python28import os29from typing import List30from chainlit.types import AskFileResponse31from aimakerspace.text_utils import CharacterTextSplitter, TextFileLoader32from aimakerspace.openai_utils.prompts import (33    UserRolePrompt,34    SystemRolePrompt,35    AssistantRolePrompt,36)37from aimakerspace.openai_utils.embedding import EmbeddingModel38from aimakerspace.vectordatabase import VectorDatabase39from aimakerspace.openai_utils.chatmodel import ChatOpenAI40import chainlit as cl41```42 43Next up, we have some prompt templates. As all sessions will use the same prompt templates without modification, and we don't need these templates to be specific per template - we can set them up here - at the application scope. 44 45```python46system_template = """\47Use the following context to answer a users question. If you cannot find the answer in the context, say you don't know the answer."""48system_role_prompt = SystemRolePrompt(system_template)49 50user_prompt_template = """\51Context:52{context}53 54Question:55{question}56"""57user_role_prompt = UserRolePrompt(user_prompt_template)58```59 60> NOTE: You'll notice that these are the exact same prompt templates we used from the Pythonic RAG Notebook in Week 1 Day 2!61 62Following that - we can create the Python Class definition for our RAG pipeline - or *chain*, as we'll refer to it in the rest of this walkthrough. 63 64Let's look at the definition first:65 66```python67class RetrievalAugmentedQAPipeline:68    def __init__(self, llm: ChatOpenAI(), vector_db_retriever: VectorDatabase) -> None:69        self.llm = llm70        self.vector_db_retriever = vector_db_retriever71 72    async def arun_pipeline(self, user_query: str):73        ### RETRIEVAL74        context_list = self.vector_db_retriever.search_by_text(user_query, k=4)75 76        context_prompt = ""77        for context in context_list:78            context_prompt += context[0] + "\n"79 80        ### AUGMENTED81        formatted_system_prompt = system_role_prompt.create_message()82 83        formatted_user_prompt = user_role_prompt.create_message(question=user_query, context=context_prompt)84 85 86        ### GENERATION87        async def generate_response():88            async for chunk in self.llm.astream([formatted_system_prompt, formatted_user_prompt]):89                yield chunk90 91        return {"response": generate_response(), "context": context_list}92```93 94Notice a few things:95 961. We have modified this `RetrievalAugmentedQAPipeline` from the initial notebook to support streaming. 972. In essence, our pipeline is *chaining* a few events together:98    1. We take our user query, and chain it into our Vector Database to collect related chunks99    2. We take those contexts and our user's questions and chain them into the prompt templates100    3. We take that prompt template and chain it into our LLM call101    4. We chain the response of the LLM call to the user1023. We are using a lot of `async` again!103 104Now, we're going to create a helper function for processing uploaded text files.105 106First, we'll instantiate a shared `CharacterTextSplitter`.107 108```python109text_splitter = CharacterTextSplitter()110```111 112Now we can define our helper.113 114```python115def process_text_file(file: AskFileResponse):116    import tempfile117 118    with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as temp_file:119        temp_file_path = temp_file.name120 121    with open(temp_file_path, "wb") as f:122        f.write(file.content)123 124    text_loader = TextFileLoader(temp_file_path)125    documents = text_loader.load_documents()126    texts = text_splitter.split_texts(documents)127    return texts128```129 130Simply put, this downloads the file as a temp file, we load it in with `TextFileLoader` and then split it with our `TextSplitter`, and returns that list of strings!131 132#### QUESTION #1:133 134Why do we want to support streaming? What about streaming is important, or useful?135 136##### ANSWER:137 138Streaming allows users to see the progress in the response, reducing the perceived latency. For larger responses it can allow for interruptions, or for the user to see the reponse being built. Streaming improves user experience by creating interactions that feel more natural and gives users insight into how the response is being created.139 140## On Chat Start:141 142The next scope is where "the magic happens". On Chat Start is when a user begins a chat session. This will happen whenever a user opens a new chat window, or refreshes an existing chat window.143 144You'll see that our code is set-up to immediately show the user a chat box requesting them to upload a file. 145 146```python147while files == None:148        files = await cl.AskFileMessage(149            content="Please upload a Text File file to begin!",150            accept=["text/plain"],151            max_size_mb=2,152            timeout=180,153        ).send()154```155 156Once we've obtained the text file - we'll use our processing helper function to process our text!157 158After we have processed our text file - we'll need to create a `VectorDatabase` and populate it with our processed chunks and their related embeddings!159 160```python161vector_db = VectorDatabase()162vector_db = await vector_db.abuild_from_list(texts)163```164 165Once we have that piece completed - we can create the chain we'll be using to respond to user queries!166 167```python168retrieval_augmented_qa_pipeline = RetrievalAugmentedQAPipeline(169        vector_db_retriever=vector_db,170        llm=chat_openai171    )172```173 174Now, we'll save that into our user session!175 176> NOTE: Chainlit has some great documentation about [User Session](https://docs.chainlit.io/concepts/user-session). 177 178### QUESTION #2: 179 180Why are we using User Session here? What about Python makes us need to use this? Why not just store everything in a global variable?181 182##### ANSWER:183 184Without the User Session, multiple users could access the same session and overwrite each other's data. Any state related to the current conversation would be lost. Global variables affect the whole application and cannot be restricted to a single user.185 186## On Message187 188First, we load our chain from the user session:189 190```python191chain = cl.user_session.get("chain")192```193 194Then, we run the chain on the content of the message - and stream it to the front end - that's it!195 196```python197msg = cl.Message(content="")198result = await chain.arun_pipeline(message.content)199 200async for stream_resp in result["response"]:201    await msg.stream_token(stream_resp)202```203 204## 🎉205 206With that - you've created a Chainlit application that moves our Pythonic RAG notebook to a Chainlit application!207 208## 🚧 CHALLENGE MODE 🚧209 210For an extra challenge - modify the behaviour of your applciation by integrating changes you made to your Pythonic RAG notebook (using new retrieval methods, etc.)211 212If you're still looking for a challenge, or didn't make any modifications to your Pythonic RAG notebook:213 2141) Allow users to upload PDFs (this will require you to build a PDF parser as well)2152) Modify the VectorStore to leverage [Qdrant](https://python-client.qdrant.tech/)216 217> NOTE: The motivation for these challenges is simple - the beginning of the course is extremely information dense, and people come from all kinds of different technical backgrounds. In order to ensure that all learners are able to engage with the content confidently and comfortably, we want to focus on the basic units of technical competency required. This leads to a situation where some learners, who came in with more robust technical skills, find the introductory material to be too simple - and these open-ended challenges help us do this! 218 219 220 221 222 223