Enhancing a RAG App

Enhancing a RAG App

SportsBuddy is already in good shape. However, one immediate concern is its limited knowledge of current sports events. When you ask SportsBuddy what it knows about Jamaica’s participation in the 2024 Summer Olympics for instance, you’ll get something along the lines of:

“The retrieved context does not provide specific information about Jamaica’s participation or status in the 2024 Olympics. It primarily discusses the bidding process and controversies surrounding the Games. Therefore,

  I don't know the answer regarding Jamaica's involvement in the 2024 Olympics."

You’ll need to increase the scope of your source data to give SportsBuddy more context for such questions. You’re not limited to a single source of data for your RAG, you can provide multiple data sources. Tons of information exists on the internet in various formats and for each one of them, you’re sure to find a suitable retriever. In the previous segment, you retrieved data from a Wikipedia page - you didn’t need to use a WebBaseLoader at all because there’s a WikipediaLoader designed to retrieve data from Wikipedia.

LangChain offers a wide array of data loaders for various formats, including CSV, file directory, HTML, JSON, markdown, Microsoft Office, PDF, Cassandra, PostgreSQL, and more. You can find a comprehensive list of document loaders for LangChain at https://python.langchain.com/v0.1/docs/integrations/document_loaders/. The AI ecosystem thrives on integrations, and most frameworks and components provide wrappers that simplify working with other components. If you’re using Llama LLM, for example, you’ll find an extensive list of supported readers at https://llamahub.ai/?tab=readers.

In a production app, you’ll likely utilize multiple data loaders. Many APIs support batch loading, which significantly speeds up the loading process. Most vector stores also offer this feature, allowing you to load data asynchronously and in parallel.

Remembering Previous Chats

Currently, SportsBuddy lacks memory of past conversations. When asked a follow-up question, it simply indicates that it doesn’t know. You’d need a memory store to provide a means to save chats. Ideally, this implementation should make use of the system’s RAM for quick access and retrieval.

RAG systems typically store either full or partial conversation histories to ensure they don’t miss relevant context. Storing the full history can lead to rapid storage growth, while storing only parts of it risks losing some contextual information.

Usually, key details like user preferences, and previous queries and responses are retained and used in subsequent operations. Conversation history can also be implemented with metadata annotation. With this, small pieces of relevant contextual information are attached to messages. Other techniques are summarization and truncation. By summarizing or removing parts considered less relevant, the context in each chat session can be stored for future conversations.

A common technique employed in remembering previous chats is session-based retrieval. Each chat session is uniquely identified tagged and used in subsequent chats. OpenAI has several useful functions that let you implement historical chats in your RAG. In the next segment, you’ll build on your basic RAG to retrieve data from multiple sources and add the ability for SportsBuddy to remember previous chats.