Retrieval-Augmented Generation with LangChain

Transcript

Demo

SportsBuddy is already in good shape. However, one immediate concern is its limited knowledge of current sports events. Try this out by asking SportsBuddy what it knows about Jamaica’s participation in the 2024 Summer Olympics.

rag_chain.invoke("What does the retrieved context say about Jamaica in 
  the 2024 Olympics?")

You’ll get something along the lines of:

“The retrieved context does not provide any specific information about Jamaica’s participation or status in the 2024 Olympics. It primarily discusses the bidding process and controversies surrounding the Games. Therefore, I don’t know the answer regarding Jamaica’s involvement in

  the 2024 Olympics."

First, you’re going to load data from multiple sources. It might interest you to know that there are tools created for retrieving data from Wikipedia. You’ll use that instead of the generic web loader.

Start by installing the Wikipedia dependency in your terminal. Locate the terminal tab or open a new one from the Launcher or the Menu dropdown:

pip install wikipedia

Back to your notebook, identify the cell where the WebBaseLoader is imported. Add the code below to import the WikipediaRetriever:

from langchain_community.document_loaders import WikipediaLoader

Now, update your documents to include data from the WikipediaRetriever. This helps gather more specific information about countries participating in the Summer Olympics. Replace docs = loader.load() with the following:

wiki_loader = WikipediaLoader('Jamaica_at_the_2024_Summer_Olympics')

docs = []
web_loader = WebBaseLoader(
    web_paths=("https://en.wikipedia.org/wiki/2024_Summer_Olympics",)
)
wiki_loader = WikipediaLoader('Jamaica_at_the_2024_Summer_Olympics')

loaders = [web_loader, wiki_loader]

for loader in loaders:
    docs.extend(loader.load())

This fetches content from https://en.wikipedia.org/wiki/Jamaica_at_the_2024_Summer_Olympics and integrates it into the LangChain documents.

Re-execute your initial query and observe the improved response:

’Jamaica is participating in the 2024 Summer Olympics in Paris, marking its eighteenth appearance as an independent nation. The country has qualified athletes in athletics, diving, judo, and swimming for the Games. Notably, Yona Knight-Wisdom will compete in diving, and there

  are entries in track and field events as well as swimming events.'

Brilliant. There are many more document loaders to allow you to build a more fine-tuned and robust RAG app. Visit https://python.langchain.com/v0.2/docs/integrations/document_loaders/ for a comprehensive list of available loaders.

Remembering Previous Chats

Currently, SportsBuddy lacks memory of past conversations. When asked a follow-up question, it simply indicates that it doesn’t know. To address this, introduce a memory store and enhance the prompt to incorporate previous messages.

Start by updating your prompt. You need to write a new prompt that uses the LLM and your previous conversation:

from langchain.chains import create_history_aware_retriever
from langchain_core.prompts import MessagesPlaceholder
from langchain_core.prompts import ChatPromptTemplate

contextualize_q_system_prompt = (
  "Given a chat history and the latest user question "
  "which might reference context in the chat history, "
  "formulate a standalone question which can be understood "
  "without the chat history. Do NOT answer the question, "
  "just reformulate it if needed and otherwise return it as is."
)

contextualize_q_prompt = ChatPromptTemplate.from_messages(
  [
    ("system", contextualize_q_system_prompt),
    MessagesPlaceholder("chat_history"),
    ("human", "{input}"),
  ]
)
history_aware_retriever = create_history_aware_retriever(
  llm, retriever, contextualize_q_prompt
)

This code dynamically constructs a new prompt that factors in the current user query, past conversation history, and the LLM’s capabilities. While this general-purpose prompt has demonstrated effectiveness in enabling conversations, it can be tailored to better align with your specific use case. The create_history_aware_retriever function generates a retriever that can fetch responses enriched with historical context.

The core structure of your existing code remains intact, with the key distinction being the incorporation of classes and functions that enable conversations with a historical perspective. You can now modify your initial prompt, excluding the {question} placeholder, as the newly formulated prompt will be utilized to ask the question. Add the following code:

from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain

system_prompt = (
  "You are an assistant for question-answering tasks. "
  "Use the following pieces of retrieved context to answer "
  "the question. If you don't know the answer, say that you "
  "don't know. Use three sentences maximum and keep the "
  "answer concise."
  "\n\n"
  "{context}"
)

qa_prompt = ChatPromptTemplate.from_messages(
  [
    ("system", system_prompt),
    MessagesPlaceholder("chat_history"),
    ("human", "{input}"),
  ]
)

question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)

rag_chain = create_retrieval_chain(history_aware_retriever, 
  question_answer_chain)

You now reintroduce your initial prompt with a slight modification and couple it with your generated historical prompt to query your application.

Next, you need a place to keep your chat sessions. In a fully-fledged app, you’re free to use a database or an in-memory storage tool, but for now you can get by with a simple key/value pair using Python’s dictionary type. The key can be a session key that uniquely identifies each conversation. In a production app, you could have different session keys for conversations between different parties.

To create the chain like before, use RunnableWithMessageHistory because your app now keeps historical contexts. This replaces the simple RunnablePassthrough class. Add the following to save the sessions and create the chain:

from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_community.chat_message_histories import ChatMessageHistory

store = {}

def get_session_history(session_id: str) -> BaseChatMessageHistory:
  if session_id not in store:
    store[session_id] = ChatMessageHistory()
  return store[session_id]

conversational_rag_chain = RunnableWithMessageHistory(
  rag_chain,
  get_session_history,
  input_messages_key="input",
  history_messages_key="chat_history",
  output_messages_key="answer",
)

All set. Ask your question:

conversational_rag_chain.invoke(
  {"input": "What does the retrieved context say about Jamaica
     in the 2024 Olympics?"},
  config={
    "configurable": {"session_id": "sports-buddy-session"}
  },
)["answer"]

The major difference here is that you specify a session ID to identify a specific historical context:

’Jamaica competed at the 2024 Summer Olympics in Paris from July 26 to August 11, marking its eighteenth appearance as an independent state. The country entered athletes in various events, including track and field, diving, judo, and swimming. Notably, Jamaica sent its largest

  delegation to the previous Olympics in 2016, with 56 athletes.'

For the first question posed to the system, the response might mirror the one received previously. However, follow it up with another question, and see how SportsBuddy performs:

conversational_rag_chain.invoke(
  {"input": "Is it their eighteenth appearance as an independent state?"},
  config={"configurable": {"session_id": "sports-buddy-session"}},
)["answer"]

A possible response reads as follows:

“Yes, it is Jamaica’s eighteenth Summer Olympic appearance

  as an independent state."

There you have it. A conversational sports AI expert — SportsBuddy. Continue to this lesson’s concluding segment.