Retrieval-Augmented Generation with LangChain

Transcript

Demo

In this demo, you’ll build a basic RAG app harnessing the power of OpenAI and LangChain.

Kick things off by opening a Jupyter Lab session. From your terminal, execute jupyter lab. This opens a fresh Launcher tab in your browser.

Next, select ‘Terminal’ from the ‘Other’ category to get a terminal up and running. Then, proceed to install the necessary libraries with the following command:

pip install langchain langchain_community langchain_chroma
pip install -qU langchain-openai

This installs the core LangChain library, its community extensions, and its specific components for Chroma and OpenAI. LangChain’s straightforward interface lets you easily integrate and work with various other providers down the line.

Before you write some code, set the OpenAI API key in your environment:

export OPENAI_API_KEY="<insert-your-api-key-here>"

The OpenAI API key is a sensitive piece of data; you don’t want to expose it in your code. Also, setting the USER_AGENT variable is useful for telling your OpenAI API sessions apart. Set it up now:

import os

os.environ['USER_AGENT'] = 'sports-buddy-demo'

Now, open a new Notebook from the Launcher tab or the File menu. Start by importing LangChain’s OpenAI component:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")

It’s as simple as calling the ChatOpenAI constructor. No arguments are required.

In a new cell, import the necessary classes for retrieving data, storing it, and creating a prompt.

from langchain import hub
from langchain_chroma import Chroma
from langchain_community.document_loaders import WebBaseLoader
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter

The WebBaseLoader() lets you load textual data from URLs. Now, use it to fetch data from the 2024 Summer Olympics Wikipedia page:

loader = WebBaseLoader(
  web_paths=("https://en.wikipedia.org/wiki/2024_Summer_Olympics",),
)
docs = loader.load()

The WebBaseLoader’s load() function converts the retrieved text into LangChain documents, a format ideal for embedding in vector stores and other LangChain components, including those designed for OpenAI.

With the data loaded, you’ll create a prompt and send it to your OpenAI LLM to generate answers. However, given what you’ve learned about the importance of databases in RAG apps, you’ll store the data in a database first.

Begin by splitting the text into smaller chunks and save them in a Chroma database:

text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, 
  chunk_overlap=200)
splits = text_splitter.split_documents(docs)
database = Chroma.from_documents(documents=splits, 
  collection_name="sports_collection", embedding=OpenAIEmbeddings())

A database retriever provides an interface to query the database. Set up a retriever for our prompt:

retriever = database.as_retriever()

Now, prepare the prompt:

prompt = hub.pull("rlm/rag-prompt")

This pulls specific text from the hub. You can visit https://smith.langchain.com/hub/rlm to see the details. Essentially, it instructs the LLM to act as a question-answering assistant, using provided context and keeping answers concise:

You are an assistant for question-answering tasks. Use the following
  pieces of retrieved context to answer the question. If you don't
  know the answer, just say that you don't know. Use three sentences
  maximum and keep the answer concise.
Question: {question} 
Context: {context} 
Answer:

A well-crafted prompt is key to effective communication with an LLM. This prompt sets clear boundaries and context, enabling the LLM to generate accurate and helpful responses. It’s adaptable: You can modify it for specific use cases, but it works well for general chat apps.

You’ll use the format_docs function to convert the source data into a long, paragraph-separated text format. This formatting enhances the prompt’s effectiveness. Here’s the function:

def format_docs(docs):
  return "\n\n".join(doc.page_content for doc in docs)

Next, assemble the chain. You’ll provide the database retriever as the context and the question directly using RunnablePassthrough. Pipe this information to the prompt, then to the LLM, and finally to the string parser, which outputs the chain’s result as a string:

rag_chain = (
  {"context": retriever | format_docs, "question": RunnablePassthrough()}
  | prompt
  | llm
  | StrOutputParser()
)

With that, you’re ready to go! Execute the chain and ask your question:

rag_chain.invoke("Which programmes were dropped from the 2024 Olympics?")

Congratulations! You’ve built SportsBuddy, a functional RAG app specializing in modern sports events. Its current knowledge is limited to information from the 2024 Summer Olympics Wikipedia page, but you’ll explore ways to expand the app’s capabilities in the upcoming section.