Introducing SportsBuddy

Introducing SportsBuddy

In this lesson, you’ll build a RAG app called SportsBuddy. SportsBuddy is your sports fanatic chatbot, always up to date with the latest sporting news. Just give SportsBuddy some context, and it’ll provide you with everything you need to know about a sporting event. Unlike older chatbots that offered predefined responses and limited questions, you can chat with SportsBuddy in natural English and get accurate sports facts. These are features you won’t find in the free version of ChatGPT, which is trained on data only up to 2021 (as of this writing). So why pay for the pro version when you have SportsBuddy? Time to get started.

Setting up an OpenAI Developer Account

To begin, ensure that you have a valid OpenAI API key. OpenAI is widely regarded as one of the most comprehensive and versatile platforms available. Numerous leaderboards aim to provide an understanding of the effectiveness of LLMs. Each leaderboard considers a variety of parameters. Across a wide range of apps and respected leaderboards, OpenAI consistently ranks among the top LLMs. Some of these leaderboards can be found at https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard, https://www.trustbit.tech/en/llm-benchmarks, and https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard. One thing to note is that there’s a lot of healthy competition. Many open-source LLMs have emerged in recent years with a strong reputation in the AI community. Be sure to explore them later.

Visit https://platform.openai.com/signup to sign up for an API key. You’ll have to pay a small fee to enable the API key. Go ahead and choose the cheapest option available; it’s enough for SportsBuddy. Soon, you can replace OpenAI with other versions that you might find equally good or even better and cheaper. Because you’ll be using LangChain, making such a change incurs little to no productivity cost. When you receive the key, store it securely on your computer. You’ll use it soon.

Retrieving Data for SportsBuddy

There are many ways to feed SportsBuddy with information. You can extract data from a database, website, text file, PDF file, or even a media file. You’ll use Wikipedia for now. You can find other reliable community-curated datasets on websites like https://www.kaggle.com/datasets and https://data.world. Open Jupyter Lab with:

jupyter lab

In the Launcher tab, open a terminal to install LangChain, LangChain for Chroma, and OpenAI if you haven’t already:

pip install langchain langchain_community langchain_chroma
pip install -qU langchain-openai

Store your OpenAI API key in an environment variable:

export OPENAI_API_KEY="<insert-your-api-key-here>"

Open the notebook for Lesson 3 from the Launcher tab or the File menu. The first cell contains the setup for your USER_AGENT environment variable. This is to help identify your OpenAI session. Keeping your usage of OpenAI API organized is a best practice. In the next cell, you set up OpenAI for chat:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini")

You’ve specified the gpt-4o-mini model of OpenAI. You can leave it out or specify another model depending on the type of subscription package you have. As of this writing, the earliest training data for this model is from 2021. Add the following to the bottom of this cell to verify:

response_message = llm.invoke(
    "What is the cutoff date for your training data?"
)

print(response_message.content)

You get something like:

My training data goes up until October 2021. If you have any questions or
need information based on that timeframe, feel free to ask!

That’s quite a long time ago! In sports, there are many events year-round, every year. But here you are with an LLM that doesn’t know about the 2024 Olympics. Well, you’re about to equip your RAG with credible information from Wikipedia about the most recent summer Olympics.

Remove the code you just added. In the next cell, the imports include a WebBaseLoader to retrieve data from a web URL. Uncomment the code below #TODO: Load documents to retrieve the data:

# TODO: Load documents
docs = loader.load()

The loader reads the textual data from the given the Wikipedia page on the 2024 Summer Olympics. loader.load() converts the data into LangChain documents.

You’ve got the data now. But your LLM has not yet received this data. Also, you already learned about databases, so you’ll store your data first before using it. In the next cell, you’ll prepare the data by splitting them into small, manageable chunks and save them in a Chroma database.

Storing Retrieved Data in the Database

In the next cell, uncomment the code below #TODO: Split documents to break your data into smaller, manageable chunks:

# TODO: Split documents
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)

The chunk_size specifies the maximum size of a chunk. Depending on the amount of text you’re analyzing, you might need to use a higher or lower value. The chunk_overlap determines how many characters are allowed to flow into other chunks. This prevents the risk of losing some of the text. It helps to preserve the context, too. A value between 100 and 300 is usually recommended.

Right below that section, uncomment the code below # TODO: Store documents in Chroma vector database to store the data in your Chroma database:

# TODO: Store documents in the Chroma vector database
database = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())

Below the code above, you get a reference of the database in retrieval mode. A retriever lets you search the vector store for documents based on their vector representations. It uses the available search methods in the database, such as similarity search, to perform queries. The retriever interface also provides additional features, such as search parameters like threshold scores and the ability to specify the number of documents to return.

The following code gives you the retriever for the Chroma database:

retriever = database.as_retriever()

Building the Prompt

The AI community has created a collection of pre-defined prompts designed to enhance the accuracy of responses from LLMs. Explore these prompts at https://smith.langchain.com/hub/rlm.

In this scenario, you’re using “rlm/rag-prompt”, which instructs the LLM as follows:

You are an assistant for question-answering tasks. Use the following pieces of
  retrieved context to answer the question. If you don't know the answer, 
  just say that you don't know. Use three sentences maximum and keep the 
  answer concise.
Question: {question} 
Context: {context} 
Answer:

As you can see, this prompt guides the LLM to provide suitable answers to chat-style questions. The placeholders {question} and {context} will be populated with the user’s query and relevant information when the prompt is executed.

The format_docs(docs) function transforms your source data into a single string, with each document separated by two new lines (\n\n). This formatted text is then incorporated into the prompt for the LLM.

The RunnablePassthrough class ensures your question is passed directly to the prompt without alterations. You also can use it to add data to the output if needed.

The StrOutputParser is responsible for converting the LLM’s response into a readable string format.

A key element of this prompt is the use of pipes (|). This powerful LangChain feature allows you to chain operations together. The | operator visually represents the flow of data, with each operation’s output feeding into the next. This flexible system lets you create complex LLM workflows tailored to your needs.

In this specific prompt, the dictionary containing the question and context is passed to the “rlm/rag-prompt” template. The resulting formatted prompt is then sent to the LLM, and its response is finally converted to a string using the StrOutputParser.

It’s crucial to remember that the quality of your prompts plays a significant role in the success of your LLM interactions, alongside the LLM’s training data.

Now, put this into action. Execute the chain by calling rag_chain.invoke() and providing your question. Because SportsBuddy has access to the 2024 Olympics data, feel free to query it based on the information retrieved from the Wikipedia page.

rag_chain.invoke("Which programmes were dropped from the 2024 Olympics?")

You get a response along the lines of:

'Four events were dropped from weightlifting for the 2024 Olympics.
  Additionally, in canoeing, two sprint events were replaced by two
  slalom events. The overall event total for canoeing remained at 16.'

You might encounter some warnings in the output. This is normal because libraries are continually updated, which can lead to certain features becoming deprecated.

And there you have it! You’ve created a basic RAG AI chat app. You’ve harnessed the power of an existing LLM to generate relevant and precise responses based on the latest information. The potential applications are vast. For instance, this could be a powerful tool for academic research: Simply provide your RAG with reliable data and get accurate and insightful answers, almost like conversing with your professor.

Next Steps

To further explore its capabilities, try another question. Create a new cell and ask:

rag_chain.invoke("Was there a podium sweep in the 2024 Olympics?")

Expect an answer like this:

"Yes, there was one podium sweep during the 2024 Olympics. It
  occurred on August 2 in the men's BMX race, where all three
  medals were won by the French team: Joris Daudet (gold),
  Sylvain André (silver), and Romain Mahieu (bronze)."

In the next section, you’ll delve into a complete demonstration of building a basic RAG app from start to finish.