Retrieval-Augmented Generation with LangChain

Open the starter notebook file for Lesson 4 to try this out. The starter notebook has been modified to reflect the basic RAG implementation. It currently doesn’t use the default RAG prompt. This specific prompt conditions the response, which doesn’t help showcase what you’re trying to prove here. Because the response will be long, the response’s length is displayed instead of the response itself. Finally, the chunk_overlap has been reduced since this is a relatively small dataset. Run the code as is to see how many documents are returned based on the given query:

4

It displayed 4 documents. It might be different for you. So how were you getting the kind of responses you were getting earlier? It’s mainly due to the prompt. In some cases, this could be undesirable because it might leave out a lot of relevant information.

In cell 4, underneath # TODO: Increase the value of 'k' to retrieve more documents, specify a k argument of 1 to reduce the returned documents to 1. Then, show the full response instead of the length if you didn’t make that change earlier:

retriever = database.as_retriever(search_kwargs={"k": 1})

At the bottom of the same cell, display the full response by printing the length and response:

print(len(response))
print(response)

Execute the cell. You’ll see that the length is 1, just as you specified earlier. The returned document is a section of the raw data you fed into the data store. Although the document is the most relevant to the given query, it contains some irrelevant information. Therefore, apart from some of the retrieved documents being irrelevant to the query, the relevant documents could also contain irrelevant text.

To tackle this issue, you’ll use Contextual Compression. It’s a technique used to compress responses based on the query to filter out irrelevant responses. This process is more like fine-tuning your RAG’s output. Although the retrieved documents are the best matches for the query, contextual compression introduces a post-processing phase to remove noise, resulting in a better response.

LangChain offers various types of compressors and filters. To try out the LLMChainFilter, run the following code in a new cell:

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor

compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(
  base_compressor=compressor, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
  "How was security during the 2024 Olympics"
)

compressed_docs

[Document(metadata={'source': 'https://en.wikipedia.org/wiki/Concerns_
  and_controversies_at_the_2024_Summer_Olympics', 'summary': 
  'Various concerns and controversies arose in relation to the 
  2024 Summer Olympics, including security concerns, human rights 
  issues, and controversy over allowing Israel to participate 
  amidst the Israel–Hamas war, and allowing Russian and Belarusian 
  athletes to compete as neutrals amidst the Russian invasion of 
  Ukraine. Despite the nominal Olympic Truce in place, the wars in 
  Ukraine and Palestine already set a more conflicted political 
  background to the 2024 Summer Olympics, before considering domestic
  and sporting issues.\n\n', 'title': 'Concerns and controversies 
  at the 2024 Summer Olympics'}, page_content='Various concerns and 
  controversies arose in relation to the 2024 Summer Olympics, 
  including security concerns, human rights issues, and controversy 
  over allowing Israel to participate amidst the Israel–Hamas war, 
  and allowing Russian and Belarusian athletes to compete as neutrals 
  amidst the Russian invasion of Ukraine.'),
Document(metadata={'source': 'https://en.wikipedia.org/wiki/Concerns_
  and_controversies_at_the_2024_Summer_Olympics', 'summary': 'Various 
  concerns and controversies arose in relation to the 2024 Summer 
  Olympics, including security concerns, human rights issues, and 
  controversy over allowing Israel to participate amidst the Israel–Hamas 
  war, and allowing Russian and Belarusian athletes to compete as 
  neutrals amidst the Russian invasion of Ukraine. Despite the nominal 
  Olympic Truce in place, the wars in Ukraine and Palestine already set 
  a more conflicted political background to the 2024 Summer Olympics, 
  before considering domestic and sporting issues.\n\n', 'title': 
  'Concerns and controversies at the 2024 Summer Olympics'}, 
  page_content='In February 2024, the French government announced that, 
  as a security precaution, the number of spectators for the opening 
  ceremony along the Seine would be reduced from 600,000 to 300,000. 
  This plan was proposed by Minister of the Interior Gérald Darmanin 
  in 2022. A security perimeter around the area designated for spectator 
  access was planned to be erected in the days leading up to the games, 
  limiting access for the public. In July 2024, it was reported that 
  there would be an expected 220,000 spectators and 45,000 police and 
  security officers present.')]

You can see right away that this is a much cleaner result set. First, only two documents are returned, compared to the four from earlier. The content of these responses isn’t just a dump of the documents, but refined responses based on the given query.

Introducing Re-ranking

A popular re-ranking tool is the Cohere API. Visit the developer page at https://dashboard.cohere.com/welcome/register to obtain a free API key. Visit the API Keys page from left menu items on the dashboard, and copy the default API key under the Trials keys section. In a new cell, install Cohere or, better still, langchain-cohere:

pip install cohere
pip install langchain-cohere

In the next cell, store your Cohere API key. Because you’re already in an active session, you might not be able to access the API key if you set it in your terminal. So go ahead and install it in a new cell:

import os
import getpass

os.environ["COHERE_API_KEY"] = getpass.getpass("Cohere API Key:")

When you run this cell, it prompts you to enter your API key in an input box located just below the cell. Paste the API key in and then create a cell. You’ll then use Cohere to get your response. Add the following code:

from langchain.retrievers.contextual_compression import 
  ContextualCompressionRetriever
from langchain_cohere import CohereRerank
from langchain_community.llms import Cohere

llm = Cohere(temperature=0)
compressor = CohereRerank(model="rerank-english-v3.0")
compression_retriever = ContextualCompressionRetriever(
  base_compressor=compressor, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
  "How was security during the 2024 Olympics"
)

compressed_docs

Review the response. It’s similar to the one obtained using the contextual compression technique you previously employed. Be aware that you haven’t yet refined your prompt to achieve this level of accuracy. You used Cohere’s Reranking API to generate improved results. The optimal outcome is the first one. However, even the last result among these documents is significantly better than what you obtain directly from the database.

The next part of the lesson will feature a demonstration of the re-ranking strategy in action.