RAG with Azure AI Search
In this demo, you’ll build a RAG app with Azure AI Search and Azure OpenAI. You’ll also learn how to tune some parameters to optimize your app.
Setting Up Your RAG App
To get started, you’ll need an LLM for your RAG. Head over to Azure OpenAI Studio, locate the Deployments button on the left menu, and follow the same steps as before to create a gpt-4o model. Choose a descriptive name for it, such as “gpt-4o”. You’ll use this as the value for the AZURE_DEPLOYMENT_MODEL variable soon.
Open the starter project for this lesson. This project uses the same URLs and keys as the previous lesson, and much of the code and structure is the same. You can review the first eight cells to see the differences.
The first cell installs the dependencies, followed by setting up the API keys. Then, you create embeddings for your television data from the television-data.json file.
Creating Embeddings and Search Index
Next, you create a search index client, configure a vector search, create a semantic search configuration, and finally, create an index on Azure AI Search.
This brings you to the final cell, which shows how little you need to build a RAG — with just an index and a prompt, you’re good to go!
Testing the RAG Prompt
Time to step through the final cell and build your RAG.
First, get access to the index. Next, prepare a RAG-style prompt for your LLM — this is crucial, as it conditions the LLM for subsequent queries. Take a good look at this prompt and you’ll see how it’s designed for your specific use case — a friendly assistant that recommends television shows. By telling your LLM “who it is”, specifying a reference, and restricting it from generating responses when it’s unsure, you’ve effectively created a chat companion for your index.
These steps make up a basic RAG; every other step you might choose to include later is an enhancement.
Next is the actual query — an example of what a user of your app might send to your app. Here, it’s asking to recommend a funny show about a group of friends.
Now, perform the search. The search runs on the Azure AI Search service and saves the results in search_results.
Troubleshooting and Optimizing Search Results
With the results, format them into a single string to be passed along with the RAG prompt to your LLM.
That’s it — you’ve built a RAG app! Run the notebook from the beginning with the Run All button at the top of the notebook to see how well your app runs.
Surprise, surprise… Your app says it doesn’t know any such movies. But why? You can confirm from television-shows.json that “Friends” is a good match for your query, so what happened?
If you printed out the content of the search results, you’d realize that it’s empty. So, the issue isn’t with your LLM working with the RAG prompt and the query, but with your search.
To retrieve relevant results on your search index, you need to configure a few things. First is the search_type, which defines the type of search being done (text-based). Next is the use_semantic_reranker flag, which specifies whether to use a semantic reranker to prioritize the most relevant documents. This is what’ll enable semantic search on your index, as discussed in the previous lessons. Finally, the sources_to_include argument limits the number of sources to include. This last option isn’t crucial to retrieving the results, but to optimizing your app.
Go to the cell below “Create an index on Azure AI Search” and uncomment the code below TODO: Enable semantic reranking to configure your search index. Run the cell to recreate your search index and update your index. Now, run the last cell again and observe the output.
Running and Refining Your RAG App
It displays something similar to: “- Friends: This beloved sitcom follows the lives of six close-knit friends—Rachel, Ross, Monica, Chandler, Joey, and Phoebe—as they navigate life in New York City.”
That’s all for this demo. Later on, you can play around with other configurations while monitoring the output. For now, proceed to the next segment to learn more about building an effective RAG app.