Instruction 02

Building Effective RAGs

Keep in mind that Azure AI Search employs several techniques to refine its output. For every stage of the RAG process, you could further refine your inputs to Azure resources to generate relevant results.

Results have a score that indicates how relevant they are, given the prompt and the model used.

RAGs can be used for simple searches, but they’re usually used conversationally — due to their ability to handle human language prompts and large amounts of data, in addition to pre-trained models with billions of parameters, people rely on them to generate relevant, accurate responses.

However, LLMs will always come up short where specializations in certain fields are required — for example, in some fields like medicine or travel, where responses must be accurate and timely. For this reason, it’s best to tune your entire RAG process to ensure you get the best results possible. Below are a few parameters you can configure to help you build effective RAGs:

• Chunking: Azure AI Search performs chunking automatically when indexing data. Chunking refers to the process of breaking down larger documents into smaller, manageable pieces. Depending on the size of each chunk, it could be harder to retrieve relevant results. It’s possible to leave out relevant results depending on the chunk size. Sometimes, results in the middle of indexed documents are skipped when searching for results. With a proper chunk size, searches can be quick and accurate.

• Search: Azure AI Search combines many useful techniques to produce highly relevant results in a short amount of time. Should the search suffer, the final generated response is ultimately going to be found lacking. The speed at which the search is conducted also affects the overall performance of the RAG.

• Context windows: Context windows refer to the number of tokens a model can handle at a go. With smaller context windows, you can only send small amounts of prompts to the model — and you’ll have to monitor its responses too, since they’re equally limited by the same — the tokens. For instance, if you exhaust your tokens for ChatGPT in a day, you may see a message asking you to upgrade to a paid subscription to keep using the service.

• Query parameters: Query parameters help refine searches. These may include filters, scoring profiles, k-nearest neighbor parameters, or others. Through testing, you can tweak these parameters until you settle on optimal options for your RAG.

• Prompt engineering: By carefully creating prompts, you’re setting your RAG app up for success — keep tweaking and testing until you get what you’re looking for! Good prompts will eliminate noise and irrelevant documents from responses, and there are also many community-inspired prompts for various scenarios you can take advantage of.

Troubleshooting Azure AI Search

There are a few things to look out for while you work with Azure AI Search.

First, remember to enable all required permissions. Permissions are one way Azure protects your resources from unauthorized access and misuse. When you create a resource, it might take anywhere from a few minutes to an hour or more before it becomes available.

You may not be able to tell from the Azure portal or the Azure OpenAI Studio whether a resource is fully ready. For instance, if you encounter an error like this:

AuthenticationError: Error code: 401 - {'statusCode': 401, 'message':
  'Unauthorized. Access token is missing, invalid, audience is
  incorrect (https://cognitiveservices.azure.com), or have expired.'}

it’s likely the resource isn’t fully ready, and you may need to wait. Permissions and resource availability delays are the two most common reasons for this error.

Another common pitfall is creating Azure AI Search and Azure OpenAI resources in different regions. In many instances, when you realize you’ll need both, create the Azure OpenAI resource first, as it’s usually less available. Then, create your Azure AI Search resource in the same region. Check the availability regions to see which regions support which resources at https://learn.microsoft.com/en-us/azure/search/search-region-support.

Error messages can be misleading at times. They may come with suggestions to help you resolve them, like the one above. The Azure OpenAI FAQ at https://learn.microsoft.com/en-us/azure/ai-services/openai/faq is another useful troubleshooting guide. Do your best to follow all instructions carefully. Look out for updates to this module, the forums, and the official documentation to stay current, as Azure AI services are constantly evolving.

Finally, feel free to contact the Azure support team. They are always ready to help, and if necessary, they will go through your code with you until your issue is resolved. They take their customers very seriously.

Considering Ethical Usage for AI

The most common use of AI is with RAGs. ChatGPT is a very popular app, and it’s mostly just a conversational AI chat app. Since ChatGPT became accessible globally, many other AI businesses have sprung up, targeting different use cases. There are AI apps for basically everything now — from image generation, speech-to-text and text-to-speech, video generation, lyric generation, software programming, customer support, product recommendations for e-commerce apps, assistants and agents that are capable of completing tasks, and a whole lot more.

However, there’s a major issue with LLMs: hallucinations. Hallucinations are when an AI app or piece of software generates incorrect or irrelevant information based on the prompt. Considering that generative AI has become a major part of many processes all over the world now, it is crucial that it generates the most accurate results possible for the millions of people using it.

Again, if you already understand that models generate responses based on their training data, then you’ll understand that training a model with false, unethical, incomplete or inaccurate data will yield incorrect or irrelevant results. Not everyone understands how these models work, so some tend to take these AI responses with certainty, no matter what they generate — there are reported incidents so damaging that they make others not want to have anything to do with AI at all.

Never forget that with great power comes great responsibility! As a developer, having learned about AI and RAGs with Azure AI Search and Azure OpenAI, you also need to learn how to use it responsibly. Otherwise, you may even find yourself violating some national and geographic laws, such as those of the EU or US — such offenses could mean you put the lives of others, your app, or yourself at risk.

You’ve just learned about one of the most useful tools that could set you and your business apart, making you a household name and bringing in billions of dollars in revenue, and you surely don’t want to lose it all. So, here are a few ethical considerations to keep in mind as you build AI apps with Azure AI Services:

• Read and follow Microsoft’s Responsible AI Standard. It spells out its policies governing the use of its AI services. It proposes four main steps for achieving responsible AI use: identify and prioritize potential threats your app could pose, measure the frequency and severity of these threats, mitigate them, and execute the operational readiness plan from your findings. Create and follow a responsible AI usage policy for your app and business.

• Cite sources to help give more credibility to your app. Because of the generative abilities of these AI tools — and their ability to understand and communicate with natural language — it’s easy for users to perceive them as a credible, authoritative source of information. But perhaps it’s better to make it clear to your users how your AI works. This way, you’ll allow users to be the final judge.

• Depending on your use case, ensure age limits or parental supervision is in place for using your API apps. Consider ways of enforcing this in a way that fits your specific app.

• When in doubt, always choose safety. If there’s damage at any point in time, it’ll be more damning and probably overshadow every win you’d have gained with your app. Iteratively test different prompts, include many carefully designed content filters, set boundaries, and do whatever you can to prevent your app from generating incorrect or inaccurate responses.

• Generate fewer responses unless the user specifies it. The lengthier your response, the more likely your app is to include irrelevant and inaccurate responses. Sometimes, for certain scenarios, it’s best to respond with predetermined responses, especially in instances where you recognize known patterns in the query.

• Choose good base models for your app. This way, you’ll have less to do by way of moderation than if you chose any model at all.

• Finally, put in measures to help detect and prevent bots from using your AI APIs. Also, be extra cautious when using your AI apps as agents — whether it’s to complete a task or post to a public forum. Be sure to scrutinize and monitor your app’s rights and activities.

Read widely when it comes to this topic — and don’t just rely on AI itself for it! New regulations are popping up in different places now and then. Stay on top of the latest on all things AI, and you’ll be good to go.