Instruction 01
Understanding of Azure OpenAI
Azure OpenAI is one of the many services available on the Azure portal. It offers numerous resources from OpenAI, including GPT-4, GPT-3, DALL-E, Codex, and Whisper. However, they’re not necessarily the same in that Azure OpenAI has customized them to fit Microsoft’s policies.
For example: There’s a considerable amount of content filtering to help make their AI offerings safer to use — this is available in addition to other specific features, such as Abuse monitoring and Default safety policies.
AI — and generative AI in particular — remains a potent tool, capable of achieving so much more than is currently known to the AI community.
Considering its power, AI is still relatively new, and it needs to be used with great caution. Remember that it’s always better to be safe than sorry! For this reason, Microsoft’s Azure OpenAI provides extra security and responsible use measures, to help make its AI offerings safe — yet still potent.
Like all other Microsoft Azure offerings, you’ll benefit from world-class infrastructure, security, delivery, support, and much more when you use Azure OpenAI when compared to using OpenAI directly.
You can manage an Azure OpenAI resource on the Azure portal, Azure AI Studio, through REST APIs, or SDKs.
Exploring Azure OpenAI Features
You’ve already learned about content filtering on top of OpenAI for Azure OpenAI — but that’s not all! Azure OpenAI also offers completions.
The completions feature suggests text to help you complete your prompts more quickly (you might have seen it already, when using Bing Chat). Not only can this help you type faster, but it can also help you type correctly. The suggestions create excellent prompts, which are likelier to yield more relevant results than without using them.
Keep in mind that the models you interface with have been pre-trained on data based on specifically chosen algorithms. Some techniques help make conversational AI chats possible by regenerating user prompts into refined prompts, before feeding them to the LLMs. For this reason, the completions feature alone gives a major boost to Azure OpenAI: It enhances prompts and makes it quicker to send them, too!
Azure OpenAI also includes the ability to customize LLMs. You’re not restricted to only using the default models provided, or bringing them in from other providers — with Azure OpenAI, you can use prompt engineering, RAGs, and fine-tuning techniques to customize your chosen LLM. This makes it possible to build highly specialized apps for your needs.
For all the different kinds of data and their respective models available, Azure OpenAI also provides embedding capabilities. It handles embeddings for prompts and all other phases in between the different steps to generating responses.
With Azure OpenAI, you have access to all supported OpenAI models in one place, with each benefiting from the general features available on Microsoft Azure. Just like you saw with Azure AI Search earlier, there are private endpoints together with many customizations you could use to configure your resource instance.
Microsoft is constantly updating their offerings, too. As of this writing, there are various features in the preview phase — including Assistants (for creating AI assistants), Text-to-speech (for translating text to speech), Realtime API (for speech-to-audio for conversations), and more.
Limitations of Azure OpenAI
OpenAI remains one of the major AI service providers (although others are available, of course!), so it isn’t surprising that Microsoft teamed up to provide Azure OpenAI. They have established, advanced, and highly capable models for text, audio, video, images, and many more. When you create an Azure OpenAI resource, you can easily deploy models based on any of OpenAI’s popular supported models, or even bring in your own, non-OpenAI-based model.
Like the other features and resources, it’s necessary to know your quota for your Azure OpenAI resource and how it’s measured. Prompts sent to these models are read in tokens — where a token could be a single word, or chunks of characters. For images and other data formats, their size or depth of detail determines how many tokens they take up.
Note: As a general guide, keep in mind that the more data you send to your prompts, and the more you get from it… and the more your quota runs out!
There are other limitations to know, such as the number of concurrent requests, requests per minute, deployments per resource, and others. Visit the Azure OpenAI resource usage or quota page to understand how tokens are read, and how they’ll affect your specific use case.
Bear in mind, too, that whenever you create an Azure OpenAI resource, you need to create the models you’ll use, giving them descriptive names so that you can easily reference them through any of the available access points.
Also, note that not every model may support all the features you want to use. You can learn more about which models can be used with each operation at https://go.microsoft.com/fwlink/?linkid=2197993 before choosing a model for your RAG, for instance.
Understanding RAGs with Azure OpenAI
As you’ll know by now, Retrieval-Augmented Generation (RAG) is a technique that combines data retrieval from AI models with information from custom data to enhance responses with highly relevant information.
Models are pre-trained, and they’re able to generate responses based on given prompts. By default, these models can only generate responses accurately based on trained data — this introduces a limitation where you’re unable to use these models on new data, or any data outside of that training data.
RAGs solve this problem by combining new data with prompts to generate relevant responses from trained models.
The main components of a RAG are indexing, querying, and response generation:
- The new data is first indexed, using preferably the same embedding model and parameters for the LLM that will generate the response.
- The prompt is carefully designed, embedded, and combined with the indexed data to query the LLM, or whichever model is chosen for the RAG app.
To build a RAG app, Azure AI Search will allow you to index your data and search through large sets of documents. It supports complex queries and can handle a large variety of data. To generate natural responses, you’ll need Azure OpenAI — it’ll use the indexed data and query from Azure AI Search to generate relevant results.
Ready to get into a RAG-related demo? Continue to the next segment.