Azure Content Safety

Transcript

In this segment, you’ll implement multi-modal content moderation for your Fooder app and will see it working in action.

Start by opening the VScode for this demo. Open the m3-azs-materials/04-advanced-content-moderation-strategies/Starter directory.

Explore the Starter Project

The starter project contains two files:

• starter/app.py: This file contains the UI code for the web app created using the Streamlit framework. The UI generated by the code allows you to select images from your system and write text content. Once the required info intended to be published is added, the user can click the Submit button to publish the content in the app.
• starter/business_logic.py: This is the file you’ll work on throughout this demo. It’ll contain the multi-modal moderation logic. Currently, it includes a check_content_safety function that’s referenced in the UI code from the earlier file.

Create Azure AI Content Safety Client

It’s time to build the app. Above check_content_safety function, write the following code:

# 1
import os
from dotenv import load_dotenv
from azure.core.credentials import AzureKeyCredential
from azure.ai.contentsafety import ContentSafetyClient

# 2. Load your Azure Safety API key and endpoint
load_dotenv()

key = os.environ["CONTENT_SAFETY_KEY"]
endpoint = os.environ["CONTENT_SAFETY_ENDPOINT"]

# 3. Create a Content Safety client
moderator_client = ContentSafetyClient(endpoint, AzureKeyCredential(key))

Here’s what you’ve done:

1. You included the modules and libraries for loading and accessing environmental variables to fetch Azure’s endpoint and key information. You also imported a class from the Azure library, which will allow you to create the content safety client.
2. Next, you’re loading the environment variables from the .env file that you’ll soon create in the coming steps using load_dotenv function. Then, you’re fetching and assigning azure key and endpoint values to key and endpoint variable respectively.
3. Finally, you’re creating the moderator_client that will be used to create and analyze calls by passing the endpoint and azure key credential to the ContentSafetyClient object.

If you observe, the import lines have these yellow curly lines on which when you hover over. It shows a warning that “Import “X” could not be resolved.” To fix this, run the following into the terminal and hit Enter

pip install azure-ai-contentsafety; pip install python-dotenv; pip install
  streamlit

Wait for the commands to run and install the requested packages. Next, add the .env file in your starter project’s root directory:

CONTENT_SAFETY_KEY=<your-content-safety-key>
CONTENT_SAFETY_ENDPOINT=<your-endpoint>

Make sure to replace <your-endpoint> and <your-content-safety-key> with the endpoint and safety key that Azure assigned to you when you created the resource. You imported the values to your main code just a while ago. These values will let your request through to your Azure Content Safety resource.

Add Text and Image Analysis Code

Once you’re done creating the moderation client, the next step will be to write the code to analyze text and image content. Open the starter/business_logic.py file again and replace # TODO: Check for the content safety with the following code:

# 1. Check for the content safety
text_analysis_result = analyze_text(client=moderator_client, text=text)
image_analysis_result = analyze_image(client=moderator_client, image_data=image_data)

# 2
## TODO: Logic to evaluate the content

1. You’re calling two functions analyze_text and analyze_image, to analyze text and image respectively. These two functions expect the following arguments: a) client - will be used to create the request, b) text or image_data - this is the data that needs to be analyzed.
2. Finally, you added the TODO comment, where you’ll place the actual logic of evaluating content soon.

Now, its time to create the analyze_text and analyze_image functions.

Add analyze_text Function

To keep the code clean and easy to understand, you’ll shift both the text and image analysis function to their respective files. Create a text_analysis.py file inside the root folder and add the following code:

# 1. Import packages
from azure.core.exceptions import HttpResponseError
from azure.ai.contentsafety.models import AnalyzeTextOptions, TextCategory,
  AnalyzeTextOutputType

# 2. Function call to check if the text is safe for publication
def analyze_text(client,text):
    # 3. Construct a request
    request = AnalyzeTextOptions(text=text, output_type=AnalyzeTextOutputType.
      EIGHT_SEVERITY_LEVELS)

    # 4. Analyze text
    try:
        response = client.analyze_text(request)
    except HttpResponseError as e:
        print("Analyze text failed.")
        if e.error:
            print(f"Error code: {e.error.code}")
            print(f"Error message: {e.error.message}")
            raise
        print(e)
        raise

    # 5. Extract results
    categories = {
        TextCategory.HATE: None,
        TextCategory.SELF_HARM: None,
        TextCategory.SEXUAL: None,
        TextCategory.VIOLENCE: None
    }

    for item in response.categories_analysis:
        if item.category in categories:
            categories[item.category] = item

    hate_result = categories[TextCategory.HATE]
    self_harm_result = categories[TextCategory.SELF_HARM]
    sexual_result = categories[TextCategory.SEXUAL]
    violence_result = categories[TextCategory.VIOLENCE]

    # 6. Check for inappropriate content
    violations = {}
    if hate_result and hate_result.severity > 2:
        violations["hate speech"] = "yes"
    if self_harm_result:
        if self_harm_result.severity > 4:
            violations["self-harm"] = "yes"
    if sexual_result:
        if sexual_result.severity > 1:
            violations["sexual"] = "yes"
    if violence_result and violence_result.severity > 2:
        violations["violent references"] = "yes"

    return violations

This code might look too much to understand in a single go, but you’ve built almost the same code in Lesson 2!

1. First, you’ll import the required packages and modules that you’ll need to perform the analysis on text.
2. Next, you define the function that will be used to analyze the text and check if the text is safe for publication.
3. Inside it, you’re constructing the request using AnalyzeTextOptions, which you’ll pass to the API to analyze the text. You’ve passed the text and the severity level type in the AnalyzeTextOptions, to analyze the provided text and share the severity level output at a more granular level.
4. You’re making an analyze text API call by calling the analyze_text method in the try-except block and handling any errors you encounter.
5. You’re then extracting the analysis result containing a list of harmful categories and their severity scores in separate variables.
6. Finally, you’re checking for inappropriate content by comparing the severity score for individual categories with their respective thresholds. Suppose the score is more than the specific threshold value — in that case, you add the harmful category in the dictionary with the "yes" value, indicating that the text has been detected as harmful and violating guidelines because of the presence of the following category. You then finally return the violation variable.

Now, the threshold values visible above are decided discretely for the sample project and may not represent the actual publication scenerios. So, feel free to test and improve the threshold values even more, if you wish :]

Add analyze_image File

Now, it’s time to move ahead and create analyze_image function. Create image_analysis.py file inside the root folder of the project and add the following code:

# 1. Import the packages
from azure.core.exceptions import HttpResponseError
from azure.ai.contentsafety.models import AnalyzeImageOptions, ImageData,
  AnalyzeImageOutputType, ImageCategory

# 2
def analyze_image(client, image_data):
    # 3. Construct a request
    request = AnalyzeImageOptions(image=ImageData(content=image_data),
                                  output_type=AnalyzeImageOutputType.
                                  FOUR_SEVERITY_LEVELS)

    # 4. Analyze image
    try:
        response = client.analyze_image(request)
    except HttpResponseError as e:
        print("Analyze image failed.")
        if e.error:
            print(f"Error code: {e.error.code}")
            print(f"Error message: {e.error.message}")
            raise
        print(e)
        raise

    # 5. Extract results
    categories = {
        ImageCategory.HATE: None,
        ImageCategory.SELF_HARM: None,
        ImageCategory.SEXUAL: None,
        ImageCategory.VIOLENCE: None
    }

    for item in response.categories_analysis:
        if item.category in categories:
            categories[item.category] = item

    hate_result = categories[ImageCategory.HATE]
    self_harm_result = categories[ImageCategory.SELF_HARM]
    sexual_result = categories[ImageCategory.SEXUAL]
    violence_result = categories[ImageCategory.VIOLENCE]

    # 6. Check for inappropriate content
    violations = {}
    if hate_result and hate_result.severity > 2:
        violations["hate speech"] = "yes"
    if self_harm_result and self_harm_result.severity > 4:
        violations["self-harm references"] = "yes"
    if sexual_result and sexual_result.severity > 0:
        violations["sexual references"] = "yes"
    if violence_result and violence_result.severity > 2:
        violations["violent references"] = "yes"

    return violations

Quickly going through the code:

1. You import the required packages and modules that you’ll need to analyze the image.
2. Next, you define the function that will be used to analyze the image, and check whether the provided image is safe for publication or not.
3. Then, you construct the request using AnalyzeImageOptions and passed the required argument. You’ll use this request variable to pass to the API to analyze the image.
4. Next, you create the analyze image API call by calling the analyze_image method in try-except block and handling any errors you encountered.
5. Then, you extract the analysis results, containing a list of harmful categories and their respective severity scores in separate variables.
6. Finally, you’re checking for inappropriate content by comparing the severity score for individual categories with their respective thresholds and adding the harmful category detected to the violation variable (if the score is more than the specific threshold value). You then finally returned the violation variable.

Implement the Logic to Evaluate the Content

Now, you’re ready to integrate everything and finalize your moderation function for the app. Head back to the file starter/business_logic.py and replace ## TODO: Logic to check evaluate the content with the following:

# 1
if len(text_analysis_result) == 0 and len(image_analysis_result) == 0:
    return None

# 2
status_detail = f'Your post contains references that violate our community guidelines.'

if text_analysis_result:
    status_detail = status_detail + '\n' + f'Violation found in text: {','
      .join(text_analysis_result)}'
if image_analysis_result:
    status_detail = status_detail + '\n' + f'Violation found in image: {','
      .join(image_analysis_result)}'

status_detail = status_detail + '\n' + 'Please modify your post to adhere to
  community guidelines.'


# 3
return {'status': "violations found", 'details': status_detail}

Here’s an explanation of the code:

1. You’re checking whether the dictionary data received from text and image analysis is empty. If both are found empty, no harmful category is detected that could potentially violate the community guidelines — meaning the content is safe.
2. If either of the following results detects harmful content, then the rest of the code is executed. You’ve defined a new variable, status_detail, and appended the harmful category to the string in a human-readable format when detected, so that the user can be informed about it. You also requested that the post be updated to adhere to community guidelines.
3. Finally, you return the result of the safety check, so that the user can be informed about the violation found in the content — and request them to update the content to address the shared concerns.

Now, import the two functions to eliminate any “not defined” errors:

from text_analysis import analyze_text
from image_analysis import analyze_image

That’s it!

With this, you’ve finished the implementation of the multi-modal content moderation system for your Fooder app. When a user posts a recipe or comment, your app calls the check_content_safety function, analyzes both images and text, and determines if the content is safe for publishing. If it’s found to be unsafe, the user is informed about it, and requested to update the content to address the concerns raised.

Text the Moderation System

Enough with the coding for now — let’s run the app and see the code in action!

Open the VSCode terminal and run the following command:

streamlit run app.py

Add the image and text from the sample-data directory provided to you in the azs-student-materials directory. Finally, click the Submit button.

Congratulations! You’ve built a multi-modal content moderation solution for your Fooder app. It’s robust, can handle a large amount of traffic, and will take away all your worries related to any potentially wrong practices. Right?

Unfortunately, that’s not always the case. You’ll learn more about this in the next segment…