OpenAI's Chat Completion API - Instruction

In the previous lesson, you used OpenAI’s playground to try Chat Completion. Now, you’ll use the API directly. This is useful for text generation in your own apps.

OpenAI Package

Open Jupyter Lab and navigate to your notebook. Then, add the following code:

%pip install openai

This will install the openai package. The % symbol refers to a magic command. In Jupyter, %pip is a magic command to run the pip package manager within the current kernel. To learn more about magics in Jupyter, go here.

In simple terms, kernel is your Python environment that runs your code. It includes packages that you installed previously and Python with a specific version. To read more about kernels in Jupyter, read up here.

Run the cell. You should see either a new package being installed or pip telling you that it’s already installed.

After running, you might need to restart the kernel. Restart it now.

In the first lesson, you learned to securely store your OpenAI key and load it in Jupyter. The package you installed earlier requires this key.

Add a new cell and insert the following code:

import os
import openai
openai.api_key = os.environ["OPENAI_API_KEY"]

The first two lines are just to import packages. The last line is about setting the API key in the openai package with the environment variable OPENAI_API_KEY.

Now run the cell.

You won’t see any output, but you’ve successfully set up the key for the openai package.

Now, you still need the client to start making completions. Add a new cell and the following code:

from openai import OpenAI

client = OpenAI()

Well done! You’re ready for chat completion.

Chat Completion API

What if you want to generate code with a prompt? Like writing a hello world in Python. You can do that with the chat completion API.

In the same cell, insert the following:

response = client.chat.completions.create(
  model="gpt-4o-mini",
  messages=[
    {"role": "user", "content": "Write hello world in Python"}
  ]
)

print(response)

Here, you’re calling the chat completion with the model gpt-4o-mini. You also passed an array of messages with one message. The message has a role and content. The content field contains your prompt, e.g. writing code for Python. Notice the role, user.

Run the cell. You should see something similar to the image below.

It’s like ‘Where’s Waldo,’ but somewhere in the printed output, there’s print("Hello, World!"). To extract only the output, add the following code:

print(response.choices[0].message.content)

Run the cell again. You should see just the content, like below:

If it doesn’t exactly match what you see, don’t worry. As you might have noticed already, the text generation outputs from OpenAI are a bit random.

Learning the Roles

Earlier, you should’ve noticed the role field in the messages parameter of the completion API. The role user means that the message is coming from the user. Here are the other roles:

• system: This is optional, but can prime your responses. For example you can tell it instructions on how to perform a task, including tone, format, and giving it examples.
• user: This is your user’s messages.
• assistant: This means the message came from a previous response from the completion API. It can also be used to give examples of responses.

Note that since assistant is a role for messages coming from the API, it can also be used to refer to the API.

You’ll use the system role in a later lesson since it can provide more advanced behaviors.

Looking at Parameters

You might recall from earlier, that there’s some randomness to the output of the completion API. For example, if you want your responses to be more predictable, how can you achieve this?

You set the model parameter of the completion API earlier. You might be wondering if there are more parameters you can change. Indeed there are, and some are helpful in controlling the randomness of the output.

Here are some of the most useful ones:

• seed: An optional integer, to help produce the same output from the same inputs, set it to the same seed. Currently being tested and doesn’t guarantee determinism.
• temperature: A number between 0-2. Closer to zero gives a less random output, and closer to two gives more random outputs.
• stream: You might have used ChatGPT and see that the output looks like someone is typing it in front of you. You can achieve that effect by setting this parameter to true. It’s also helpful if you want to produce incomplete outputs to your user to provide a sense of responsiveness.
• tools: An array of objects. You can use this to provide functions that completion can choose from. For example, getting content from the internet, performing arithmetic, formatting text into JSON, and so on.

To get a full overview of the parameters available to the chat completion API, refer to this page.

Handling Errors

Remember that you looked at your account limits in an earlier lesson. What if you reach those limits? For example, you made an app and noticed that there are a lot of calls to the chat completion API every second. That sounds like your app is getting popular! But, your users might see unexpected behavior afterward. You don’t want this, of course.

In order to prepare for possible errors that might occur, you should add code to handle known errors.

Try turning off your internet. Then, run the last cell.

You should see an error like this. This is because the chat completion API needs the internet, that you turned off, to function. A better way is to expect that this can happen and display a friendly message to the user.

Create a new cell. Add the following code to handle the connection error:

# 1
try:
  # 2
  response = client.chat.completions.create(
      model="gpt-4o-mini",
      messages=[
        {"role": "user", "content": "Write hello world in python"}
      ]
  )
  print(response.choices[0].message.content)
# 3
except openai.APIConnectionError as e:
  print(f"What do we want? Faster Internet! When do we want it? Now!")

Here’s what you did:

1. Start a try catch to wrap the chat completion call.
2. The same call to write hello world using the chat completion call from the before.
3. Catch the APIConnectionError exception and print a friendly message.

Now run the cell again while keeping your internet off, and you should see the message below.

What do we want? Faster Internet! When do we want it? Now!

Don’t forget to turn your internet back on. :]

What if you want to catch that rate limit error that you heard about earlier? Don’t worry, and insert the following code into the previous one:

# 1
try:
...
except openai.APIConnectionError as e:
  ...
# 2
except openai.RateLimitError as e:
  print(f"DON'T PANIC! slow down and try again later.")
# 3
except openai.APIError as e:
  print(f"Does not compute 🤖.")

Here’s a rundown:

1. The previous try-except code.
2. You handle the RateLimitError exception and print a message.
3. You handle all APIError exceptions from openai, e.g. RateLimitError inherits from APIStatusError, which inherits from `APIError. Then, you print a message.

To see all the error codes from the chat completion API, go to this reference

Well done. Now you know the basics of the chat completion API!