Python for AI: A Crash Course
Demo
Retrieving Information from Web Resources
You may occasionally need to GET the contents of web pages to “scrape” text information from sites that wouldn’t otherwise be accessible via an API. Fortunately, the requests library is equally useful for retrieving the contents of web pages since it’s for accessing APIs.
One example use case is performing sentiment analysis on messages in online forums. You could GET the content of pages on a particular topic, use string methods to extract relevant information, and then either use that information to train a model or provide that information to an LLM for analysis.
In this demo, you’ll create requests for getting data through webpages, and getting and setting data through APIs. Start Jupyter Lab and open the file, REST.ipynb.
GETting a Web Page from a URL
Start with the kind of GET request that billions of people use daily: a request for the contents of a web page. In this case, we’ll use a GET request to retrieve the contents of Kodeco’s home page.
Enter the following into a code cell:
import requests
response = requests.get("https://kodeco.com/")
if response.status_code == 200:
print("Successfully retrieved the web page!")
print(f"Status code: {response.status_code}.")
print(f"Content type: {response.headers["Content-Type"]}.")
print("=====")
print(response.text)
else:
print(f"Failed to retrieve the web page. Status code: {response.status_code}.")
You’ll see this:
Successfully retrieved the web page!
Status code: 200.
Content type: text/html; charset=utf-8.
=====
<!DOCTYPE html>
(A whole lot of HTML goes here.)
This code sends a GET request to Kodeco’s server for the resource located at the URL https://kodeco.com/, which doesn’t include a pathname or filename. Since the URL ends with a directory, not a filename, it returns the default resource located at kodeco.com, the HTML file defining Kodeco’s home page.
Because a resource exists at kodeco.com, the server can fulfill the GET request made by the code. As a result, the server’s response includes the status code 200, meaning “OK” and indicating success. This value is contained in the response object’s status_code property.
Although you know that kodeco.com is the location of Kodeco’s home page, the code confirms that the resource is HTML data. It does this by accessing the response object’s headers property, which contains the response headers in Python dictionary form. The Content-Type header — the value corresponding to headers’ Content-Type key — for this request is text/html; charset=utf-8, indicating that the response is HTML content using the UTF-8 encoding standard, which supports a wide array of characters.
Since the response contains HTML data, you use the response’s text property to access the response body, which returns it as a string. The code then displays this string.
Your browser executes a similar GET request when you point it at kodeco.com. The difference is that it renders the HTML as a web page instead of simply displaying it.
GETting a Web Page from an Invalid URL
What happens when you try to GET from a URL that doesn’t correspond to a web resource? You can find out by changing the URL in the previous code to one for a resource that doesn’t exist.
Update the definition of the response variable in the previous code to the following, then run it in a new cell:
import requests
response = requests.get("https://kodeco.com/non-existent-page")
if response.status_code == 200:
print("Successfully retrieved the web page!")
print(f"Status code: {response.status_code}.")
print(f"Content type: {response.headers["Content-Type"]}.")
print("=====")
print(response.text)
else:
print(f"Failed to retrieve the web page. Status code: {response.status_code}.")
You’ll see this result:
Failed to retrieve the web page. Status code: 404.
This time, since the code attempts to GET a web resource that doesn’t exist, the server’s response includes the well-known status code 404, which means “Not found.”
In the case of a web page, some content may still be included in the response. Many websites, Kodeco’s included, present the user with a “404” page when they try to navigate to a URL that doesn’t have a corresponding web page. You’ll find the HTML for Kodeco’s “404” page in the text property of the response object.
Making the Simplest Possible GET Request
The Star Wars API is fun to play with. It’s an API that acts as an encyclopedia of Star Wars people, species, planets, vehicles, spaceships, and films, or at least episodes one through seven. You don’t need to pay or register for an API key to use it — you can start making calls to it. The API has a website at swapi.dev and its base URL is swapi.dev/api.
This API is a rather friendly design. If you make a GET request to the base URL, you’ll receive a JSON object listing all the API endpoints.
Make this call by entering the following code in a new cell and running it:
import json
SWAPI_BASE_URL = "https://swapi.dev/api/"
response = requests.get(SWAPI_BASE_URL)
if response.status_code == 200:
data = response.json()
pretty_json = json.dumps(data, indent=4, sort_keys=True)
print(pretty_json)
When you run the code above, you’ll see a dictionary listing the Star Wars API’s endpoints, where the keys are the names of the endpoints and the values are the corresponding endpoint URLs. You’ll use one of these URLs shortly.
Since you’re calling an API rather than retrieving a web page’s contents, the code uses the response object’s json() method instead of its text property. The json() method automatically converts the JSON text in the response into Python lists and dictionaries, from which you can extract the data you need.
To display the retrieved JSON data in a readable format, the code uses the json.dumps() method to convert it into a string representation. Using the optional indent parameter makes json.dumps() format the string using one line per list or dictionary element, using the specified number of spaces for each level of indentation. Setting the optional sort_keys parameter to True causes json.dumps() to present dictionary keys in ascending alphabetical order.
Making a GET Request with Path Parameters
Use one of the Star Wars API endpoints: “people”.
The Star Wars API uses path parameters, which means that you make requests by appending the ID of the resource you want to the end of the URL for that resource’s endpoint. In the case of people, you append the ID of the person whose information you want to the “people” endpoint and use the resulting URL for a GET request.
You’ve probably already guessed who the Star Wars person with the ID of 1 is. You can confirm it by running the following in a new code cell:
PEOPLE_ENDPOINT = "people/"
person_id = 1
response = requests.get(f"{SWAPI_BASE_URL}{PEOPLE_ENDPOINT}{person_id}")
if response.status_code == 200:
data = response.json()
pretty_json = json.dumps(data, indent=4)
print(pretty_json)
You’ll see an extensive dictionary full of information about the Star Wars person with the ID of 1: Luke Skywalker himself. The JSON object in the response has been converted into a Python dictionary that contains all sorts of information about the character.
You may have noticed that some of the information about Luke, such as his homeworld, contains one or more URLs. These URLs act as “pointers” to where that particular information is accessible through the API. To get this information, you perform another GET request using these URLs. Run the code below to get information about Luke Skywalker’s homeworld:
PEOPLE_ENDPOINT = "people/"
person_id = 1
person_response = requests.get(f"{SWAPI_BASE_URL}{PEOPLE_ENDPOINT}{person_id}")
if person_response.status_code == 200:
person_data = person_response.json()
homeworld_endpoint = person_data["homeworld"]
homeworld_response = requests.get(homeworld_endpoint)
if homeworld_response.status_code == 200:
homeworld_data = homeworld_response.json()
pretty_json = json.dumps(homeworld_data, indent=4)
print(pretty_json)
You’ll see a dictionary containing information about Luke’s homeworld, Tatooine.
The code above first GETs Luke Skywalker’s information, which is a dictionary. From that dictionary, it takes the value corresponding to the homeworld key, a string containing the URL for the resource representing Luke’s homeworld. The code GETs this resource and displays it.
Making a GET Request with Query Parameters
Now, take a look at an API that uses query parameters, where you provide parameters in the form of a query string at the end of the endpoint URL. The API in question is the Open-Meteo weather API (meteo is the French word for “weather”). Like the Star Wars API, the Open-Meteo API is free of charge and doesn’t require registering for an API key.
Suppose you want to know the current weather in London, England. To do so, you need to provide Open-Meteo with the following parameters:
-
latitude: The location’s latitude. Positive values mean degrees north of the equator, negative values mean degrees south of the equator, and 0 means the equator. For this example, you’ll use the value51.5072. -
longitude: The location’s longitude. Positive values mean degrees east of the prime meridian, negative values mean degrees west of the prime meridian, and 0 means the prime meridian. For this example, you’ll use the value-0.1276. -
current: The weather values for the current weather that you want to retrieve (listed on Open-Meteo’s API docs page), in the form of a comma-separated list. For this example, you’ll request three values:-
weathercode: A number code for the current weather at the requested location. -
temperature_2m: The air temperature at the requested location, as measured at a height of 2 meters, about 6 feet, above the ground. This value is in degrees Celsius. -
relativehumidity_2m: The relative humidity at the requested location, as measured at 2 meters, about 6 feet, above the ground. This value is expressed as a percentage — between 0 and 100 inclusive.
-
Open-Meteo’s endpoint URL for the current weather at a given location is https://api.open-meteo.com/v1/forecast/. One way to form the URL for the GET request is to build the URL string starting with the endpoint URL, adding a ? to the end to denote the start of the query parameters, followed by the query parameters. Here’s the URL for your example.
While this approach works, it’s cumbersome and error-prone. It’s much better to send the GET request to the endpoint URL and use the params parameter of the request.get() method to define the query parameters. Run the following in a new code cell:
WEATHER_ENDPOINT = "https://api.open-meteo.com/v1/forecast"
parameters = {
"latitude": 51.5072,
"longitude": -0.1276,
"current": "weathercode,temperature_2m,relativehumidity_2m",
}
response = requests.get(WEATHER_ENDPOINT, params=parameters)
if response.status_code == 200:
data = response.json()
pretty_json = json.dumps(data, indent=4)
print(pretty_json)
The part of the response you’re most interested in is the value for the "current" key, a dictionary containing the values for the weathercode, temperature_2m, and relativehumidity_2m parameters.
To translate the weathercode value into something more meaningful than a number, enter the following into a code cell and run it:
WEATHER_CODE_TABLE = {
0: "clear sky",
1: "mainly clear",
2: "partly cloudy",
3: "overcast",
45: "fog",
48: "depositing rime fog",
51: "light drizzle",
53: "moderate drizzle",
55: "dense drizzle",
56: "light freezing drizzle",
57: "dense freezing drizzle",
61: "slight rain",
63: "moderate rain",
65: "heavy rain",
66: "light freezing rain",
67: "heavy freezing rain",
71: "slight snow",
73: "moderate snow",
75: "heavy snow",
77: "snow grains",
80: "light rain showers",
81: "moderate rain showers",
82: "violent rain showers",
85: "slight snow showers",
86: "heavy snow showers",
95: "thunderstorm",
96: "thunderstorm with slight hail",
99: "thunderstorm with heavy hail",
}
print(f"The weather in London is: {WEATHER_CODE_TABLE[data["current"]["weathercode"]]}")
Adding, Updating, and Deleting Data in APIs
In this part of the demo, you’ll use ReqRes, a REST API designed specifically for testing applications that make calls to an API. You can send GET, POST, PUT, PATCH, and DELETE requests to its endpoints, and it’ll provide a simulated response. As such, it’s perfect for trying out requests other than GET.
ReqRes has a users endpoint, located at https://reqres.in/api/users/, which you’ll use to add, update, and delete resources in the API.
Adding a Resource
Pretend that you want to add a new user to the database: Guido van Rossum, the creator of Python. You’ll provide his name and programming language as parameters by making a POST request to create a new resource in the API.
Enter the code below into a new code cell and run it:
USERS_ENDPOINT = "https://reqres.in/api/users/"
post_data = {
"language": "Python",
"creator": "Guido van Rossum",
}
response = requests.post(USERS_ENDPOINT, data=post_data)
if response.status_code == 201:
print(f"Successfully POSTED the resource. Status code: {response.status_code}.")
print("=====")
data = response.json()
pretty_json = json.dumps(data, indent=4)
print(pretty_json)
else:
print(f"Failed to POST the resource. Status code: {response.status_code}.")
The response contents are typical of many APIs that take POST requests. They contain the ID of the newly created resource, in this case, a new record for Guido van Rossum, the information you provided as POST data and the date and time when the resource was created.
Note that in the code above, you checked for a response status code of 201, not 200. That’s because the 201 status code indicates the successful creation of a new resource.
Updating a Resource
There are two ways to update a resource in an API. The first is to make a PUT request, which updates all the data in the resource and requires you to provide all the data.
Suppose you want to update the entry for the user whose ID is 123 by overwriting it with new data: the creator of the Ada programming language, Jean Ichbiah.
Run the following in a new code cell:
path_parameter = 123
url = f"{USERS_ENDPOINT}{path_parameter}"
put_data = {
"language": "Ada",
"creator": "Jean Ichbiah",
}
response = requests.put(url, data=put_data)
if response.status_code == 200:
print(f"Successfully PUT the resource. Status code: {response.status_code}.")
print("=====")
data = response.json()
pretty_json = json.dumps(data, indent=4)
print(pretty_json)
else:
print(f"Failed to PUT the resource. Status code: {response.status_code}.")
The response contents are typical for APIs that accept PUT requests. It’ll contain all the data for the updated resource and the date and time when the update was made.
The other way to update a resource is to make a PATCH request, which updates only certain data in the resource and requires you to provide only the data to be updated.
Suppose you want to update the entry you just updated so that the language property specifies that the Ada programming language is named after Ada Lovelace, famous for her work on Charles Babbage’s analytical engine. Do this by entering the code below into a new code cell and running it:
path_parameter = 123
url = f"{USERS_ENDPOINT}{path_parameter}"
patch_data = {
"language": "Ada (named after Ada Lovelace)",
}
response = requests.put(url, data=patch_data)
if response.status_code == 200:
print(f"Successfully PATCHed the resource. Status code: {response.status_code}.")
print("=====")
data = response.json()
pretty_json = json.dumps(data, indent=4)
print(pretty_json)
else:
print(f"Failed to PATCH the resource. Status code: {response.status_code}.")
The response’s contents are typical for APIs that accept PATCH requests. It’ll contain the updated resource data and the date and time when the update was made.
Deleting a Resource
Now, delete the resource with the ID of 123 using the DELETE request. Run the following in a new code cell to do this:
path_parameter = 123
url = f"{USERS_ENDPOINT}{path_parameter}"
response = requests.delete(url)
if response.status_code == 204:
print(f"Successfully DELETEd the resource. Status code: {response.status_code}.")
else:
print(f"Failed to DELETE the resource. Status code: {response.status_code}.")
Note that in the code above, you checked for a response status code of 204, not 200. That’s because the 204 status code indicates a resource’s successful deletion.