23. Just Enough Web Stuff

Written by Audrey Tam

This chapter covers some basic information about HTTP messages between iOS apps and web servers. It’s just enough to prepare you for the following chapter, where you’ll implement downloads from the metmuseum.org server.

There’s no SwiftUI in this chapter.

If you already know all about HTTP messages, skip down to the section “Exploring metmuseum.org” to familiarize yourself with the API you’ll use in the following chapters.

Servers & Resources

Many apps communicate with computers on the internet to access databases and other resources. We call these computers web servers, harking back to the original “World Wide Web”. Or cloud servers because nowadays everything is “in the Cloud”. “Host” is another term for “server”.

Apps like Safari and TheMet are clients of these servers. A client sends a request to a server, which sends back a response. This communication consists of plain-text messages that conform to the Hypertext Transfer Protocol (HTTP). Hypertext is structured text that uses hyperlinks between nodes containing text. Web pages are written in HyperText Markup Language (HTML).

HTTP has several methods, including POST, GET, PUT and DELETE. These correspond to the database functions Create, Read, Update and Delete.

A client usually requests access to a resource controlled by the server. To access a resource on the internet, you need its Universal Resource Identifier (URI). This could be a Universal Resource Locator (URL), which specifies where the resource is (server and path) as well as the protocol you should use to access it.

For example, https://www.metmuseum.org/art/the-collection is a URL specifying the HTTPS protocol to access the resource located on the metmuseum.org server with the path art/the-collection.

Note: HTTPS is the secure, encrypted version of HTTP. It protects your users from eavesdropping. The underlying protocol is the same but, instead of transferring plain-text messages, everything is encrypted before it leaves the client or server.

HTTP Messages

A client’s HTTP request message contains headers. A POST or PUT request has a body to contain the new or updated data. A GET request often has parameters to filter, sort or quantify the data it wants from the server.

A server’s HTTP response message also has headers and a body. A key part of the response is the status code — ideally, 200 OK in response to a GET request or 201 Created in response to a POST request. You don’t want to see any error status codes like 404 Not Found:

There are many many HTTP response status codes. You’ll find a fun representation of them at http.cat. For example:

Mozilla (mzl.la/3o6qWNM) provides more conventional descriptions of status codes:

The HTTP 418 I’m a teapot client error response code indicates that the server refuses to brew coffee because it is, permanently, a teapot. A combined coffee/tea pot that is temporarily out of coffee should instead return 503. This error is a reference to Hyper Text Coffee Pot Control Protocol defined in April Fools’ jokes in 1998 and 2014. Some websites use this response for requests they do not wish to handle, such as automated queries.

Note: The 1998 HTCPCP April Fools’ joke was inspired by the Trojan Room coffee pot, the subject of the world’s first web cam. It was set up in 1991, long before the Internet of Things (IoT).

If an HTTP message has a body, it also has a Content-Type header. Content-Type specifies the internet media type of the data in the HTTP message body.

Usually, you’ll work with three content types for text data, depending on the structure:

• JSON (JavaScript Object Notation) is the most common data format used for HTTP communication by app clients. It’s a structured data format consisting of numbers, strings, and arrays and dictionaries that can contain strings, numbers and nested arrays and dictionaries.
• Web forms use form-encoded, which looks like a query string. A query string is a collection of key-value pairs, separated by & and preceded by ?.
• Web pages are HTML.

When working with binary data some of the most used types are PDF, image formats and multi-part form data, when the client sends any kind of binary file along with text elements.

REST API

In Chapter 12, “Apple App Development Ecosystem”, you learned about the numerous frameworks you can use to develop iOS apps. An Apple framework is one kind of Application Programming Interface (API). It tells you how to use the standard components created by Apple engineers.

Another kind of API is the set of rules for clients to request resources from a server. Most of the APIs you’ll use for your apps are REST APIs, which use HTTP. For each resource available on the server, the REST API documentation tells you how to construct a request:

• The resource’s URL, called its endpoint.
• Which HTTP method to use.
• Which HTTP headers to include.
• What to put in the request body.

Note: REST is the acronym of “REpresentational State Transfer”, the name created by Roy Fielding for the architectural style underlying the World Wide Web. The term describes how a well-designed Web application works: A user selects a resource identifier from a network of Web resources (a virtual state-machine) and uses methods like GET or POST to create a state transition that transfers the resource’s representation to the user.

In the next chapter, you’ll set up TheMet to communicate with the museum’s REST API. In this chapter, you’ll explore this API’s documentation.

Sending & Receiving HTTP Messages

Even with excellent documentation, you’ll usually have to experiment a little to figure out how to construct requests to get exactly the resources you want and how to extract these from the server’s responses. So how do you send requests and examine responses?

Browser

The easiest way to make a simple HTTP GET request is to enter the URL in a browser app like Safari.

➤ Enter this URL in your favorite browser:

https://www.metmuseum.org/art/the-collection

This is the endpoint of the metmuseum.org art collection. You get a page similar to this:

This is the body of the server’s response, but you don’t get to see the headers. And, you can’t do much more than a simple GET request.

cURL

A browser is a fully-automated HTTP tool. At the other end of the spectrum is the command-line tool cURL — “the internet transfer backbone for thousands of software applications”.

The documentation for a REST API often provides sample requests to show you how to use it. Very often, these use cURL.

➤ Open Terminal and enter this command:

curl https://api.github.com/zen

You send an HTTP request to GitHub’s API server. The response is a random item from their design philosophies, like “Favor focus over features” or “Avoid administrative distraction”. There are lots more request examples at GitHub’s Getting started with the REST API.

But, you exclaim, curl doesn’t show any response headers either! Well, like all Unix commands, curl has a wealth of options, including --include and its shortcut -i, to include the HTTP response headers in its output.

➤ Enter this command:

curl -i https://api.github.com/zen

And you see quite a lot more output:

HTTP/2 200
date: Tue, 19 Nov 2024 03:17:38 GMT
content-type: text/plain;charset=utf-8
...
server: github.com
x-ratelimit-limit: 60
x-ratelimit-remaining: 58
x-ratelimit-reset: 1731989820
x-ratelimit-resource: core
x-ratelimit-used: 2
accept-ranges: bytes
content-length: 35
x-github-request-id: D731:2D7D45:54002E:59E74E:673C0352

Approachable is better than simple.

Headers beginning with x- are custom headers set up by the organization. For example, x-ratelimit-limit and x-ratelimit-used indicate how many requests a client can make in a rolling time period (typically an hour) and how many of those requests the client has already made.

The curl --verbose or -v option displays request headers and a lot more.

➤ Enter this command:

curl -v https://api.github.com/zen

Replacing -i with -v produces quite a lot more output — every handshake interaction between the terminal and the server, the encryption algorithms used, the server certificate details, as well as the response headers. The request headers are just the five lines that start with >:

> GET /zen HTTP/2
> Host: api.github.com
> User-Agent: curl/8.7.1
> Accept: */*
>

Lines that start with < are response headers, and lines that start with * are additional information provided by cURL.

You might not enjoy typing long structured command lines, especially something like this sample cURL command to create a new GitHub repository:

curl -i -H \
  "Authorization: token 5199831f4dd3b79e7c5b7e0ebe75d67aa66e79d4" \
  -d '{ \
      "name": "blog", \
      "auto_init": true, \
      "private": true, \
      "gitignore_template": "nanoc" \
    }' \
  https://api.github.com/user/repos

This POST command sends authorization data in a request header and the request body as data in JSON format. The endpoint doesn’t name a specific user because GitHub knows that from the token value.

Another problem with using cURL: If the response is complex, it’s hard to examine it in the terminal.

➤ Enter this command:

curl https://collectionapi.metmuseum.org/public/collection/v1/objects/437133

This is a request to the API you’ll use for TheMet. The response is pretty mind-numbing:

If you concentrate, you might be able to see from this output that the response body is a dictionary where a couple of items are arrays of dictionaries. You can use a tool like codebeautify.org/jsonviewer to format and beautify this so it’s easier to read.

But there’s a better solution: apps that make your HTTP messaging easier.

Exploring metmuseum.org

Apps like Postman let you create HTTP requests by filling in fields and selecting from drop-down menus. You can pretty-print responses, and this also gives you syntax highlighting.

➤ Download Postman for your Mac’s chip and open the app. If you don’t want to create an account, click Continue without an account, then tap Open Lightweight API Client. Press *Command-* to hide the sidebar:

Requesting Objects

➤ In Postman, enter this URL in the GET field:

https://collectionapi.metmuseum.org/public/collection/v1/objects/437133

You set the resource endpoint. How do you know what to ask for? Scroll down in metmuseum.github.io: In the Endpoints section, you’ll see four API endpoints: Objects, Object, Departments and Search. Scroll further to see, for each endpoint, its description, request parameter list, request endpoint, response field list and examples of requests and responses.

The URL you entered in the GET field is an Object request for the example objectID:

➤ Back in Postman, click Send, then drag the response window up and select Body and Pretty:

Scrolling through the response body, it’s much easier to see all the keys in the top level dictionary: objectID, isHighlight and so on. Your app needs only a few of these keys.

➤ Click Headers to see the status code 200 OK, response time, size and other response headers:

Content-Type is application/json; charset=UTF-8. The key information here is json. This tells you how to decode the response body. In the next chapter, you’ll use JSONDecoder to extract the attributes you want and store them in the Object structure so you can display them in your app.

Note: UTF-8 string encoding is a version of Unicode that is very efficient for storing regular text, but less so for special symbols or non-Western alphabets. Still, it’s the most popular way to deal with Unicode text today.

Media URLs

➤ Go back to the Body tab: This object isn’t in the public domain, so it has empty strings for its primary image values. In the GET field, replace the object ID 437133 with our old friend 452174, then click Send:

This is the “Bahram Gur Slays the Rhino-Wolf” object. In the previous chapter, you used AsyncImage to download its primaryImageSmall.

➤ The value of the primaryImageSmall key is a link. Click it to insert it into the GET field, then send the request.

Postman is able to display the image. You can also Command-click the link to open it in your browser.

➤ Select the response’s Headers tab.

The Content-Type is now image/jpeg.

➤ At the top-level, click the + to add a new request and send a request for this URL:

https://collectionapi.metmuseum.org/public/collection/v1/search?q=rhino

This request has a query parameter with key q and value rhino. The response contains an objectIDs array and the total number of object IDs in the array.

This is the request you’ll send from your app when the user enters a query term. Then, to display the list of matching objects, you’ll request the object for each object ID in the objectIDs array.

URL-encoding

➤ Now, send a search request for “rhino wolf”:

https://collectionapi.metmuseum.org/public/collection/v1/search?q=rhino wolf

➤ Below the response window, open the Console:

The console shows the “official” request URL, where the query string is actually rhino%20wolf.

Postman URL-encoded the space between rhino and wolf to %20. URLs sent over the internet can contain only letters, digits and these punctuation marks: -, _, . and ~.

Other punctuation marks, including /, ? and %, are encoded as a pair of hexadecimal digits preceded by the escape character %. The hexadecimal value is the character’s byte value in ASCII, for example, 20 (32 in decimal) for the space character and 25 (37 in decimal) for the % character. The space character can also be encoded as +. For a non-ASCII character, URL-encoding uses its UTF-8 byte value.

When / and ? are delimiters in the URL, they don’t get encoded.

POST Request & Authentication

TheMet doesn’t need anything from this section, but your future apps might.

TheMet only needs to GET resources from the server, and your users don’t need to authenticate.

You usually need to implement authentication for apps that let users access restricted materials or create, update or delete server records. If you’re building an app that requires this capability, consider using Sign In with Apple. You can learn more by following our video course Sign in with Apple.

To try out a POST request, you’ll use Postman to send something like this GitHub curl example:

curl -i -H \
  "Authorization: token 5199831f4dd3b79e7c5b7e0ebe75d67aa66e79d4" \
  -d '{ \
      "name": "blog", \
      "auto_init": true, \
      "private": true, \
      "gitignore_template": "nanoc" \
    }' \
  https://api.github.com/user/repos

This example shows how to create a new GitHub repository, so it requires GitHub-user authentication. Remember when you set up your GitHub account in Xcode, you had to generate a personal access token? You’ll need one here, too.

➤ If you haven’t saved a plain-text copy of your GitHub personal access token, generate a new one with repo scope:

➤ In Postman, select the request’s Authorization tab, then select Bearer Token from the Type menu. Paste your personal access token in the Token field:

➤ Set the method to POST and the endpoint to https://api.github.com/user/repos. In the Body tab, select form-data and set the following keys and values:

• KEY: name, VALUE: api-test-repo
• KEY: auto_init, VALUE: true

➤ Click Send:

This was a deliberate “oops” to show you what happens if the server expects the POST request body to be in JSON but you send form-encoded data instead. You get a helpful link to a documentation_url.

➤ In the request Body tab, select raw, set the format to JSON, then type this in the text view:

{
  "name": "api-test-repo",
  "auto_init": true
}

➤ Click Send:

That worked! Check your GitHub account to see there really is a new repository named api-test-repo:

➤ Click Send again.

If you try to create the same repo again, the server returns the error message “name already exists on this account”.

Key Points

• Client apps send HTTP requests to servers, which send back responses.

• An HTTP response from an API endpoint contains a status code and some content. Text content is usually in JSON format and may contain URIs the client app can use to access media resources.

• HTTP requests follow the rules of the server’s REST API, whose documentation specifies resource endpoints, HTTP methods and headers, and how to construct POST and PUT request bodies.

• You can send simple GET requests in a browser app. Use cURL or an app like Postman to create and send requests and inspect responses.