23. Just Enough Web Stuff

Written by Audrey Tam

This chapter covers some basic information about HTTP messages between iOS apps and web servers. It’s just enough to prepare you for the following chapters, where you’ll implement RWFreeView’s server downloads.

There’s no SwiftUI in this chapter.

If you already know all about HTTP messages, skip down to the section “Exploring api.raywenderlich.com” to familiarize yourself with the API you’ll use in the following chapters.

Servers and resources

Many apps communicate with computers on the internet to access databases and other resources. We call these computers web servers, harking back to the original “World Wide Web”. Or cloud servers because nowadays everything is “in the Cloud”. “Host” is another term for “server”.

Apps like Safari and RWFreeView are clients of these servers. A client sends a request to a server, which sends back a response. This communication consists of plain-text messages that conform to the Hypertext Transfer Protocol (HTTP). Hypertext is structured text that uses hyperlinks between nodes containing text. Web pages are written in HyperText Markup Language (HTML).

HTTP has several methods, including POST, GET, PUT and DELETE. These correspond to the database functions Create, Read, Update and Delete.

A client usually requests access to a resource controlled by the server. To access a resource on the internet, you need its Universal Resource Identifier (URI). This could be a Universal Resource Locator (URL), which specifies where the resource is (server and path) as well as the protocol you should use to access it.

For example, https://raywenderlich.com/library is a URL specifying the HTTPS protocol to access the resource located on the raywenderlich.com server with the path library. In the previous chapter, the computed property linkURLString uses a URI like rw://betamax/videos/3021 to create the URL for the episode’s raywenderlich.com page.

Note: HTTPS is the secure, encrypted version of HTTP. It protects your users from eavesdropping. The underlying protocol is the same but, instead of transferring plain-text messages, everything is encrypted before it leaves the client or server.

HTTP messages

A client’s HTTP request message contains headers. A POST or PUT request has a body to contain the new or updated data. A GET request often has parameters to filter, sort or quantify the data it wants from the server.

A server’s HTTP response message also has headers and a body. A key part of the response is the status code — ideally, 200 OK in response to a GET request or 201 Created in response to a POST request. You don’t want to see any error status codes like 404 Not Found:

There are many many HTTP response status codes. You’ll find a fun representation of them at http.cat. For example:

Mozilla (mzl.la/3o6qWNM) provides more conventional descriptions of status codes:

The HTTP 418 I’m a teapot client error response code indicates that the server refuses to brew coffee because it is, permanently, a teapot. A combined coffee/tea pot that is temporarily out of coffee should instead return 503. This error is a reference to Hyper Text Coffee Pot Control Protocol defined in April Fools’ jokes in 1998 and 2014. Some websites use this response for requests they do not wish to handle, such as automated queries.

Note: The 1998 HTCPCP (bit.ly/3olM42q) April Fools’ joke was inspired by the Trojan Room coffee pot (bit.ly/3pkCDSb), the subject of the world’s first web cam. It was set up in 1991, long before the Internet of Things (IoT).

If an HTTP message has a body, it also has a Content-Type header. Content-Type specifies the internet media type of the data in the HTTP message body.

Usually, you’ll work with three content types for text data, depending on the structure:

• JSON (JavaScript Object Notation) is the most common data format used for HTTP communication by app clients. It’s a structured data format consisting of numbers, strings and arrays and dictionaries that can contain strings, numbers and nested arrays and dictionaries.
• Web forms use form-encoded, which looks like a query string. A query string is a collection of key-value pairs, separated by & and preceded by ?.
• Web pages are HTML.

When working with binary data some of the most used types are: PDF, image formats and multi-part form data, when the client sends any kind of binary file along with text elements.

REST API

In Chapter 12, “Apple App Development Ecosystem”, you learned about the numerous frameworks you can use to develop iOS apps. An Apple framework is one kind of Application Programming Interface (API). It tells you how to use the standard components created by Apple engineers.

Another kind of API is the set of rules for clients to request resources from a server. Most of the APIs you’ll use for your apps are REST APIs, which use HTTP. For each resource available on the server, the REST API documentation tells you how to construct a request:

• The resource’s URL, called its endpoint.
• Which HTTP method to use.
• Which HTTP headers to include.
• What to put in the request body.

Note: REST is the acronym of “REpresentational State Transfer”, the name created by Roy Fielding for the architectural style underlying the World Wide Web. The term describes how a well-designed Web application works: A user selects a resource identifier from a network of Web resources (a virtual state-machine) and uses methods like GET or POST to create a state transition that transfers the resource’s representation to the user.

In the next chapter, you’ll set up RWFreeView to communicate with the REST API api.raywenderlich.com. In this chapter, you’ll explore this API’s documentation at raywenderlich.docs.apiary.io.

Sending and receiving HTTP messages

Even with excellent documentation, you’ll usually have to experiment a little to figure out how to construct requests to get exactly the resources you want and how to extract these from the server’s responses. So how do you send requests and examine responses?

Browser

The easiest way to make a simple HTTP GET request is to enter the URL in a browser app like Safari.

➤ Enter this URL in your favorite browser:

https://www.raywenderlich.com/library

This is the endpoint of the raywenderlich.com library. You get a page similar to this:

This is the body of the server’s response, but you don’t get to see the headers. And, you can’t do much more than a simple GET request.

cURL

A browser is a fully-automated HTTP tool. At the other end of the spectrum is the command-line tool cURL (curl.se) — “the internet transfer backbone for thousands of software applications”.

The documentation for a REST API often provides sample requests to show you how to use it. Very often, these use cURL.

➤ Open Terminal and enter this command:

curl https://api.github.com/zen

You send an HTTP request to GitHub’s API server. The response is a random item from their design philosophies, like “Favor focus over features” or “Avoid administrative distraction”.

There are lots more request examples at GitHub’s Getting started with the REST API (bit.ly/3iD717R).

But, you exclaim, curl doesn’t show any response headers either! Well, like all Unix commands, curl has a wealth of options, including --include and its shortcut -i, to include the HTTP response headers in its output.

➤ Enter this command:

curl -i https://api.github.com/zen

And you see quite a lot more output:

HTTP/2 200 
server: GitHub.com
date: Tue, 27 Apr 2021 01:40:56 GMT
content-type: text/plain;charset=utf-8

...

x-ratelimit-limit: 60
x-ratelimit-remaining: 58
x-ratelimit-reset: 1619491224
x-ratelimit-used: 2
accept-ranges: bytes
x-github-request-id: EF14:706C:A3D763:B0E977:60876BA7

Avoid administrative distraction.

Headers beginning with x- are custom headers set up by the organization. For example, x-ratelimit-limit and x-ratelimit-used indicate how many requests a client can make in a rolling time period (typically an hour) and how many of those requests the client has already made.

The curl --verbose or -v option displays request headers and a lot more.

➤ Enter this command:

curl -v https://api.github.com/zen

Replacing -i with -v produces quite a lot more output — every handshake interaction between the terminal and the server, the encryption algorithms used, the server certificate details, as well as the response headers. The request headers are just the five lines that start with >:

> GET /zen HTTP/2
> Host: api.github.com
> User-Agent: curl/7.64.1
> Accept: */*
> 

Lines that start with < are response headers, and lines that start with * are additional information provided by cURL.

You might not enjoy typing long structured command lines, especially something like this sample cURL command to create a new GitHub repository:

curl -i -H \
  "Authorization: token 5199831f4dd3b79e7c5b7e0ebe75d67aa66e79d4" \
  -d '{ \
      "name": "blog", \
      "auto_init": true, \
      "private": true, \
      "gitignore_template": "nanoc" \
    }' \
  https://api.github.com/user/repos

This POST command sends authorization data in a request header and the request body as data in JSON format. The endpoint doesn’t name a specific user because GitHub knows that from the token value.

Another problem with using cURL: If the response is complex, it’s hard to examine it in the terminal.

➤ Enter this command:

curl https://api.raywenderlich.com/api/contents

This is a request to the API you’ll use for RWFreeView. The response is pretty mind-numbing:

If you concentrate, you might be able to see from this output that the response body is a dictionary where the first value "data" is an array of dictionaries. You can use a tool like codebeautify.org/jsonviewer to format this so it’s easier to read.

But there’s a better solution: apps that make your HTTP messaging easier.

Exploring api.raywenderlich.com

Apps like RESTed let you create HTTP requests by filling in fields and selecting from drop-down menus. You can pretty-print responses and use syntax highlighting.

➤ In a browser, open apple.co/3cb5CnP, click View in Mac App Store and install the app.

Requesting contents

➤ Open RESTed and replace http://localhost:3000/ with this URL:

https://api.raywenderlich.com/api/contents

You set the resource endpoint. How do you know what to ask for? Look through the table of contents sidebar at raywenderlich.docs.apiary.io: In the References section, /contents sounds like the most general, highest level of data.

How do you know what to write in front of /contents? Select /contents then scroll down to find this gray field with 200 OK and a disclosure indicator:

➤ It’s a button: click it!

A sidebar opens, showing the Request. This sidebar has lots of features. You’ll see some of them soon.

➤ Back in RESTed, leave the GET method selected. Open preferences and check the boxes for Pretty-print response and Apply syntax highlighting:

➤ Close preferences and click Send Request.

And here’s the response time, the request headers, response headers with status code 200 OK.

Content-Type is application/vnd.api+json; charset=utf-8. The key information here is json. This tells you how to decode the response body.

Note: UTF-8 string encoding is a version of Unicode that is very efficient for storing regular text, but less so for special symbols or non-Western alphabets. Still, it’s the most popular way to deal with Unicode text today.

➤ Scroll down to view the response body.

Scrolling through the response body, it’s much easier to see the top level dictionary with four keys: data, included, links and meta. The information you need for your app is in the data value, which is an array of dictionaries.

Each dictionary in the array contains the attributes for one content item. bit.ly/3sMFjdy describes these attributes. In the next chapter, you’ll use JSONDecoder to extract the attributes you want and store them in the Episode structure so you can display them in RWFreeView.

Media URLs

➤ Notice that card_artwork_url is a URL. Go ahead and copy-paste one of these URLs in RESTed and send the request.

The response header Content-Type is now image/png and the Server is AmazonS3. (The server for the contents response is nginx.) RESTed is able to display the image.

You’ve already used the uri attribute to open an episode in a browser from RWFreeView.

You’ll use the video_identifier value to fetch the video url to play in PlayerView. For example, the video_identifier of “SwiftUI vs. UIKit” is 3021.

➤ Send this request in RESTed:

https://api.raywenderlich.com/api/videos/3021/stream

The response body contains this url:

"url": "https://player.vimeo.com/external/357115704.m3u8?
s=19d68c614817e0266d6749271e5432675a45c559&oauth2_token_id=897711146"

➤ This is the link you’ll pass to PlayerView. RESTed isn’t able to play it, so paste it into Safari to see it load the video.

This is how the server provides access to additional resources. Images and videos aren’t embedded directly into the search results, but you get a URL that allows you to access each item separately.

Sorting

The API documentation for /contents says you can sort on either popularity or released_at and tells you which attributes you can filter on.

➤ In RESTed, re-send the contents request:

https://api.raywenderlich.com/api/contents

➤ Scan the released_at values for the first few array items, and you’ll see the default sort order is reverse chronological order. The first item was released most recently, and later items were released earlier.

So change the request to show you the most popular items first.

➤ In the parameter section, click + to add Parameter Name sort with Parameter Value -popularity:

You ask for reverse numerical order because higher popularity values indicate more popular content items.

➤ Click Send Request.

Now the first array item has a released_at date in 2013, and its popularity value is 657882! It’s probably higher by the time you send this request.

Filtering

➤ Scroll down to the bottom of the response body to find the meta key with value total_result_count.

2420 items match the filters in the /contents request. It’s probably more by the time you read this. Many of these items aren’t about iOS or Swift. How can you request only iOS and Swift items?

In the API documentation, the You can filter on: list includes domain_ids, which sounds like a possibility for a solution.

➤ In the sidebar of the API web page, click /domains then click its 200 OK button:

The domains request is already done, and you’ll look at the response soon. But first…

➤ Click Select Language… (click the text; the disclosure arrow on this button doesn’t do anything):

This tool generates code for this HTTP request in several programming languages, including cURL and Swift!

➤ Select Swift:

You’ll write something similar to this in the next chapter. This tool is useful either as a starting point for your code or to check you haven’t forgotten anything.

Below the Code Example section is Response.

➤ Keep scrolling to view the Body:

It’s pretty-printed and a little more colorful than RESTed. And, no surprise, “iOS & Swift” has id value 1. This is the parameter value you need to create your filter endpoint.

Note: The response to the /contents request already contains this information in the value of the included key.

➤ Close the /domains sidebar and go back to the /contents documentation to see how to create the filter endpoint.

Here’s the example:

➤ So, in RESTed, add a parameter with name filter[domain_ids][] and value 1:

➤ Add two more parameters:

• Name: filter[content_types][]
• Value: episode

And

• Name: filter[subscription_types][]
• Value: free

➤ Click Send Request.

Now the first item is “SwiftUI vs. UIKit” with popularity value 43193.

With these parameters set up in RESTed, you can easily turn any of them off by unchecking its checkbox. Or, you can change a parameter value to retrieve a different domain or content type.

Apiary console

The sidebar has a feature that looks wonderful but doesn’t quite work.

➤ Open the /contents sidebar and click Try console to get a form where you can add the same parameters:

But when you Send request, the response is We encountered an error. Please try later.

Changing Production to Debugging Proxy works, but doesn’t pretty-print the response body. Using Mock Server works and pretty-prints, but contains no results after July 24, 2018. Also, you can’t turn off just some parameters: Reset values deletes all parameters.

URL-encoding

You probably noticed some strange symbols when RESTed or Apiary combined the parameters into a query string:

https://api.raywenderlich.com/api/contents?
  filter%5Bsubscription_types%5D%5B%5D=free&
  filter%5Bdomain_ids%5D%5B%5D=1&
  filter%5Bcontent_types%5D%5B%5D=episode&
  sort=-popularity

Note: I indented these lines for readability. This string won’t work as a URL.

RESTed and Apiary URL-encoded the square brackets you used in the parameter names to %5B and %5D. URLs sent over the internet can contain only letters, digits and these punctuation marks: -, _, . and ~.

Other punctuation marks, including /, ? and %, are encoded as a pair of hexadecimal digits preceded by the escape character %. The hexadecimal value is the character’s byte value in ASCII, for example, 20 (32 in decimal) for the space character and 25 (37 in decimal) for the % character. The space character can also be encoded as +. For a non-ASCII character, URL-encoding uses its UTF-8 byte value.

When / and ? are delimiters in the URL, they don’t get encoded.

Challenges

Challenge 1: Change the page size

➤ Scroll down to the bottom of the response body to find the links item.

The links value is a dictionary of five keys: self, first, prev, next and last. The values are query URLs. The current response is both self and first. The prev key has no value because there’s no previous page.

Paging is controlled by two query terms that you didn’t set. For example, the next URL includes these parameters:

page%5Bnumber%5D=2&page%5Bsize%5D=20

URL-decoded, this is page[number]=2&page[size]=20. So the next page has 20 items, as do the self and first pages. The last page has page[number]=22 and page[size]=20, but actually has only 9 items because total_result_count is 429.

One of the features you’ll implement in the next chapter is letting the user change the page size. You’re probably pretty sure you’ll need to set the page[size] parameter.

Your challenge is to send a RESTed request that changes the page size to 5 (it’ll be easier to count a small number of response items).

Note: This is all documented in the Apiary’s Introduction ▸ Pagination section.

Here’s my RESTed request and response:

The final folder in the project materials has saved RESTed requests for all the requests in this section.

You’re now well-prepared to implement HTTP messages in RWFreeView.

Challenge 2: POST request and authentication

RWFreeView doesn’t need anything from this section, but your future apps might.

RWFreeView only needs to GET resources from the server and, because it gets only free items, your users don’t need to authenticate.

You usually need to implement authentication for apps that let users access restricted materials or create, update or delete server records. Consider using Sign In with Apple. Follow our tutorial Sign in with Apple Using SwiftUI bit.ly/3iHqNix.

To try out a POST request, you’ll use RESTed to send something like this GitHub curl example:

curl -i -H \
  "Authorization: token 5199831f4dd3b79e7c5b7e0ebe75d67aa66e79d4" \
  -d '{ \
      "name": "blog", \
      "auto_init": true, \
      "private": true, \
      "gitignore_template": "nanoc" \
    }' \
  https://api.github.com/user/repos

This example shows how to create a new GitHub repository, so it requires GitHub-user authentication. Remember when you set up your GitHub account in Xcode, you had to generate a personal access token? You’ll need one here, too.

➤ If you haven’t saved a plain-text copy of your GitHub personal access token, generate a new one at bit.ly/2Y71Ofh.

➤ In RESTed, add Header Field Authorization with Header Value token. Paste your personal access token after “token ” to complete this value:

➤ Set the endpoint to https://api.github.com/user/repos, the method to POST, then select Form-encoded and set parameters name api-test-repo, auto_init true, private false:

Note: If you don’t see both the parameter fields and the HTTP body, make sure both buttons are active.

Instead of appending to the endpoint, POST request parameters appear in the body field.

➤ Click Send Request.

This was a deliberate “oops” to show you what happens if the server expects the POST request body to be in JSON but you send form-encoded data instead. Form-encoded data looks like the query string part of a URL because that’s how a web form sends it.

➤ Change Form-encoded to JSON-encoded, then click Send Request.

That worked! Check your GitHub account to see there really is a new repository named api-test-repo:

➤ Click Send Request again.

If you try to create the same repo again, the server returns the error message “name already exists on this account” along with a documentation_url for this endpoint.

Key points

• Client apps send HTTP requests to servers, which send back responses.
• An HTTP response contains a status code and some content. Text content is usually in JSON format and may contain URIs the client app can use to access media resources.
• HTTP requests follow the rules of the server’s REST API, whose documentation specifies resource endpoints, HTTP methods and headers, and how to construct POST and PUT request bodies.
• You can send simple GET requests in a browser app. Use cURL or an app like RESTed to create and send requests and inspect responses.
• The documentation for api.raywenderlich.com includes a sidebar where you can create an HTTP request then generate Swift code for your app or send the request to a Mock Server or Debugging Proxy server.