Monitoring HTTP Traffic with Instruments

Learn to monitor and analyze HTTP traffic using Instruments Network profiling in your iOS SwiftUI apps. By Vijay Subrahmanian.

Login to leave a rating/review
Download materials
Save for later
Share

Learn to monitor and analyze HTTP traffic using Instruments Network profiling in your iOS SwiftUI apps.

Xcode comes bundled with Instruments, a tool that helps measure app metrics like memory usage, network activity and time profile. Using Instruments to profile your app provides valuable insights into app behavior and performance. Instruments works with all Apple platforms and allows live profiling and inspection while testing the app on a device!

In this tutorial, you’ll learn:

  • How to use the HTTP Traffic Instrument in Xcode.
  • About tasks and transactions in the instrument, and profiling networking activities.
  • How to archive and share HTTP traffic traces with your fellow web developers.
Note: You’ll need an Apple Developer account to run the app on a physical device.

Getting Started

Use the Download Materials button at the top or bottom of this tutorial to download the starter project.

The starter project is a tab-based app called Shuffle built using Swift UI. Shuffle has three tabs:

  • Today: Shows the Quote of the Day along with a Shuffle button.
  • Explore: A list of quotes the user can select, view and bookmark.
  • Profile: Lets the user login and displays user activity on login.

Open the starter project. Then build and run it on your device. Navigate around the app to familiarize yourself with the UI.

App Screen Today and Explore Tabs
App Screen Profile Tab

You’ll use the APIs provided by https://favqs.com/ and https://picsum.photos/ for this tutorial. You can use these test credentials to login to the app or signup for an account at https://favqs.com/login.

Username: Shuffleuser
Password: Shuffle123$

Explore the project. Pay particular attention to DataProvider.swift. This file is the networking helper class that manages all the API requests and responses in the app. Check out the data models, views and view models to understand the implementation.

Try using the app, and you’ll notice a few issues that need addressing. Instruments can help to identify the underlying issues. The instrument you’ll use here is the HTTP Traffic Instrument.

What is the HTTP Traffic Instrument?

In its Network profiling template, Instruments 13 introduces a new HTTP Traffic Instrument which lets you track and inspect all the HTTP traffic flowing in the app.

HTTP Traffic not only helps you see what’s happening with the request/response you receive while using a web service API, it also helps you see what’s happening with the connection. In this way, it’s unlike other proxy web debugging tools available, such as Charles Proxy or Proxyman.

HTTP Traffic also enables network debugging without needing to install any certificates on the device. It even works when SSL pinning is enforced or when using a VPN, all while showing the entire requests and responses in a pretty format for HTTP/s connections.

While it’s a great tool, it has limitations. Currently, the HTTP Traffic Instrument doesn’t support simulator, mocking requests or responses, injecting or modifying payloads.

Now that you have a better understanding of the tool, next you’ll use it to inspect the app.

Inspecting the App

Before testing the app on a device, change the bundle identifier if necessary and set the provisioning profile. Follow the steps below to profile using Instruments:

  1. Connect your iOS device to your Mac.
  2. Open the starter project and set the run target as your device.
  3. Choose Product ▸ Profile from the menu or press Command-I.

    Select Product Profile

  4. Choose Network when the profiling template selection dialog appears.

    Select Network

  5. To begin recording, press Record in the toolbar or press Command-R.

    Tap Choose

  6. While the recording is in progress, use the app and visit different screens.
  7. To stop the recording, press Record again.

You can remove the Network Connections Instrument as it’s unnecessary for this tutorial.

Start recording

Note: You can also profile the app if it’s already installed on the device by opening Instruments and choosing your iOS device and app from the target device and process list.

Now you can inspect and examine the data collected while you use the app. The data includes details related to each of the networking tasks and transactions that occurred during the session.

A typical inspection session looks like this:

Instrument Profiling Sample

HTTP Traffic Instrument shows all the HTTP activity in a timeline of Tracks. Each track is specific to a URL Session.

A. Selected Instrument: HTTP Traffic
B. Process: Application
C. Session: URLSession — Shared
D. Domain: URLSession Task — HTTP Request domain

Now, observe the filters:

Filters

  • Filter 1: Lets you choose whether to display tasks or transactions in the track. You’ll learn about these later in this tutorial.
  • Filter 2: Has options to choose a list of URL Session Tasks or HTTP Transactions or a summary of transaction durations.
Note: You can also save the session to reference later by clicking File ▸ Save. The recorded session will save as a .trace file.

Next, you’ll learn how HTTP Traffic Instrument actually works.

How the HTTP Traffic Instrument Works

For communicating with web services, you use higher-level networking APIs such as URLSession. But, Instruments relies on the lower-level networking stack of Apple’s core frameworks. Thus, it works on all Apple devices and all HTTP traffic passing through the app when using any networking API.

Task and Transaction

A Task is akin to a URLSessionDataTask or URLSessionDownloadTask. It begins when you call resume and calls the respective completion closures before it ends.

A Transaction refers to a single instance of communication with a web service. It includes:

  • Establishing the connection.
  • Performing any cache lookup.
  • Spinning up a thread.
  • Sending an HTTP request.
  • Blocking the thread while waiting.
  • And receiving the incoming response.

In other words, and in its simplest terms, a transaction starts when you call resume() on a task, and it ends when the associated completion block is called.

Transaction Depiction

However, in some instances, a task may have more than one transaction, for example when there are redirects or URL forwarding.

Task Depiction

Now that you know about all the information the tool provides, it’s time to profile the app and see how it fares!

Profiling and Inspecting the App

First, profile the starter app as a new session and start recording. In the Today tab, tap Shuffle and wait for a moment. Then stop recording the session.

Now that you’ve gathered the data, observe the timeline and tasks. The Instrument timeline will look something like this:

Inspection Step 1

Note: Use Command-plus or Command-minus to zoom in or zoom out in the timeline.

As seen in the image above, there are two tasks in the Today tab.

Check the task to get the quote of the day. It calls the favqs.com web service API.

Inspection Step 2

The detail area and tooltip show all the important information related to this task:

  • Endpoint and Query, if any: api/qotd
  • HTTP Version: 1
  • HTTP Method: GET
  • Time taken/Duration/em>: example (100 ms)
  • Response — Status code: 200
  • Response mime type: application/json

As you can see, this task was successful and marked in green. Perfect, isn’t it? :]

Now, under Shuffle, choose the Track Display as HTTP Transactions. Then, choose the filter option in the detail area and select List: HTTP Transactions to view the transaction’s details.

Filter Selection

You’ll see the request/response details in the inspection area for the selected transaction.

Inspection Pane Details Request/Response

There’s one problem, though: The call to picsum.photos shows as canceled and is marked in grey. Not so good, is it? :[

To find out what went wrong, choose List: URLSession Task as the filter in the detail area. Then scroll right to see more information:

  • Error domain
  • Error code
  • Localized error description

Select the particular task in the detail list to inspect it. The inspection detail to the right shows the backtrace.

Inspection Step 3

Note: If the function name isn’t displayed in the inspector, there could be a problem with symbolication. In such cases, right-click the hexadecimal address and choose Locate/Load dSYM.

Open getRandomPicture(completion:) in DataProvider.swift. Now you’ll see where the task starts, which helps you quickly check for possible problems.

Inspection Step 4

Fortunately, in this case, you’ll see a .cancel sent to the task dataTaskFetchImage in getQOTD(completion:). It must be a typo that the developer inadvertently added that canceled this task when trying to get the quote of the day!

Well, how interesting is a quote without a nice picture in the background?! :] You’ll fix the missing picture in the next section.

Applying the Code Fix

In DataProvider.swift, find getQOTD(completion:). The intention here is to cancel any ongoing task to fetch a quote before a new one triggers.

Replace dataTaskFetchImage?.cancel with:

dataTaskFetchQuote?.cancel()

To identify the download task, add the taskDescription for the tasks.

At the end of getQOTD(completion:), add the following line before calling dataTaskFetchQuote?.resume():

dataTaskFetchQuote?.taskDescription = "QuoteOfTheDayDownloadTask"

Likewise, add this line to getRandomPicture(completion:) before calling dataTaskFetchImage?.resume:

dataTaskFetchImage?.taskDescription = "RandomImageDownloadTask"

The system doesn’t interpret taskDescription. You can use this value for whatever purpose you see fit. Here, it’ll help you better identify this task in Instruments.

View taskDescription value in Instrument for debugging purposes

Build and run. Now you’ll see the background picture loads successfully. Yay!

Quote loaded with background

Tap Shuffle and observe.

Shuffle action not working

Notice that the background image keeps shuffling, but the quote remains the same.

But Why? Meme

Think back to the previous session where you inspected the favqs.com API task.

Inspection Step 2

Notice anything?

Hint: In HTTP Traffic Instrument > Shuffle > Shared Session, make sure the favqs.com API track is selected. From the filter option in the detail area, select List: HTTP Transactions.

Cache Lookup

Notice Cache Lookup! This tells you the API call never went to the web service as there was already a cached response available. The cached data was immediately returned in just a few milliseconds and hence it always showed the same quote. The culprit is this line in getQOTD(completion:):

let urlRequest = URLRequest(url: url, cachePolicy: .returnCacheDataElseLoad)

To fix the issue, replace the line above with:

let urlRequest = URLRequest(
  url: url,
  cachePolicy: .reloadIgnoringLocalAndRemoteCacheData)

By setting .reloadIgnoringLocalAndRemoteCacheData as the cache policy, you ask the URLRequest to always load the data by connecting to the remote web service API.

Build and run. Aha!

Shuffle action working

Profile the app again and record the session for the Today tab. Now you’ll see a non-cached response and the task description.

Shuffle fix inspection

Yay!

Success Kid Meme

The task description RandomImageDownloadTask has a URL redirection before it receives the image. Thus, it has two transactions in one task! Check the detail and inspector areas for more details on the responses from both the transactions.

HTTP Redirection

You can also right-click the task and choose Set Inspection Range and Zoom to zoom on the selected task in the track view.

Tip to zoom to selected task

Next, you’ll inspect a secure API along with its associated cookies in HTTP Track Instrument.

Inspecting an Authentication API

Profile the app and start recording. Then, select the Profile tab and log in. If you didn’t create your own username and password, you can use these:

Username: Shuffleuser
Password: Shuffle123$

Wait for the activity list to load.

User Activity List

Then, stop the recording and inspect as below. Use the HTTP Transaction filter mode to see the request/response contents in the inspector area.

Authentication Task Inspection

When you take a closer look at the track, you’ll see the details such as HTTP Method, API Endpoint and Response Code.

Authentication Task Inspection Closer look

The track also contains symbols that show the HTTP Version used, Authentication header, and if Cookies are being sent/received.

If you don’t see any activity for your user, navigate to the Explore screen in the App. Select a quote and choose bookmark after login.

Warning for exposed data

Note: A warning: As you see, the HTTP Traffic Instrument exposes all the sensitive information sent as part of the request/response. Take proper care while using this to avoid any misuse.

Generating an HTTP Archive

In this tutorial, the issues were at the app level. In real-world projects, while working with web service API integration, often there are problems in the API responses. The HTTP Traffic Instrument lets you export the trace as a HAR file to share the details of your investigation with the web service developers. HAR is a JSON-formatted archive file format for logging an app’s interaction with a web API.

Before exporting, save the recording by selecting File > Save from the Instruments menu.

Once saved, open Terminal. Go to the directory where you saved the .trace file and run the command below.

xcrun xctrace export --input <MySavedSession.trace> --har

This code saves the .har file in the current directory, which you can easily share since it doesn’t require Xcode Instruments to open.

Note: You can use Google Admin Toolbox – HAR Analyzer to view the HAR file. You’ll find a sample HAR file, Shuffle_Sample.har, in the downloadable materials.

HAR Analyser

That’s all for this tutorial.

References

Here are a few helpful resources and tutorials to learn more in this area:

WWDC Video 2021 has an excellent introduction to this topic.:

Analyze HTTP traffic in Instruments

This tutorial uses these APIs:

Other Xcode Instrument Tutorials:

Refer to these tutorials to learn more on proxy debugging tools:

Other References:

Where to Go From Here?

Download the completed project files by clicking Download Materials at the top or bottom of the tutorial.

In this tutorial, you learned how the HTTP Traffic Instrument works and how you can benefit from it. Try updating the project by adding additional APIs and exploring various options/filters available in the Instruments. Study the information obtained from profiling the app.

We hope you enjoyed this tutorial. If you have any questions, comments or want to show off what you did to improve this project, please join the forum discussion below!