Using AI for Testing in iOS

Welcome to the first lesson of Module 5. In this lesson, you’ll cover testing with AI agents.

The previous module implemented a new feature in the first lesson, and the later lessons ran audits on the feature that was already implemented. Some of them were revisions against the plan itself, and others were based on the specialized skills Apple introduced in Xcode 27.

In this lesson, we’ll take those audit files, consolidate them, and create a new plan that identifies or defines valuable, meaningful tests to be implemented in this lesson.

You don’t want to tell an AI agent or even a human developer to just implement tests or just an open-ended goal of increasing coverage. This can be the intention, but it still needs to be refined to create meaningful tests, not something that is written now out of haste that gets in your way later on. Or to make it worse, a test that ends up being meaningless with low chance for it to actually fail. I promise you, such a thing exists. Non-failable tests that gives you a fake sense of confidence.

The files Acceptance Corner Cases Audit, Goals and Non-goals Audit, Dynamic Type Audit, Swift Surface Audit and VoiceOver Audit are all audits against a feature implemented by reviewed feature plan, execution plan. All those files are on the same folder as the project itself.

The plan intentionally skipped the additions of the test. I want you to consolidate the audit files and create a new plan for the new tests to be implemented. Ensure the tests are meaningful and the plan guarantees fixes for all the outcomes of the audit, and create valuable tests that ensure they wouldn’t break by accident in the future with future changes. The plan needs to include some details on each test that will be introduced. Name of the test, function or functions it will cover, description on what the test will cover, assertions, and what data the test will check to fail or succeed on.

The files, "Acceptance-CornerCases-Audit.md", "Goals-NonGoals-Audit.md", "Dynamic-Type-Audit.md", "SwiftSurface-Audit.md", "VoiceOver-Audit.md" are all audits against a feature implemented by "reviewed-feature-plan.md" and "execution-plan.md".  All those files are on the same folder as the project file itself.
The plan intentionally skipped the addition of the tests. I want you to consolidate the audit files and create a new plan for the new tests to be implemented. Ensure that the tests are meaningful and the plan guarantees fixes for all the outcomes of the audit and creates valuable tests that ensures they wouldn't break by accident in the future with future changes. This plan needs to include some details on each test that will be introduced:
- Name of test
- Function(s) it will cover
- Description on what the test will cover
- Assertions and what data the test will check to fail or succeed on.

The idea is to have a plan that includes as much details as possible, so when I review it, I can actually understand the value or have a proper insight on what is the actual intention of this test, so I can challenge myself, as a human, the value that this test will introduce. The plan I asked for is created. Let’s start going through it.

Let’s open the file properly within Xcode. The test plan is quite verbose, as I asked. Going through it, it’s broken down into multiple tasks, similar to how the original implementation was. It’s pointing out there are two tasks that are actually optional. And as for the details, it’s specifying for each file what are the tests or the test name and some information as requested.

Ideally, I should go through those one by one and make sure that they’re actually meaningful. Overall, I went through those tests, and I think they are meaningful. It’s possible that I missed a few that I can actually skip, or that test is not as meaningful as I think it is. However, in my defense, I implement it, even myself as a human, not as an AI model. I can use this information inside the text as an explanation of my intentions or what I wanted to cover. If anything happens and if there are any sort of failures, that information can be very, very valuable.

Going through the test deeper, I noticed that there are UI tests that are intended to be covered. I want to focus only for the implementation of unit tests within this lesson. Next lesson, we’ll cover AI-assisted UI tests.

Going back to the conversation, implement the plan using workflows, skip the optional points, and don’t implement UI tests now. Only focus on unit tests.

Implement the plan using workflows. Skip the optional points and don't implement UI Tests now, only focus on Unit Tests.

This probably took quite a while for you. In my case, using workflows, the sub-agents actually got stuck. So in the middle of the conversation, I asked the main agent to take over control and stop using sub-agents. This can happen, so it’s always good to be paying attention to how the conversation is happening and follow up on what the sub-agents are doing if you feel things are taking longer than they should be. The agent finished the implementation of all the tests. It iterated on build, test, build, until all the tests were passing.

But I want to return back. The main intention is not just to write the tests. I want to make sure that the issues discovered by the audits are actually fixed. So I’m going to do one last step. Ask the agent to confirm that the findings from the audit are actually fixed at the end. Revise again that the issues found in the audits are fixed.

Revise again that the issues found in the audits are fixed.

The AI agent finished. It actually documented what is done and fixed within the audit files, what isn’t fixed, and what is out of scope. Let’s have a look at one of the files. Let’s open it properly. Here, those are the original findings. It’s pointing out the status now that it’s fixed. Some are not. VoiceOver is fixed. Another one that isn’t. And it has a justification on why it is not. This documentation is very important.

You can go through all the different audit files, have another iteration of improving the test quality implemented in the app, or another iteration of fixes on the app. It’s not always necessary that a single pass is enough. As you’ve seen, when the original plan was implemented, it took different sessions and different iterations to improve what the plan or how the feature should look like. And when we started this implementation, we even iterated on top of it again with some execution details.

This is just a preview on what you can do or how you can organize the work for testing in specific. In the next lesson, we’ll use AI to take control on the app to test it directly on the simulator. So the test case itself is ran by the AI and is not written in code. See you in the next lesson.