Testing

If you remember back to the proposed AI-assisted development process from the previous lesson, you might notice something missing:

1. Develop, and refine a spec with a chat-based LLM.
2. Review and approve the spec, checking high-risk areas carefully.
3. Ask an AI agent to write an implementation plan.
4. Review the plan, if necessary refine it, or cut it down.
5. Ask the AI agent to implement the spec according to the agreed plan.
6. Debug generated code.
7. Perform a code review, and iterate as required.

Nowhere in this plan does it mention writing any tests.

The Importance of Tests in AI-assisted Coding

Automated tests in software serve several purposes:

• Ensure that the application code behaves as expected
• Exercise edge cases
• Protect against regressions as the code evolves

All three of these become even more important when you’re asking AI to help you write the code. Your AI Agent can generate vast swathes of code, and it’s challenging to code review every line. Having some tests in place can help ensure the veracity in the same way it does with human-written code.

Testing Strategies

There are several different approaches to writing tests in an AI-assisted world:

• You manually write all the tests. And then get AI to write the code to satisfy them, and the software spec. This is a fairly dull option—if we’re writing any code it should be the fun code, not the test code!
• Use AI to assist with writing test code. AI is really good at taking test descriptions (i.e. the test name) and writing the code for it. It is also good at seeing a pattern in tests and help you complete it. Obviously the generated code needs review, but this is on quite a nice test-by-test scope.
• Let AI write the tests, you write the app code. This is the inverse of the first strategy. In order for this to work, the spec has to be really detailed, and you need to review the test code.
• Let AI write the test suite and the app code. This might seem a little foolish, but provided the code review approach is in place, then this can be quite a reasonable way to build up a test suite. It is strongly recommended to also think about how you would test a particular module, and add extra tests if required.

In the next video we’ll ask Claude Code to write a test suite for Yesterday’s Weather, and then iterate on it until all the tests pass, and we’re happy that we’re exercising the app code appropriately.