Using AI for CI & Automation in iOS Projects

Nov 11 2026 · Swift 5, iOS 27, Xcode 27

Lesson 02: Comparative Evaluation of AI Review Tools in CI

Demo

Episode complete

Play next episode

Next

Written by the author, AI-editedAI-generated, author-verified How this module was made

Transcript

In the previous lesson we added a GitHub Copilot-based agentic review that runs automatically on every pull request. Today we extend that foundation by placing the identical change set in front of two additional agents—Claude Code and OpenAI Codex—so that we can compare quality, latency and token count under controlled conditions. We begin from the same sample iOS repository. The Starter folder of this lesson is again the root of the repository and has been committed as the main branch so that GitHub Actions can discover the new workflow files. It already contains the Lesson 1 Copilot workflow, CODE_REVIEW.md, CI_GUIDANCE.md, and the shared helpers under resources/.github/scripts/. Token accounting for all three agents is kept consistent through those helpers—we will call them from the workflows rather than pasting their bodies into the YAML. The scripts stay unchanged; each new workflow reuses the same CI_GUIDANCE parameter.

Claude Code and Codex are new tools for this program; they are not the Claude-in-Xcode or Cursor setups from earlier modules. Before adding workflows, confirm that an Anthropic credential and an OpenAI API key are stored as repository secrets. Follow the official Claude Code Action and Codex Action documentation if you still need to create those accounts. We will not walk through signup on camera.

I create a new workflow file named claude-code-review.yml. The trigger, concurrency group, CI_GUIDANCE parameter and permission set deliberately mirror the Copilot workflow so that the only material difference is the agent itself. We listen for pull_request events of types opened, synchronize and reopened, cancel any in-progress run for the same pull-request number, and request contents read, pull-requests write, and id-token write for the Claude action’s default authentication path.

name: Claude Code Review
on:
  pull_request:
env:
  CI_GUIDANCE: CI_GUIDANCE.md
jobs:
  claude-review:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - name: Checkout repository
        uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - name: Build review prompt
        id: prompt
        run: |
          {
            echo 'text<<PROMPT_EOF'
            ./.github/scripts/build-review-prompt.sh claude \
              "${{ github.base_ref }}" \
              "${{ github.repository }}" \
              "${{ github.event.pull_request.number }}" \
              --guidance "${{ env.CI_GUIDANCE }}"
            echo 'PROMPT_EOF'
          } >> "$GITHUB_OUTPUT"
      - name: Run Claude Code review
        id: claude
        uses: anthropics/claude-code-action@v1
        with:
          anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
          # Use the workflow GITHUB_TOKEN so we do not require the Claude Code
          # GitHub App (OIDC app-token exchange). Comments are posted below
          # with the same token.
          github_token: ${{ secrets.GITHUB_TOKEN }}
          prompt: ${{ steps.prompt.outputs.text }}
          claude_args: |
            --model claude-sonnet-5
            --max-turns 20
            --allowedTools "Read,Glob,Grep,Bash(git *)"
      - name: Print token usage
        if: always() && steps.claude.outputs.execution_file != ''
        run: ./.github/scripts/claude-token-usage.sh "${{ steps.claude.outputs.execution_file }}"
      - name: Extract review text
        if: success() && steps.claude.outputs.execution_file != ''
        run: |
          jq -r '
            [.[] | select(.type == "result" and .subtype == "success")]
            | last
            | .result // empty
          ' "${{ steps.claude.outputs.execution_file }}" > claude-review.md
          if [ ! -s claude-review.md ]; then
            echo "No review text found in Claude execution result." >&2
          fi
      - name: Post review comment
        if: success()
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
        run: ./.github/scripts/post-review-comment.sh claude-review.md "Claude Code Review"

The action receives the Anthropic API key from a repository secret. The prompt is built by .github/scripts/build-review-prompt.sh with the same CI_GUIDANCE parameter the Copilot workflow already uses, so Claude receives CODE_REVIEW.md plus the one-line CI_GUIDANCE.md instruction. The prompt includes the repository and pull-request identifiers and asks Claude to inspect the base-to-head git diff. A modest max-turns limit keeps the job predictable; allowed tools are limited to reading the tree and running git so the agent can examine the change set without unrelated privileges.

After the Claude step, three follow-ups keep the job comparable to the other agents. First we always print token usage by calling .github/scripts/claude-token-usage.sh with the action’s execution_file output the script reads the Claude session result and writes the same Step Summary table shape used by the Copilot review. Next we extract the final success-result text into claude-review.md. Finally .github/scripts/post-review-comment.sh posts that markdown as a pull-request comment under a Claude heading, but only when the body is long enough to be a real review.

Next I add a parallel workflow for Codex. Again the event trigger, concurrency group, least-privilege permissions and the CI_GUIDANCE parameter are identical. The OpenAI API key is injected from a secret. I pin codex-home under the runner temp directory so a later step can locate the session rollout for token accounting. build-review-prompt.sh writes the same shared prompt—CI_GUIDANCE.md plus CODE_REVIEW.md to review-prompt.txt; the action writes the final message to codex-review.md.

name: Codex Review
on:
  pull_request:
env:
  CI_GUIDANCE: CI_GUIDANCE.md
jobs:
  codex-review:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - name: Checkout repository
        uses: actions/checkout@v7
        with:
          fetch-depth: 0
      - name: Build review prompt
        run: ./.github/scripts/build-review-prompt.sh codex "${{ github.base_ref }}" --guidance "${{ env.CI_GUIDANCE }}" > review-prompt.txt
      - name: Run Codex review
        id: codex
        uses: openai/codex-action@v1
        with:
          openai-api-key: ${{ secrets.OPENAI_API_KEY }}
          # Pin home so the token-usage step can find session rollouts reliably.
          codex-home: ${{ runner.temp }}/codex-home
          prompt-file: review-prompt.txt
          output-file: codex-review.md
      - name: Print token usage
        if: always()
        run: ./.github/scripts/codex-token-usage.sh "${{ runner.temp }}/codex-home"
      - name: Post review comment
        if: success()
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
        run: ./.github/scripts/post-review-comment.sh codex-review.md "Codex Review"

After Codex finishes we call .github/scripts/codex-token-usage.sh against that pinned home directory so token usage appears in the job log and the GitHub Step Summary in the same format as Copilot and Claude. The post-comment step then publishes codex-review.md under a Codex heading when the body is substantial enough. If your organization has already enabled the managed Codex code-review feature, you may prefer that path for simplicity; the action-based approach shown here gives explicit control over the prompt and the output format, which is useful for side-by-side comparison. In either case the architectural pattern remains the same: trigger, authenticate, assemble limited context, publish structured results, and record comparable usage metrics.

The Lesson 1 Copilot workflow already posts its review as a pull-request comment and records tokens through .github/scripts/ai-code-review.sh, passing the same --guidance "${{ env.CI_GUIDANCE }}" argument. The Claude and Codex helpers share the same Step Summary layout via .github/scripts/token-usage-summary.sh, so comparing token count across the three jobs is a matter of reading three identically shaped tables rather than reconciling different log formats.

I would like to remind you that the expected token range for a typical review of this codebase is roughly 50 000 to 100 000 tokens depending on the size of the diff. This figure will be refined once we observe a real run.

Workflow files must live on the default branch before GitHub Actions will execute them for pull requests. I will now push the two new workflow files. The supporting scripts, CODE_REVIEW.md, and CI_GUIDANCE.md are already on the default branch from Lesson 1, so they do not need to change. I then verify in the Actions tab that Claude Code Review and Codex Review appear alongside the existing Copilot workflow.

I now open, or force a synchronise on, the same focused pull request that was examined in the previous lesson. Because the three workflows share identical triggers, all three jobs start.

Now we wait for completion. While the jobs run I note the wall-clock start times so that latency can be compared later. Typical ranges observed on this sample are two to four minutes for the lighter configurations and up to eight or ten minutes when an agent performs multiple tool-using turns.

All three agents have now posted comments—Copilot, Claude under its Claude Code Review heading, and Codex under its Codex heading. We will now examine them side by side, beginning with correctness and concurrency findings. Observe which issues appear in every review, which appear in only one or two, and which are unique to a single agent. Particular attention is paid to MainActor isolation, potential retain cycles in closures, and SwiftUI state-ownership annotations—domains that matter for iOS correctness.

Inline comments that reference concrete lines are far more actionable than free-form summaries. We note the proportion of each agent’s output that is attached to specific locations versus the proportion that remains high-level. We also count how many comments must later be classified as false positives or pure style nits that fall outside the CODE_REVIEW.md priorities.

Opening the workflow logs and each job’s Summary tab we extract the approximate token counts reported by each integration. Because Claude and Codex call their token scripts after the agent step—and Copilot already does the same inside .github/scripts/ai-code-review.sh—we see a uniform “AI Code Review – Token Usage” table with token types, a total, and a call count. In this particular run the figures fall inside the ranges we recorded earlier, yet the relative ranking is informative: one agent may consume noticeably more tokens while producing a longer but not necessarily higher-signal comment set. Latency differences are likewise visible in the job duration columns.

A final glance at the two repository guidance files confirms that all three agents received the same prioritized list of concerns. Each workflow passes CI_GUIDANCE.md into the shared prompt builder, so the instructions stay identical. Differences in output therefore stem primarily from model behaviour and the internal context-assembly strategies of each integration rather than from divergent prompts or edited scripts.

This completes the comparative exercise. We have configured two additional agents using the same architectural pillars, subjected a realistic iOS change set to three independent reviews, measured latency and token count with a shared summary shape, and practised a disciplined triage policy. The essential insight is that no single agent is universally superior; the value of the pipeline lies in the combination of consistent first-pass analysis and human judgement that decides which findings deserve action. In subsequent work you can repeat the same side-by-side measurement whenever a new model version appears.

See forum comments
Cinema mode Download course materials from Github
Previous: Comparing Review Agents Next: Conclusion