Write every failure to a file and let fixer agents work through it by area
Failing tests and compiler errors are saved to a file, grouped by package, and each group goes to an implementer, reviewers and a fixer.
How Anthropic used this technique
Read what Anthropic changed and follow the original sources.
Write every failure to a file and let fixer agents work through it by area
Translated to GitHub Actions
- What Anthropic built
- Bun's Rust port wrote about 16,000 compiler errors to a file, grouped by crate, and a workflow divided them among 64 Claudes.
- Your GitHub Actions version
- A workflow step that writes every failure to a file, grouped by area, for fixer agents to work through.
Once Bun's Rust port was written, cargo check wrote about 16,000 compiler errors to a file, grouped by crate, and a workflow divided them among 64 Claudes: 16 loops across 4 worktrees, each with one Claude fixing, two reviewing and one applying. A second workflow looped on test files: each failing test saved its stack trace and errors to a file, one implementer proposed a fix, two adversarial reviewers checked it and a fixer applied it. The test runner gained a --results-json option that writes per-file results as JSON for those test-fix workflows.
Two days after the first CI run, the failing list was down from 972 test files to 23. Source
- Use it when
- A change leaves hundreds of failing tests or errors, or an agent fixes failures from a log too large to read at once.
- Before applying it
- Run the failing check once at the start of each group and commit at the end, so parallel agents do not start overlapping builds, and have a person read the loop's output and edit its instructions when it drifts.
- GitHub Actions adaptation
- Compare the source system with your own workflow before applying this translation.
Company Playbooks
- Read Anthropic’s playbookRun only affected tests
- Read DoorDash’s playbookRun build and test in parallel, sharded across CI agents
- Read Linear’s playbookBenchmark the same job on faster hardware
- Read Mastra’s playbookStarSling runners for the slowest test workflow
- Read OpenAI’s playbookRank CI fixes by the minutes they save
- Read Ramp’s playbookMove a CPU-bound test suite to a faster CPU
- Read Shopify’s playbookLet an agent iterate against one CI metric
- Read Spotify’s playbookBuild arm64 on native arm64 runners
- Read Stripe’s playbookRun only the tests a change can reach
- Read Uber’s playbookBuild the merge-queue speculations most likely to be needed