- All playbooks
- Merge without re-testing against the latest main, except migrations
Merge without re-testing against the latest main, except migrations
Drop the up-to-date-with-main merge rule, test the main branch before every deploy, and make database migrations conflict in Git so only they must update.
How Ramp used this technique
Read what Ramp changed and follow the original sources.
Merge without re-testing against the latest main, except migrations
Every merge made the other open pull requests out of date with the main branch, so each one re-ran the 12-minute suite before it could merge, and with five in the queue the average pull request waited over an hour. Ramp removed the rule that pull requests must be up to date with the main branch and required the tests to pass on the main branch before any deployment. Two branches that each add a database migration from the same revision pass on their own but leave two migration heads once both merge, so Ramp committed migrations/migration-hash.txt, a hash of every migration filename: any two pull requests that add migrations conflict in Git, and the second must update.
2022: The average time to merge decreased from 1+ hour to 12 minutes, the time the test suite takes to run. Source
- Use it when
- Most pull requests touch independent code, a branch rule makes each one re-run the full suite after every merge, and the main branch can run the suite before each deploy.
- Before applying it
- Two pull requests that pass separately can still fail together; the test run on the main branch before deployment is what catches that, so it stays required.
Company Playbooks
- Read Anthropic’s playbookRun only affected tests
- Read DoorDash’s playbookRun build and test in parallel, sharded across CI agents
- Read Linear’s playbookBenchmark the same job on faster hardware
- Read Mastra’s playbookStarSling runners for the slowest test workflow
- Read OpenAI’s playbookRank CI fixes by the minutes they save
- Read Ramp’s playbookMove a CPU-bound test suite to a faster CPU
- Read Shopify’s playbookLet an agent iterate against one CI metric
- Read Spotify’s playbookBuild arm64 on native arm64 runners
- Read Stripe’s playbookRun only the tests a change can reach
- Read Uber’s playbookBuild the merge-queue speculations most likely to be needed