Stop downloading the test image on every build
Cache the container image the tests run in, so each build reuses the copy it already has.
How Ramp used this technique
Read what Ramp changed and follow the original sources.
Stop downloading the test image on every build
Each build pulled the last-built Docker image from the registry, which took 1 to 2 minutes. Ramp's new test runners were self-hosted, so the image stayed in the runner's local Docker cache between builds. On GitHub-hosted runners, which start empty, a job that builds the image may cut the same overhead with a persisted build cache (the OPT4 recipe), and a job that only pulls a prebuilt public image may cut it by saving and loading the image through the Actions cache; measure the restore time against the pull it replaces.
Local Docker caching shaved over a minute off a run that took about fifteen minutes by then. Source
- Use it when
- Jobs pull or rebuild the same container image on every run and spend a minute or more on it.
- Before applying it
- A cached image must still rebuild when its inputs change, so key the cache on the Dockerfile and the lockfiles it copies in.
Company Playbooks
- Read Anthropic’s playbookRun only affected tests
- Read DoorDash’s playbookRun build and test in parallel, sharded across CI agents
- Read Linear’s playbookBenchmark the same job on faster hardware
- Read Mastra’s playbookStarSling runners for the slowest test workflow
- Read OpenAI’s playbookRank CI fixes by the minutes they save
- Read Ramp’s playbookMove a CPU-bound test suite to a faster CPU
- Read Shopify’s playbookLet an agent iterate against one CI metric
- Read Spotify’s playbookBuild arm64 on native arm64 runners
- Read Stripe’s playbookRun only the tests a change can reach
- Read Uber’s playbookBuild the merge-queue speculations most likely to be needed