- All playbooks
- Queue and runner speed
- Move bottlenecked jobs to faster runners
Move bottlenecked jobs to faster runners
Benchmark the same workload on faster hardware and compare elapsed time and cost before changing runner capacity.
How companies used this technique
Read what each company changed and follow the original sources.
Benchmark the same job on faster hardware
Linear moved the same pipeline to runners with faster CPUs, higher-performance storage and better cache infrastructure. Across the two days either side of the switch, jobs ran 34% faster on average.
- Use it when
- Build or test execution still dominates after cheaper setup changes.
- Before applying it
- Match operating system, architecture, workload and event mix. Compare elapsed time, queue time and runner cost; StarSling sells a runner option worth benchmarking.
Move the slowest test workflow onto StarSling runners
StarSling’s agent moved every job in Mastra’s Combined store Tests workflow onto StarSling runners. Follow-up PRs moved the memory, E2E, workspace, prebuild and changed-test workflows the same way.
StarSling measured Combined store Tests at 5.87x faster wall-clock p95 after the runner swap and the agents’ test fixes: 29m 56s to 5m 06s. Successful push and same-repository PR runs across all branches: Feb 21 to 24 before migration versus Apr 1 to 4, 2026 after. The after window was the highest-ranked of seven candidate windows. Source
- Use it when
- One workflow sets the critical path and its jobs run on standard hosted runners.
- Before applying it
- Swap one workflow first and compare wall-clock p95 over matched windows before moving the rest.
Put the slowest job on a faster runner
Codex moved its Windows Bazel jobs, often the long pole for PR checks, from the default hosted runners to a dedicated runner group.
Windows Bazel jobs moved to a dedicated runner group Source
- Use it when
- The critical-path job is compute-bound and cheaper setup, cache and test fixes are exhausted.
- Before applying it
- Benchmark comparable workloads and include runner cost. Fork or event restrictions may require the existing label for some runs.
Move a CPU-bound test suite to a faster CPU
In July 2023 Ramp's Python monolith test suite took 45 minutes, about 38 of them in the pytest invocation. The team watched the resource usage of a machine running the suite and found it CPU bound. Moving the suite to a faster CPU was the first change they made and the single largest speedup of the project.
2023: The faster CPU made the tests run about 50% faster, the first step of the week that took the suite from 45 minutes to under 5. Source
- Use it when
- Profiling the machine that runs the slowest test job shows the CPU saturated for most of the run.
- Before applying it
- A faster runner class bills at a different rate, so compare runner minutes and cost per run alongside elapsed time.
Company Playbooks
- Read Anthropic’s playbookRun only affected tests
- Read DoorDash’s playbookRun build and test in parallel, sharded across CI agents
- Read Linear’s playbookBenchmark the same job on faster hardware
- Read Mastra’s playbookStarSling runners for the slowest test workflow
- Read OpenAI’s playbookRank CI fixes by the minutes they save
- Read Ramp’s playbookMove a CPU-bound test suite to a faster CPU
- Read Shopify’s playbookLet an agent iterate against one CI metric
- Read Spotify’s playbookBuild arm64 on native arm64 runners
- Read Stripe’s playbookRun only the tests a change can reach
- Read Uber’s playbookBuild the merge-queue speculations most likely to be needed