--- title: "Balance test work across shards" description: "Use measured runtimes or smaller test files to reduce the slowest shard, which determines when the suite finishes." url: https://starsling.dev/ci-playbooks/techniques/balance-test-shards canonicalUrl: https://starsling.dev/ci-playbooks/techniques/balance-test-shards --- # Balance test work across shards Use measured runtimes or smaller test files to reduce the slowest shard, which determines when the suite finishes. ## How to choose an approach Compare the examples by repository setup, prerequisites, and risks before adopting one. Each company implemented this in its own environment; measure each change against your own baseline. - **[Linear](https://starsling.dev/ci-playbooks/linear#balance-the-shards)**: Use it when A few files dominate a file-based shard assignment. Check before adopting: Preserve test discovery, assertions and artifacts. Vitest assigns work by file, so one large file still sets its shard’s time. - **[OpenAI](https://starsling.dev/ci-playbooks/openai#balanced-shards)**: Use it when A test job dominates the critical path, or existing shards finish at very different times. Check before adopting: Balance shard duration and total runner minutes together. A checked-in timing table goes stale as new tests arrive, so refresh it or use the runner's own timing data. - **[Ramp](https://starsling.dev/ci-playbooks/ramp#pack-tests-by-duration)**: Use it when A test job runs in parallel workers or matrix shards, and the last one regularly finishes well after the first. Check before adopting: A packing algorithm that uses test timings needs them recorded; refresh them as tests are added so new slow tests spread across the workers. - **[Stripe](https://starsling.dev/ci-playbooks/stripe#rebalance-and-cover-shards)**: Use it when One shard sets the finish time while others wait idle. Check before adopting: Keep the union of test IDs identical and track new, renamed and deleted tests. ## Apply this technique ````text Evaluate and apply this CI technique to my repository: Balance test work across shards. Use measured runtimes or smaller test files to reduce the slowest shard, which determines when the suite finishes. Read the technique and its source-backed company examples: https://starsling.dev/ci-playbooks/techniques/balance-test-shards.md Inspect this repository’s CI workflows, build and test commands, dependencies, and recent run timings. Identify where this technique applies and explain the expected benefit using this repository’s evidence. Measure a baseline from recent comparable runs: elapsed time, queue time, runner minutes and failures where available. If data is unavailable, say what cannot be measured. Compare relevant company approaches, checking prerequisites, platform differences and tradeoffs. Rank applicable options by evidence, fit and measurable opportunity; do not present company results as predictions for this repo. Recommend the smallest safe change, show what checks and fallback paths remain, and ASK ME WHICH CHANGE TO APPROVE BEFORE EDITING ANY FILES. Only after approval, work on a new feature branch, make one reversible change, and open a reviewable pull request. Never merge or push to a default or protected branch. Keep required checks, test coverage, deployment and release behavior intact. Validate the change and compare equivalent runs where possible; report limitations and what should be monitored after merge. ```` ## How companies used this technique Read what each company changed and follow the original sources. ### Linear: Balance test files before buying more shards Linear split oversized test files into smaller files, then benchmarked shard and runner configurations. Moving from four to eight shards made the critical job roughly 19% faster and 19% cheaper in the initial benchmark. Evidence: Documented technique Use it when: A few files dominate a file-based shard assignment. Before applying it: Preserve test discovery, assertions and artifacts. Vitest assigns work by file, so one large file still sets its shard’s time. [Read more in the Linear playbook](https://starsling.dev/ci-playbooks/linear#balance-the-shards) [Primary source](https://linear.app/now/ci-bottleneck-reworked#balance-work-the-way-the-test-runner-sees-it) ### OpenAI: Split long test jobs into balanced shards Codex split its single Windows Bazel test job into four shard jobs behind one aggregate check. A later change assigns targets by estimated duration, longest first, to the least-loaded shard. Evidence: Single-run demonstration Windows tests: one sharded run in about 7m29s, against a recent single-job median around 22m26s ([Source](https://github.com/openai/codex/pull/22408)) Use it when: A test job dominates the critical path, or existing shards finish at very different times. Before applying it: Balance shard duration and total runner minutes together. A checked-in timing table goes stale as new tests arrive, so refresh it or use the runner's own timing data. [Read more in the OpenAI playbook](https://starsling.dev/ci-playbooks/openai#balanced-shards) [Primary source](https://github.com/openai/codex/pull/22408) ### Ramp: Pack tests onto workers so they finish together With the suite running in parallel pytest-xdist workers, Ramp saw the workers finish at different times, so the slowest worker set the job's length. The team wrote a more effective packing algorithm, with ChatGPT's help, as a proof of concept on a fork of pytest. Evidence: Company-reported result or observation The more effective packing algorithm saved 2 to 3 minutes. ([Source](https://builders.ramp.com/post/faster-python-tests-part-two)) Use it when: A test job runs in parallel workers or matrix shards, and the last one regularly finishes well after the first. Before applying it: A packing algorithm that uses test timings needs them recorded; refresh them as tests are added so new slow tests spread across the workers. [Read more in the Ramp playbook](https://starsling.dev/ci-playbooks/ramp#pack-tests-by-duration) [Primary source](https://builders.ramp.com/post/faster-python-tests-part-two) ### Stripe: Balance shards and keep a catch-all for new tests Stripe iOS balanced PaymentSheet UI tests across four jobs: three with explicit assignments and a fourth that runs everything else, including new tests. A validator rejects duplicate assignments and stale exclusions. Evidence: Source PR/commit evidence In Stripe’s combined CI runs, the slowest PaymentSheet UI job went from about 26 to about 12 minutes, with worker and machine changes in the same PR. ([Source](https://github.com/stripe/stripe-ios/pull/7333)) Use it when: One shard sets the finish time while others wait idle. Before applying it: Keep the union of test IDs identical and track new, renamed and deleted tests. [Read more in the Stripe playbook](https://starsling.dev/ci-playbooks/stripe#rebalance-and-cover-shards) [Primary source](https://github.com/stripe/stripe-ios/pull/7333) ## CI optimization is continuous work. Apply this technique yourself, or use StarSling Runners for faster GitHub Actions and optional agent-driven optimization PRs, which teams on eligible paid plans can turn on. [Get self-improving CI](https://starsling.dev/products/runners) ## Related techniques ### [Split long test suites across runners](https://starsling.dev/ci-playbooks/techniques/shard-test-suites) Distribute a suite across CI jobs while keeping complete test discovery and collecting the results from every shard. 7 implementations · 7 companies (Anthropic, DoorDash, Mastra, OpenAI, Shopify, Spotify, Stripe) ### [Reduce the setup floor before increasing shard count](https://starsling.dev/ci-playbooks/techniques/setup-cost-limits-sharding) Each extra shard repeats checkout, dependencies and container setup. 1 implementation · 1 company (Linear) ### [Split independent work off the longest job](https://starsling.dev/ci-playbooks/techniques/split-long-lanes) Run independent suites in parallel jobs when the overlap saves more than the repeated setup costs. 1 implementation · 1 company (Stripe) ## Company Playbooks ### [Anthropic](https://starsling.dev/ci-playbooks/anthropic) Run only affected tests ### [DoorDash](https://starsling.dev/ci-playbooks/doordash) Run build and test in parallel, sharded across CI agents ### [Linear](https://starsling.dev/ci-playbooks/linear) Benchmark the same job on faster hardware ### [Mastra](https://starsling.dev/ci-playbooks/mastra) StarSling runners for the slowest test workflow ### [OpenAI](https://starsling.dev/ci-playbooks/openai) Rank CI fixes by the minutes they save ### [Ramp](https://starsling.dev/ci-playbooks/ramp) Move a CPU-bound test suite to a faster CPU ### [Shopify](https://starsling.dev/ci-playbooks/shopify) Let an agent iterate against one CI metric ### [Spotify](https://starsling.dev/ci-playbooks/spotify) Build arm64 on native arm64 runners ### [Stripe](https://starsling.dev/ci-playbooks/stripe) Run only the tests a change can reach ### [Uber](https://starsling.dev/ci-playbooks/uber) Build the merge-queue speculations most likely to be needed ## Related guides Implementation guides for this technique. ### [Shard tests across parallel jobs](https://starsling.dev/best-practices/github-actions/shard-tests) Split a long suite with a matrix and your test framework’s sharding support. ### [Cut repeated test setup](https://starsling.dev/best-practices/github-actions/cut-repeated-test-setup) Move shared fixture work out of individual test cases and keep necessary cleanup. ### [Diagnose slow tests in CI](https://starsling.dev/github-actions/problems/slow-tests) Check common causes in order and confirm each with workflow evidence. ### [Optimize Playwright tests](https://starsling.dev/github-actions/optimizations/optimize-playwright) Choose trace and video retention and install the browsers your suite uses. ### [Make Vitest run mode explicit](https://starsling.dev/github-actions/optimizations/optimize-vitest) Pin run mode to the test command and bound the job with a timeout. ### [Parallelize independent jobs](https://starsling.dev/github-actions/optimizations/parallelize-independent-jobs) Run independent validation lanes concurrently and keep genuine job dependencies.