Parallelization and sharding

Shorten the path to a CI result with independent jobs and balanced test shards. Compare how companies divided and scheduled their work.

View in Markdown

Prompt

Evaluate this CI improvement area in my repository: Parallelization and sharding.

Shorten the path to a CI result with independent jobs and balanced test shards. Compare how companies divided and scheduled their work.

Read the techniques and source-backed company examples: https://starsling.dev/ci-playbooks/categories/parallel.md

Inspect this repository’s CI workflows, recent run timings, build and test commands, and dependency graph. Identify the relevant bottleneck and rank applicable techniques using evidence from this repository.
Measure a baseline and compare the relevant techniques, prerequisites, company context, and primary sources. Rank candidate changes by fit and observable opportunity.
Recommend the smallest useful change and the verification plan. ASK ME TO APPROVE THE EXACT CHANGE BEFORE EDITING ANY FILES.
Only after approval, implement on a new feature branch and open a reviewable pull request. Never merge or push to a default or protected branch.
Preserve required checks, test coverage, release and deployment behavior, credential boundaries, and necessary fallback paths. Run relevant checks and compare equivalent runs for elapsed time, queue time, runner minutes, and reliability. Report any unmeasured outcomes.

View all playbooks

Where to start

Diagnose first: which dependency chain determines the final required check, and which jobs can start without outputs from the others?

Parallel work helps when it shortens the chain of jobs that gates a result. Identify real dependencies, shared setup costs, and the longest shard. Measure both elapsed time and runner minutes as you change the split.

17 techniques with source-backed company examples.

CI technique

Split long test suites across runners

Distribute a suite across CI jobs while keeping complete test discovery and collecting the results from every shard.

Split test files across CI shards that stay balanced to within one file

Bun v1.3.13 added bun test --shard=M/N to split test files across CI runners, in the syntax Jest, Vitest and Playwright use. Files are sorted by path for determinism and distributed round-robin, so each shard stays balanced to within one file of the others. Inside one runner, bun test --parallel distributes files across worker processes, the CPU count by default, and idle workers steal work from the busiest remaining queue. Output stays identical to serial execution.

Split test files across multiple CI runners with the new --shard flag, matching the syntax used by Jest, Vitest, and Playwright. Source

Use it when: One test job runs a large suite on a single runner, and the suite can be split by file or run in more than one process.
Before applying it: Splitting by path balances file counts. Compare shard durations after the change, and move to duration-based splitting if one shard stays the long pole. Tests that share state need isolation before they run in parallel.

Shard the test phase across agents

Translated to GitHub Actions

What DoorDash built: DoorDash added sharding to Rush: a sharded operation becomes a pre-shard node, n shard nodes and a collate node that merges coverage.
Your GitHub Actions version: A test matrix of shards with each shard's index and its own output folder, followed by a job that merges coverage into one report.

DoorDash's tests already ran eight Jest workers per machine and had started to hit memory errors, so DoorDash added sharding to Rush upstream. A sharded operation becomes a pre-shard node that keeps the dependencies, n shard nodes, and a collate node that merges coverage into one report; each shard gets its index and its own output folder so the build cache stays valid. A follow-up clusters shards by operation name, so shards of one package can run on different agents. Four shards with seven workers each ran fastest, and every shard finished within six minutes, the same time as the build. DoorDash's Kotlin build service shards unit tests the same way.

All of the test shards completed within six minutes, which tied build time and test time. Source

Use it when: One test job runs long, or runs out of memory, after raising its worker count on one machine.
Before applying it: Every shard repeats setup, so pick the shard count by measurement, and collate coverage so the report still covers the whole suite.
GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation.

Split a long end-to-end suite across runners

StarSling’s agent added three Playwright shards to the kitchen-sink job, passed each shard index to the test runner and kept fail-fast false.

Mastra’s kitchen-sink E2E suite still runs sharded today, now across four shards in e2e-tests.yml. Source

Use it when: A long test job remains the critical path after removing repeated waits and setup.
Before applying it: Each shard repeats setup and consumes runner minutes. Verify the union of tests, unique artifacts and required aggregate status.

Split the slowest job across runners

The CLI's help-compatibility workflow split its dispatch and contract tests into separate jobs on Intel macOS and Windows ARM, and moved an exhaustive flag-inventory test, which starts 482 subprocesses per platform, out of that matrix; the full Linux suite still runs it.

Workflow: 7m10s in the first successful run after merging current main, against the earlier 23-run median of 10m21s Source

Use it when: One job sets the workflow's finish time and its tests fall into independent groups.
Before applying it: Each split job repeats setup and adds runner minutes.

Give a long suite more than one runner

After measuring worker and shard combinations, the CLI raised Playwright's workers setting from 5 to 10 and added a two-shard matrix to its end-to-end job, giving each shard's artifacts a unique name. Each shard is a separate job that runs Playwright with the configured worker limit, so ten workers means up to ten on each machine. The default branch still runs two shards. The CLI's Windows unit tests are sharded the same way.

The PR reports ~4 min faster wall clock: 7m 44s against 11m 43s, a 34% reduction, comparing 10 workers on two shards with 5 workers on one runner. Source

Use it when: One test job is the longest in the workflow after its tests are isolated and its worker count is tuned.
Before applying it: Each shard repeats checkout, install and authentication, so total runner minutes can rise while elapsed time falls.

Shard a serial end-to-end suite across runners

The confidence-cli end-to-end suite runs serially because it uses a fixed auth callback port, and it took about 4.4 minutes of a 5.5-minute pull request run. The change splits it into four matrix shards with vitest --shard, each on its own runner, and adds an aggregate E2E tests job so the check name stays the same.

Before the change, E2E took about 4.4 minutes of a 5.5-minute pull request run; the author estimated about 1.5 minutes after it, with the slowest test file at 74 seconds as the floor (estimate). Source

Use it when: A test suite runs serially because its tests share a fixed resource on one machine, and it takes most of the pull request run.
Before applying it: Each shard repeats checkout and setup, and the slowest test file sets the floor for every shard.

Shard a long suite across runners

Stripe Android partitions discovered PaymentSheet end-to-end test classes across five parallel workflows. Stripe React Native split its end-to-end tests into matrix shards, and Stripe iOS shards full SDK size reports into three jobs on master and releases.

Use it when: One long test job remains the bottleneck after setup and worker tuning.
Before applying it: Each shard repeats setup. Verify the shards together run every original test exactly once, and keep device and diagnostic coverage.

CI technique

Isolate state before running tests in parallel

Give concurrent tests independent databases, services, or fixtures so they can share runner capacity without interfering with each other.

Run isolated end-to-end journeys in parallel

The packaged end-to-end suite ran 66 journeys on one worker. Each journey already had its own home directory, workspace, runtime and server port, so the PR set Playwright to three workers with fullyParallel. Global setup sweeps orphaned processes once and installs the Electron runtime once, before any worker starts.

Use it when: A browser or desktop suite runs on one worker while its tests share no files, ports or processes.
Before applying it: Parallel journeys are safe when each owns its state; a shared file, port or install step becomes a race. Measure the worker count on your own runner.

Give parallel test workers more databases

After the CPU change, raising the pytest-xdist worker count left the suite's time flat at first. Profiling showed the test database buckling under the load: it hit lock contention once more than 20 workers shared it. Ramp ran more test databases, measured the ratio of workers to databases, settled on about 12 workers per database, and tuned Postgres connection settings.

Together with the faster CPUs, the database changes took Ramp to a ~70% speedup of the test suite two days in, a saving of about 30 minutes. Source

Use it when: Tests run in parallel workers against one shared database or service, and adding workers leaves the job's time flat while the database is saturated.
Before applying it: Every extra database uses CPU and memory the test workers would otherwise get; Ramp saw slight regressions past its measured ratio, so keep the count that measured fastest.

Isolate tests before adding workers

The CLI end-to-end suite shared one development store. Shopify/cli#7279 gave each test its own store and each worker its own ports; #7309 then turned on fullyParallel and raised Playwright workers from one to five. The default branch runs the suite fully parallel with four workers.

The PR reports about 10 minutes with 5 workers, down from about 27 minutes with 1 worker. Source

Use it when: Tests run one at a time because they share ports, temporary files, accounts or database state.
Before applying it: More workers add load on shared external APIs; keep teardown observable and keep genuinely serial tests on a serial lane.

Give concurrent tests their own state

Stripe sync-engine gives each end-to-end test file its own Postgres container on a random port and enables file parallelism; webhook tests run in a separate sequential configuration because they share a remote endpoint. TaxJar, a Stripe company, made its remote transaction IDs unique per version and build so its WooCommerce 7.x and 9.x test lanes can run in parallel.

Use it when: A shared database, port or remote account forces the whole suite to run serially.
Before applying it: Keep intentionally shared-state tests in a serial lane, keep teardown reliable, and use isolated sandbox accounts for remote services.

Give concurrent suites their own services

SubmitQueue replaced shared manual service setup with a Compose stack per test context. Docker allocates an ephemeral host port and the test discovers it at runtime, so the suites run as separate parallel CI jobs. OPT88's resource-isolation step is adapted; the readiness and image-tag fixes on this page complete it.

The PR creates separate CI jobs for the suites and an isolated Compose project per test context. Source

Use it when: Independent integration suites collide on one port, database or container name, forcing serial execution.
Before applying it: Use collision-proof namespaces and guaranteed cleanup after every run. Separate data and image tags as well as ports, and use a protocol-level readiness probe in place of a fixed startup sleep.

CI technique

Run independent checks in parallel

Remove false dependencies between lint, build, test, and formatting checks so independent work can start together.

How it works

Identify checks that can run without another check’s output. Give those checks independent execution paths so lint, formatting, builds or tests can start together. Keep dependencies where a job consumes another job’s artifacts, and keep each required result in the final merge decision. The examples below show both splitting sequential work and removing unnecessary job dependencies.

Run build-free checks without waiting for the build

Bun's source-lint tests read only the source tree, yet each test shard ran them after waiting for the full build. They now run in their own GitHub Actions workflow, and the test shards exclude them.

Use it when: A check depends on a build job through needs: or runs inside a build-gated job, but never reads the build's output.
Before applying it: The moved check stays required, and only checks that never read the build output move.

Run lint beside the build, as its own step

Next.js can lint as part of its build, but that lint runs late in the build. DoorDash decoupled ESLint from the Next.js build and runs its three minutes of linting in parallel with the build, so developers see lint results five minutes sooner. Together with the cobuild fix and the cache change, this cut CI runtime by six minutes and halved billed agent minutes.

Implementing these three changes decreased CI runtime by six minutes and cut billed minutes for agent usage in half. Source

Use it when: A build command also runs linting or type checks, so the job waits for both one after the other.
Before applying it: Disable the build's own lint only once the separate lint job is a required check, so pull requests stay gated on lint.

Run independent lint and formatting checks together

Mastra scheduled ESLint and formatting checks in one Turbo invocation, applying the parallel-steps recipe through the task runner. The write/autofix commands stayed sequential, and formatting tasks remained uncached.

Use it when: Lint and formatting independently inspect the same checkout.
Before applying it: Preserve the previous file filters and both exit statuses. Concurrent writers are a different case; keep autofixes ordered and measure CPU contention.

Let tests start without waiting for lint

Tecnix's test legs waited for its basic checks before starting. Shopify/tecnix#74 removes that edge; lint problems still turn the required aggregate basic-checks job red while the much longer test legs run alongside it.

Merged 2026-10-01; the test job on the default branch has no needs edge. Source

Use it when: Long jobs wait on another job whose outputs and artifacts they do not use.
Before applying it: Keep dependencies that are deliberate gates, such as checks that must pass before a deploy.

Run independent checks at the same time

Translated to GitHub Actions

What Uber built: Uber's landing validation ran environment preparation, pre-checks, changed-targets calculation, build and test, and publishing in sequence.
Your GitHub Actions version: Run a check that depends on neither the build nor the change calculation in parallel with them.

Uber's landing validation ran environment preparation, pre-checks, CTC, Bazel build and test, and result publishing in sequence. Its P95 and P99 breakdown showed Bazel build and test, CTC and the resolve-dependencies check as the main consumers. The dependency check depends on neither CTC nor the build, so Uber moved it to run in parallel.

Use it when: A long job runs heavy steps one after another, and at least two of them read none of each other's outputs.
Before applying it: Separate jobs repeat checkout and setup. Measure the longest path, and keep the required aggregate waiting on every split check.
GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation.

CI technique

Balance test work across shards

Use measured runtimes or smaller test files to reduce the slowest shard, which determines when the suite finishes.

Balance test files before buying more shards

Linear split oversized test files into smaller files, then benchmarked shard and runner configurations. Moving from four to eight shards made the critical job roughly 19% faster and 19% cheaper in the initial benchmark.

Use it when: A few files dominate a file-based shard assignment.
Before applying it: Preserve test discovery, assertions and artifacts. Vitest assigns work by file, so one large file still sets its shard’s time.

Split long test jobs into balanced shards

Codex split its single Windows Bazel test job into four shard jobs behind one aggregate check. A later change assigns targets by estimated duration, longest first, to the least-loaded shard.

Windows tests: one sharded run in about 7m29s, against a recent single-job median around 22m26s Source

Use it when: A test job dominates the critical path, or existing shards finish at very different times.
Before applying it: Balance shard duration and total runner minutes together. A checked-in timing table goes stale as new tests arrive, so refresh it or use the runner's own timing data.

Pack tests onto workers so they finish together

With the suite running in parallel pytest-xdist workers, Ramp saw the workers finish at different times, so the slowest worker set the job's length. The team wrote a more effective packing algorithm, with ChatGPT's help, as a proof of concept on a fork of pytest.

The more effective packing algorithm saved 2 to 3 minutes. Source

Use it when: A test job runs in parallel workers or matrix shards, and the last one regularly finishes well after the first.
Before applying it: A packing algorithm that uses test timings needs them recorded; refresh them as tests are added so new slow tests spread across the workers.

Balance shards and keep a catch-all for new tests

Stripe iOS balanced PaymentSheet UI tests across four jobs: three with explicit assignments and a fourth that runs everything else, including new tests. A validator rejects duplicate assignments and stale exclusions.

In Stripe’s combined CI runs, the slowest PaymentSheet UI job went from about 26 to about 12 minutes, with worker and machine changes in the same PR. Source

Use it when: One shard sets the finish time while others wait idle.
Before applying it: Keep the union of test IDs identical and track new, renamed and deleted tests.

CI technique

Match test concurrency to the runner

Measure worker counts against available CPU and memory. More workers help only while the runner has capacity.

Set test worker width from the actual host

Bun removed the CPU and thread caps that held its Linux test agents to two usable cores, and sized the parallel test width from the available cores. A follow-up capped the width at four on macOS, where each M2 Ultra host runs two tart virtual machines and every coordinator forks its own workers.

Use it when: A fixed worker count has never been measured, or workers contend for memory, physical cores or a shared host.
Before applying it: Nested worker processes and other virtual machines on the same host count toward the width.

Fit test concurrency to the runner’s memory

The memory integration configuration capped Vitest threads at two after the source reported runner disconnects and memory pressure.

The source reports 61 failures in 771 runs. Source

The current Vitest configuration expresses the limit as maxWorkers: 2. Source

Use it when: Parallel tests and build output compete for limited runner memory.
Before applying it: Preserve the separate build heap requirement and measure peak memory, elapsed time and failures across worker counts.

Measure how many tests to run at once

Codex compared Windows Bazel shards at 4, 8 and 12 local test jobs in a forced-test-execution experiment, then set the CI default to 8 with one .bazelrc line. Sharding spreads work across runners; this tunes how much runs at once on each runner.

Slowest shard: 835.5s to 609.3s at 8 local test jobs, in the experiment the PR cites Source

Use it when: Test parallelism is a default or a guess for this runner.
Before applying it: More parallel workers can increase memory pressure and contention. Compare the same workload and runner, including reliability and total minutes.

Measure workers against the runner

Stripe iOS tried six simulator workers and kept four, which was faster and more reliable, and split Crypto UI tests into separate classes so Xcode could run them in parallel. Stripe Android sized individual device-heavy lanes and raised its managed-device ceiling to match its six shards. Metronome, a Stripe company, moved a publishing job whose doc build was timing out to an 8-core runner, on a suggestion from the Stainless team.

In the same combined runs, the Crypto UI test step went from about 10 to about 6 minutes. Source

Use it when: Tests run serially, or worker counts and runner sizes were chosen without measurement.
Before applying it: Isolate shared accounts and ports before adding workers, and keep tests that verify serial behavior in a serial lane. Measure each change separately; the source PRs changed workers and machines together.

CI technique

Run independent test packages concurrently

Let independent packages or test environments run at the same time instead of invoking each one serially.

Bun's Miri script builds every crate once, then runs one cargo miri test per crate, as many at a time as the runner has cores, longest crates first. It waits for all of them and exits nonzero if any crate reports an error.

The PR expects the Miri job to go from about 14.5 minutes to about 4 to 5 minutes (estimate), with the concurrent crates and a smaller Miri-sized tracker test together. Source

Use it when: A serial test driver runs independent packages, crates or modules one after another on a multi-core runner.
Before applying it: Each process keeps its own log, and the job reports every failing package.

Run the slowest test environment in parallel workers

This is a community contribution to Spotify's luigi project. Luigi's core test environment took about five minutes while the others took about a minute or less. The change adds pytest-xdist and runs the core tests with -n auto --dist=loadfile, which spreads test files across one worker per CPU and keeps each file's tests on one worker.

The author reports the core workflows running in 2 to 4 minutes, 1 to 2 minutes faster. Source

Use it when: One test environment takes most of the CI time and runs in a single process on a multi-core runner.
Before applying it: Tests that share files, ports or global state need per-worker isolation; grouping by file keeps a file's tests together.

Let the test runner schedule independent packages

After isolating its unit suite, Cadence replaced a shell loop that ran go test once per package with one invocation over the package list, so Go schedules independent packages concurrently and writes one coverage profile. OPT88's serial-suite diagnosis is adapted.

The PR reports unit and integration test runs and builds on the dependency isolation in cadence-workflow/cadence#6800. Source

Use it when: A shell loop runs independent packages one at a time even though the language test runner schedules packages in parallel.
Before applying it: Remove shared-resource dependencies first, keep special-case tests, race settings and combined coverage, and measure CPU and memory at several concurrency levels.

CI technique

Build the test database once for all workers

Have one parallel worker apply the schema while the others wait for it, so more workers stop repeating the same setup.

Ramp's test setup recreated the database from scratch by applying every migration, and each pytest-xdist worker ran that setup in parallel. At higher worker counts the concurrent schema changes ran Postgres out of working memory and repeated the same work. Ramp restructured the setup so one worker applies the schema, which let it run more workers on a machine twice the size.

Coordinating the schema setup let Ramp double the machine size and halve CI test times, which had been creeping up towards 20 minutes. Source

Use it when: Tests run in parallel workers that each repeat the same database or fixture setup, and raising the worker count stops paying off.
Before applying it: The other workers wait for the one that builds the schema, so the setup itself still sets a floor on the job's time.

CI technique

Clean up independent test resources concurrently

Independent index deletions do not need a separate serial wait per index.

Couchbase cleanup replaced serial index deletion loops with Promise.all followed by one wait. This applies independent-step parallelism inside a test helper.

Use it when: Suite setup or cleanup serializes operations on independent resources.
Before applying it: Check service concurrency limits, error reporting and shared state, and confirm deletions completed before later tests run.

CI technique

Compress release artifacts in parallel

Compress independent release binaries two at a time.

Codex replaced the loop that archived each Windows release binary in turn with xargs running two workers, so independent binaries compress at the same time.

Compression step: serial 121 s, 123 s, 121 s against parallel 73 s, 73 s, 74 s, three trials each on a Windows x64 release runner Source

Use it when: A release packaging step compresses or archives independent files one after another.
Before applying it: More workers compete for CPU and disk on the same runner; measure the worker count, and keep the step failing when any worker fails.

CI technique

Let shared packages without a build step stay cacheable

Treat a package with nothing to build as cacheable, so its dependents spread across every CI agent.

Translated to GitHub Actions

What DoorDash built: DoorDash builds its web monorepo with Rush cobuilds, where each CI agent picks up any task whose dependencies are done.
Your GitHub Actions version: Split the build across parallel jobs by the project graph, and let projects with no build output run as separate tasks.

DoorDash builds its web monorepo with Rush cobuilds, where each CI agent picks up any task whose dependencies are done. Cobuilds kept a project and its dependents on one agent when the project had no cacheable output, and DoorDash's shared rigs and ESLint profile have no build step, so nearly every project landed in one cluster. DoorDash contributed the fix upstream: a no-op operation now counts as cacheable. With the clusters gone, DoorDash cut its cobuild agents from ten to five, and cobuilds used all five.

Use it when: A distributed or sharded build keeps work on one machine whenever it depends on a package that produces no output.
Before applying it: A no-op has nothing to restore, so this holds only for packages with no build or test script; a package that writes output still needs its outputs declared.
GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation.

CI technique

Move unit tests to a parallel-by-default framework, gated on ten clean runs

Migrate serial tests to a framework that runs them in parallel, and ship each package after ten green runs.

Mostly serialized XCTest runs had stretched DoorDash's iOS CI test step into double-digit minutes. DoorDash migrated its shared iOS packages to Swift Testing, which runs synchronous and asynchronous tests in parallel by default. A Cursor rule made the mechanical edits, and the migration prompt told the agent to run each package's tests with Bazel after every file and fix failures before moving on. A package counted as migrated once its tests passed ten times in a row, which surfaced tests that depended on order, shared state or timing. The unit test step in CI got about 60% faster, and overall builds for shared iOS packages about 40% faster.

The unit test step in CI became roughly 60% faster. Source

This contributed to an average of about 40% faster overall builds for shared iOS packages. Source

Use it when: A test step runs mostly serially, and the test framework or a newer one can run the same tests in parallel.
Before applying it: Parallel runs expose hidden order, shared-state and timing assumptions; fix each root cause, and leave tests on the old framework where the new one lacks a needed feature.

CI technique

Overlap independent setup with explicit joins

Pull images while dependencies install, and wait for each background step explicitly.

Stripe sync-engine pulled its engine and service images while installing and building Node dependencies, since the two steps use different outputs.

Use it when: Independent image downloads and local dependency work run one after another.
Before applying it: Shared CPU, disk and network can erase the benefit. Wait on each background process by ID and propagate every exit status, and keep gates and publishing sequential.

CI technique

Reduce the setup floor before increasing shard count

Each extra shard repeats checkout, dependencies and container setup.

Each extra shard repeats checkout, dependencies and container setup. Before its setup work, Linear’s test job used 4 shards and spent 8.3 minutes on setup; afterward it ran 8 shards with 7.5 minutes of setup.

Use it when: More shards would repeat substantial fixed setup work.
Before applying it: Remeasure per-shard setup before changing shard count.

CI technique

Split independent work off the longest job

Run independent suites in parallel jobs when the overlap saves more than the repeated setup costs.

Stripe iOS split installation and integration tests into separate jobs, keeping all five installation variants together so they share setup. Earlier, it moved one measured slow test class into its own job.

The combined job took about 27 minutes; after the split, installation took about 11 to 12 minutes and integration about 13 minutes, running in parallel, in Stripe’s combined CI runs. Source

Use it when: Two independent suites run one after another in the job that holds up the pull request.
Before applying it: Each new job repeats setup and may queue. Keep every test variant and simulator version, and keep one stable required check that reports every job’s result.

CI technique

Test supported runtimes concurrently

Give each target framework its own job that installs only what it needs.

Stripe .NET replaced one sequential build-and-test job with per-framework matrix jobs and a native Windows .NET Framework lane. Each lane installs the latest SDK plus only its target runtime, and example builds run in their own job.

Use it when: A multi-runtime suite runs sequentially or installs every toolchain in every lane.
Before applying it: Keep every supported runtime gated, let every diagnostic matrix leg run to completion, and update branch protection for the new job names.

CI technique

Weight memory-heavy steps so build and test run in parallel on separate agents

Give heavy operations a weight, so the scheduler never places build and test on the same agent.

Translated to GitHub Actions

What DoorDash built: DoorDash finished and extended weighted operations in Rush, so an agent starts an operation only when its memory weight fits.
Your GitHub Actions version: Run build and test as parallel jobs on separate runners, sized to the memory each needs.

Once its Next.js build no longer needed a pre-compile step, DoorDash could start Jest tests alongside the build. The build needs 12 GB of memory and the tests need 20 GB, so the two could never share an agent. DoorDash finished and extended weighted operations in Rush upstream, building on earlier work: each operation declares how many concurrency units it takes, and an agent starts an operation only when its weight fits. Build and test then ran in parallel on different agents, halving CI time to 11 minutes. DoorDash's Kotlin build service schedules by memory the same way, with tokens sized from each project's past memory use.

Use it when: Two heavy steps run one after the other because running them together on one runner would exhaust its memory.
Before applying it: Weights stand in for memory or CPU, so measure each step's peak memory before you set them, and re-measure as the suite grows.
GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation.

Where to start

Split test files across CI shards that stay balanced to within one file

Shard the test phase across agents

Split a long end-to-end suite across runners

Split the slowest job across runners

Give a long suite more than one runner

Shard a serial end-to-end suite across runners

Shard a long suite across runners

Run isolated end-to-end journeys in parallel

Give parallel test workers more databases

Isolate tests before adding workers

Give concurrent tests their own state

Give concurrent suites their own services

How it works

Run build-free checks without waiting for the build

Run lint beside the build, as its own step

Run independent lint and formatting checks together

Let tests start without waiting for lint

Run independent checks at the same time

Balance test files before buying more shards

Split long test jobs into balanced shards

Pack tests onto workers so they finish together

Balance shards and keep a catch-all for new tests

Set test worker width from the actual host

Fit test concurrency to the runner’s memory

Measure how many tests to run at once

Measure workers against the runner

Run independent test packages concurrently

Run the slowest test environment in parallel workers

Let the test runner schedule independent packages

Build the test database once for all workers

Clean up independent test resources concurrently

Compress release artifacts in parallel

Let shared packages without a build step stay cacheable

Move unit tests to a parallel-by-default framework, gated on ten clean runs

Overlap independent setup with explicit joins

Reduce the setup floor before increasing shard count

Pay setup twice instead of seven times

Split independent work off the longest job

Test supported runtimes concurrently

Weight memory-heavy steps so build and test run in parallel on separate agents

Company Playbooks

Related guides

Shard tests across parallel jobs

Parallelize independent jobs