--- title: "Reduce repeated process and build-graph startup" description: "Batch compatible test work or share its build graph to avoid paying process startup and initialization for every file or target." url: https://starsling.dev/ci-playbooks/techniques/reduce-test-process-overhead canonicalUrl: https://starsling.dev/ci-playbooks/techniques/reduce-test-process-overhead --- # Reduce repeated process and build-graph startup Batch compatible test work or share its build graph to avoid paying process startup and initialization for every file or target. ## How to choose an approach Compare the examples by repository setup, prerequisites, and risks before adopting one. Each company implemented this in its own environment; measure each change against your own baseline. - **[Anthropic](https://starsling.dev/ci-playbooks/anthropic#batch-test-processes)**: Use it when Process startup for each test file takes a large share of a big suite, and a reviewed set of files shares no mutable state. Check before adopting: Files in unlisted directories stay on the per-file path, and a new test file in a listed directory joins the batch; the batch reports every file it runs, and any error in the batch turns the job red. - **[OpenAI](https://starsling.dev/ci-playbooks/openai#fewer-processes)**: Use it when A suite repeatedly starts interpreters, CLIs or separate test binaries. Check before adopting: Sharing a process can expose leaked state and environment changes. Keep tests isolated and compare results as well as runtime. - **[Stripe](https://starsling.dev/ci-playbooks/stripe#share-framework-build-graph)**: Use it when Separate invocations rebuild shared dependencies or repeat test-runner startup. Check before adopting: Keep separate lanes for targets whose environment, test plans or process isolation differ. ## Apply this technique ````text Evaluate and apply this CI technique to my repository: Reduce repeated process and build-graph startup. Batch compatible test work or share its build graph to avoid paying process startup and initialization for every file or target. Read the technique and its source-backed company examples: https://starsling.dev/ci-playbooks/techniques/reduce-test-process-overhead.md Inspect this repository’s CI workflows, build and test commands, dependencies, and recent run timings. Identify where this technique applies and explain the expected benefit using this repository’s evidence. Measure a baseline from recent comparable runs: elapsed time, queue time, runner minutes and failures where available. If data is unavailable, say what cannot be measured. Compare relevant company approaches, checking prerequisites, platform differences and tradeoffs. Rank applicable options by evidence, fit and measurable opportunity; do not present company results as predictions for this repo. Recommend the smallest safe change, show what checks and fallback paths remain, and ASK ME WHICH CHANGE TO APPROVE BEFORE EDITING ANY FILES. Only after approval, work on a new feature branch, make one reversible change, and open a reviewable pull request. Never merge or push to a default or protected branch. Keep required checks, test coverage, deployment and release behavior intact. Validate the change and compare equivalent runs where possible; report limitations and what should be monitored after merge. ```` ## How companies used this technique Read what each company changed and follow the original sources. ### Anthropic: Run reliable test files in one parallel invocation per shard Each Bun test shard now runs its fast, historically reliable test files as one bun test --parallel batch, an invocation that distributes the files across a pool of worker processes, where each file used to start its own process. Files that have shown batch sensitivity go on a denylist and run on their own, and so do memory-measuring tests. Evidence: Source PR/commit evidence Per-lane test shard time for the whole change (batching, the full core count on test agents and one up-front addon build per shard), one main build against one branch build: debian x64-asan 8.5 to 4.9 minutes on average (42% lower), windows 2019 x64 8.6 to 6.0 minutes (30%), darwin 26 aarch64 15.5 to 12.6 minutes (19%). ([Source](https://github.com/oven-sh/bun/pull/36175)) Use it when: Process startup for each test file takes a large share of a big suite, and a reviewed set of files shares no mutable state. Before applying it: Files in unlisted directories stay on the per-file path, and a new test file in a listed directory joins the batch; the batch reports every file it runs, and any error in the batch turns the job red. [Read more in the Anthropic playbook](https://starsling.dev/ci-playbooks/anthropic#batch-test-processes) [Primary source](https://github.com/oven-sh/bun/pull/36175) ### OpenAI: Run tests in fewer processes Codex moved every integration test under tests/ into one suite, so cargo builds and starts a single test binary. The Python SDK compiles its dependency-policy checker once per pytest worker and runs each of its policy cases in-process with fresh globals, keeping real subprocess tests for CLI parity. Evidence: Source PR/commit evidence Python 3.14 test job: 7m00s to 2m41s (2m24s before the policy matrix landed) ([Source](https://github.com/openai/openai-python/pull/3830)) Use it when: A suite repeatedly starts interpreters, CLIs or separate test binaries. Before applying it: Sharing a process can expose leaked state and environment changes. Keep tests isolated and compare results as well as runtime. [Read more in the OpenAI playbook](https://starsling.dev/ci-playbooks/openai#fewer-processes) [Primary source](https://github.com/openai/openai-python/pull/3830) ### Stripe: Build related test targets in one graph Stripe iOS replaced many separate framework-test invocations with a combined FrameworkTests-CI scheme, split into PaymentSheet and non-PaymentSheet jobs. Targets with distinct package graphs keep their own paths. Evidence: Source PR/commit evidence Use it when: Separate invocations rebuild shared dependencies or repeat test-runner startup. Before applying it: Keep separate lanes for targets whose environment, test plans or process isolation differ. [Read more in the Stripe playbook](https://starsling.dev/ci-playbooks/stripe#share-framework-build-graph) [Primary source](https://github.com/stripe/stripe-ios/pull/7332) ## CI optimization is continuous work. Apply this technique yourself, or use StarSling Runners for faster GitHub Actions and optional agent-driven optimization PRs, which teams on eligible paid plans can turn on. [Get self-improving CI](https://starsling.dev/products/runners) ## Related techniques ### [Let an agent break import cycles and make the app load lazily](https://starsling.dev/ci-playbooks/techniques/agent-import-cycles) Point a coding agent at the import cycles and eager imports behind a slow app boot, and merge what it lands. 1 implementation · 1 company (Ramp) ### [Share module initialization only for explicitly safe tests](https://starsling.dev/ci-playbooks/techniques/share-module-state) Linear added an opt-in Vitest project with isolate: false, so safe files share a module registry within each worker. 1 implementation · 1 company (Linear) ### [Balance test work across shards](https://starsling.dev/ci-playbooks/techniques/balance-test-shards) Use measured runtimes or smaller test files to reduce the slowest shard, which determines when the suite finishes. 4 implementations · 4 companies (Linear, OpenAI, Ramp, Stripe) ## Company Playbooks ### [Anthropic](https://starsling.dev/ci-playbooks/anthropic) Run only affected tests ### [DoorDash](https://starsling.dev/ci-playbooks/doordash) Run build and test in parallel, sharded across CI agents ### [Linear](https://starsling.dev/ci-playbooks/linear) Benchmark the same job on faster hardware ### [Mastra](https://starsling.dev/ci-playbooks/mastra) StarSling runners for the slowest test workflow ### [OpenAI](https://starsling.dev/ci-playbooks/openai) Rank CI fixes by the minutes they save ### [Ramp](https://starsling.dev/ci-playbooks/ramp) Move a CPU-bound test suite to a faster CPU ### [Shopify](https://starsling.dev/ci-playbooks/shopify) Let an agent iterate against one CI metric ### [Spotify](https://starsling.dev/ci-playbooks/spotify) Build arm64 on native arm64 runners ### [Stripe](https://starsling.dev/ci-playbooks/stripe) Run only the tests a change can reach ### [Uber](https://starsling.dev/ci-playbooks/uber) Build the merge-queue speculations most likely to be needed ## Related guides Implementation guides for this technique. ### [Cut repeated test setup](https://starsling.dev/best-practices/github-actions/cut-repeated-test-setup) Move shared fixture work out of individual test cases and keep necessary cleanup. ### [Shard tests across parallel jobs](https://starsling.dev/best-practices/github-actions/shard-tests) Split a long suite with a matrix and your test framework’s sharding support. ### [Diagnose slow tests in CI](https://starsling.dev/github-actions/problems/slow-tests) Check common causes in order and confirm each with workflow evidence. ### [Optimize Playwright tests](https://starsling.dev/github-actions/optimizations/optimize-playwright) Choose trace and video retention and install the browsers your suite uses. ### [Make Vitest run mode explicit](https://starsling.dev/github-actions/optimizations/optimize-vitest) Pin run mode to the test command and bound the job with a timeout.