Test selection
Choose the tests a change can affect. Compare dependency-based and observed-file approaches, their prerequisites, and their fallback behavior.
Where to start
Diagnose first: does your repo have a trustworthy dependency or observed-file map, and what triggers a full-suite fallback?
Test selection depends on knowing which code a change can reach. Compare how these companies build that relationship, handle shared dependencies, and fall back when a change cannot be classified. Validate the selector against a complete suite before relying on its results.
7 techniques with source-backed company examples.
Skip jobs whose inputs have not changed
Gate expensive workflows and jobs on the files they consume, keeping required check behavior intact.
Skip workflows a change cannot affect
Translated to GitHub Actions
- What Anthropic built
- Anthropic runs a selected set of tests on each pull request.
- Your GitHub Actions version
- Path and job-level filters that keep expensive workflows to changes in the files they depend on.
Anthropic runs a selected set of tests on each pull request. In GitHub Actions, path and job-level filters keep expensive workflows to changes in the files they depend on.
- Use it when
- An expensive workflow runs for unrelated files, or app-specific jobs ignore the dependency graph.
- Before applying it
- Required checks stay reachable: a changes job with job-level conditions keeps a skipped job reporting success.
- GitHub Actions adaptation
- Compare the source system with your own workflow before applying this translation.
Start the native matrix when its inputs change
The self-update PR added native updater journeys on macOS arm64, macOS Intel and Linux arm64. A step in the test job diffs the change against its base and sets an output; the native matrix runs when that output is true. When the checkout lacks the base commit, or the run is a manual dispatch, the matrix runs. Linux amd64 runs the complete process suite on every change.
- Use it when
- An expensive matrix verifies one subsystem and runs on every change to the repository.
- Before applying it
- The path list must cover every input: source, shared packages, lockfiles and the workflow file itself. Required checks must still report when the matrix skips.
Skip whole jobs only when their inputs are unchanged
Mastra gated AGENTS and peer-dependency validation by relevant inputs and generated workspace matrices from affected packages. Job-level conditions and validated matrices sit on top of trigger-level paths filters.
- Use it when
- A workflow serves several independent packages or checks with distinct inputs.
- Before applying it
- Workflow files, shared config, dependency changes and routing-tool changes can affect every target. Keep required status reporting and treat invalid routing artifacts as full coverage.
Skip independent jobs only for proven narrow edits
For modifications to existing top-level pytest modules, Codex Security keeps all five Python jobs and the checks that depend on them, and skips the independent SDK, Windows install, Action and eval jobs. A related change also skips container jobs once it proved those modules sit outside the image's build context.
Across seven earlier PRs that only edited existing pytest modules, the jobs this route skips used 17h29m53s of runner time (139 jobs) Source
- Use it when
- Edits to existing test modules launch unrelated SDK, packaging or container work.
- Before applying it
- Independence needs more than path names: added, deleted, renamed, shared-helper and mixed changes run the full set.
Run the platform checks a binding change affects
Checkout Kit replaced its single protocol signal with language-specific binding signals and a shared-input signal. A Kotlin-only binding change now starts the Android consumers, and a schema or generator change still starts every platform.
Merged 2026-08-03; the per-binding signals are on the default branch. Source
- Use it when
- A monorepo starts every platform job for changes confined to one generated binding.
- Before applying it
- Shared schemas, generators and lockfiles reach several languages, so they map to every consumer.
Run only the tests a change can affect
Use dependency or observed-file relationships to select affected tests, with a full-suite fallback for changes the selector cannot classify.
How it works
Start with the files changed in a pull request, then use the repository’s dependency graph or recorded file usage to identify the tests and build targets that depend on them. Run that affected set for scoped changes. Keep the full-suite path for changes the selector cannot safely classify. The examples below use different selectors, so choose the one that matches your repository’s tooling.
Run only affected tests
Translated to GitHub Actions
- What Anthropic built
- Anthropic's test selection service determines which tests run on each change.
- Your GitHub Actions version
- Scope the slowest build or test job to the changed packages and their dependents.
Anthropic picks which tests run on each pull request from package relevance and past results. In GitHub Actions, the same idea scopes the slowest build or test job to the changed packages and their dependents.
- Use it when
- The merge-gating build or test step dominates and existing tooling can compute affected targets.
- Before applying it
- The scoper falls back to the full suite on an error or an empty result, and the default branch and merge queue keep their full runs.
- GitHub Actions adaptation
- Compare the source system with your own workflow before applying this translation.
Route tests through the dependency graph
Mastra evolved from task hashes to a symbol-aware import graph. Runtime and type-test dependencies require separate treatment. This adapts changed-package filtering to test-file selection.
- Use it when
- A monorepo repeatedly tests unrelated packages for small changes.
- Before applying it
- Follow-ups repaired exported subpaths, merge-base comparisons, Vitest project discovery and JavaScript Vitest configs. Release PRs and alpha publishes run the full suite. Unknown inputs expand to the full suite.
Build only the data models a change modified
Ramp's data CI created an isolated database for each pull request and built every dbt model in it, a full duplicate of production. After each production run, Ramp now saves dbt's manifest.json, and CI builds with --select state:modified+1 against it. Heavy upstream models are cloned from production with zero-copy clones, so CI builds and tests only the modified models and their direct downstream dependencies. The OPT70 recipe scopes a build to changed targets; here the comparison is against the saved production state.
Ramp saw a dramatic dip in the rolling 7 day average cost of its CI warehouses, mostly sustained as pull request volume grew. Source
- Use it when
- CI builds or tests every model, package or target on each pull request, and the tool can select by state against a saved artifact.
- Before applying it
- The selection is only as current as the saved state, so save the artifact after every production run and keep a full build on the default branch or a schedule.
Run only the tests a change can reach
Translated to GitHub Actions
- What Stripe built
- Stripe instruments its Ruby test runner so every file a test opens is recorded against that test, then reruns the tests whose recorded files changed.
- Your GitHub Actions version
- Record the files each test reads, compare the change's file inventory with a recent completed build, and rerun only the tests whose files changed, always including previously failing tests.
Stripe instruments its Ruby test runner so every file a test opens is recorded against that test, including files opened by child processes. For a new change it hashes the file inventory, compares it with a recent completed build and reruns the tests whose recorded files changed. Guardrails always rerun previously failing tests and tests that discover files by globbing, and linters receive the changed-file list.
On average Stripe runs roughly 5% of its full Ruby test suite per build, with a median of 0.5%, using under 10% of the compute of running everything. Source
- Use it when
- A large suite runs in full on every change while most changes touch a small part of the code.
- Before applying it
- Selection is only as complete as the dependency signal. Keep full runs on the default branch, always run tests whose inputs are discovered at runtime, and rerun every test that was red last time.
- GitHub Actions adaptation
- Compare the source system with your own workflow before applying this translation.
Select changed modules and their dependents
Stripe iOS re-records snapshots only for modified modules and the modules that depend on them, walking Xcode target links including shared test utilities. SDK size reports check only affected SDKs and their dependents. When the selection is uncertain, both run in full.
In Stripe’s combined CI runs, framework tests went from about 22 to about 12 to 14 minutes, and snapshots and localization from about 15 to about 13 to 14 minutes, with the related build and job changes. Source
- Use it when
- A monorepo rebuilds and tests modules unrelated to a change.
- Before applying it
- Shared configuration, generated inputs and test utilities can affect untouched modules. Run the full set for anything the graph leaves unplaced, and keep full validation on the default branch.
Install and build only the workspace packages a job uses
Scope dependency installation and builds to the package or lane under test, including its actual dependencies.
Install or build only the workspace packages this step needs
Linear restricted its API test install to the API package and its dependencies, which cut pnpm install from 44-73 seconds to 16-18 seconds, then applied the same filter to API-adjacent jobs. In its public SDK, the release-preparation job builds only the package schema generation imports before updating the schema.
- Use it when
- A job’s dependency closure is smaller than the workspace.
- Before applying it
- Include transitive dependencies and generated prerequisites.
Build only the dependencies the test consumes
The memory test build replaced the entire stores wildcard with libsql, pg and upstash while retaining its other required packages. Another change moved a docs-server build out of the changed-test gate.
- Use it when
- A narrow test job pays for unrelated workspace builds.
- Before applying it
- Trace transitive dependencies and generated artifacts. Before moving a build out of a gate, confirm another required check still runs it.
Give each CI lane a defined package set
The app libraries ran everything in ci.yml in sequence, once per Node version. Shopify/shopify-app-js#2777 split that into a lint job and three package-family jobs that run in parallel; each builds its own dependency closure with pnpm filters and tests only its family. The pattern match is partial: the filters scope each lane, and the speedup comes from running the lanes in parallel.
The PR reports CI running in about 5 minutes against about 11 before. Source
- Use it when
- A monorepo runs one long sequential job that builds and tests every package on every run.
- Before applying it
- A new package has to be added to a lane, and each lane repeats install and setup.
Move selected heavy checks after merge
Keep a deliberately chosen pull request gate and schedule heavier validation after merge, with explicit ownership of the resulting risk.
Run the heavy matrix after merge
Codex kept Bazel build, test and clippy on pull requests, along with Cargo hygiene and its required aggregate, and moved the full Cargo matrix to pushes on main.
Full Cargo matrix moved to post-merge pushes on main Source
- Use it when
- A heavyweight matrix repeats a signal another required workflow already provides.
- Before applying it
- A regression only the deferred legs catch now surfaces after merge. Count those leg-unique failures first, and give the post-merge run an owner who is notified and can block a release.
Move heavyweight checks after merge, by decision
Shopify/tecnix#74 keeps the component tests and cheap flake checks on pull requests. VM tests, the remaining flake checks, the installer tests and flake regressions run on pushes to main, merge-queue runs and manual dispatches, and as optional jobs on pull requests. The required check names still report on every pull request. The pattern match is partial: the PR moves whole suites after merge with their tradeoff written down in the PR.
Merged 2026-10-01; the pull-request conditions are on the default branch. Source
- Use it when
- Heavy suites sit on the required path of every pull request, and the cheaper checks catch nearly every break first.
- Before applying it
- Failures in the moved suites show up on main after merge, so someone has to watch main and act on them.
After the first full agent scan, review only what changed
Run the full agent review once and keep its artifacts, then compare each new commit with the previous one and scan only the changes.
After the first full agent scan, review only what changed
Shopify's security harness, Dispatch, builds code partitions on its first run against an application and deploys hunting agents in parallel, one per partition. That run also produces reusable artifacts, such as documentation of the available APIs and data models, which are stored and kept up to date. On follow-up runs the pipeline compares the latest commit with the previous commit, then scans only the latest changes and updates the stored artifacts. Verifier agents run one after another, so tests that prove a finding do not collide on ports, databases and fixtures.
2026: A full application scan with publicly available frontier models costs between $50 and $300, and incremental diff scans cost roughly $5 to $50. Source
- Use it when
- A scheduled or per-commit agent scan covers a large repository, and most commits touch a small part of it.
- Before applying it
- Run the full scan again when the partitions or the stored artifacts change materially, so a diff scan always works from current context.
Package once, and build full installers after merge
Drop the build step that packaging repeats, and build the unpacked app pull requests test.
Package once, and build full installers after merge
The desktop job ran a Build step and then packaging, which runs the same electron-vite build again. The PR removed the standalone Build step. Pull requests now package the unpacked app that the end-to-end journeys consume; pushes to main still build and inspect the full installers.
- Use it when
- A job builds explicitly before a packaging or test command that builds again, or pull requests build release artifacts that no test reads.
- Before applying it
- Read the wrapper scripts before removing a build step. The full installers still need a post-merge run that someone watches.
Reduce the setup floor before increasing shard count
Each extra shard repeats checkout, dependencies and container setup.
Reduce the setup floor before increasing shard count
Each extra shard repeats checkout, dependencies and container setup. Before its setup work, Linear’s test job used 4 shards and spent 8.3 minutes on setup; afterward it ran 8 shards with 7.5 minutes of setup.
- Use it when
- More shards would repeat substantial fixed setup work.
- Before applying it
- Remeasure per-shard setup before changing shard count.
Company Playbooks
- Read Anthropic’s playbookRun only affected tests
- Read DoorDash’s playbookRun build and test in parallel, sharded across CI agents
- Read Linear’s playbookBenchmark the same job on faster hardware
- Read Mastra’s playbookStarSling runners for the slowest test workflow
- Read OpenAI’s playbookRank CI fixes by the minutes they save
- Read Ramp’s playbookMove a CPU-bound test suite to a faster CPU
- Read Shopify’s playbookLet an agent iterate against one CI metric
- Read Spotify’s playbookBuild arm64 on native arm64 runners
- Read Stripe’s playbookRun only the tests a change can reach
- Read Uber’s playbookBuild the merge-queue speculations most likely to be needed