Agent PR volume
Keep verification moving as agents open more pull requests. Explore techniques for reducing obsolete work and focusing CI on each change.
Where to start
Diagnose first: are agents creating obsolete runs, longer queues, duplicated checks, or more changes that genuinely need verification?
Compare queue time, run count per commit, cancellation rate and critical-path duration before choosing more compute, selective tests or merge-queue changes. Start by measuring how much work is superseded, which tests each change needs, and where jobs wait. Compare these company examples before choosing a change for your repository.
11 techniques with source-backed company examples.
Skip jobs whose inputs have not changed
Gate expensive workflows and jobs on the files they consume, keeping required check behavior intact.
Skip workflows a change cannot affect
Translated to GitHub Actions
- What Anthropic built
- Anthropic runs a selected set of tests on each pull request.
- Your GitHub Actions version
- Path and job-level filters that keep expensive workflows to changes in the files they depend on.
Anthropic runs a selected set of tests on each pull request. In GitHub Actions, path and job-level filters keep expensive workflows to changes in the files they depend on.
- Use it when
- An expensive workflow runs for unrelated files, or app-specific jobs ignore the dependency graph.
- Before applying it
- Required checks stay reachable: a changes job with job-level conditions keeps a skipped job reporting success.
- GitHub Actions adaptation
- Compare the source system with your own workflow before applying this translation.
Start the native matrix when its inputs change
The self-update PR added native updater journeys on macOS arm64, macOS Intel and Linux arm64. A step in the test job diffs the change against its base and sets an output; the native matrix runs when that output is true. When the checkout lacks the base commit, or the run is a manual dispatch, the matrix runs. Linux amd64 runs the complete process suite on every change.
- Use it when
- An expensive matrix verifies one subsystem and runs on every change to the repository.
- Before applying it
- The path list must cover every input: source, shared packages, lockfiles and the workflow file itself. Required checks must still report when the matrix skips.
Skip whole jobs only when their inputs are unchanged
Mastra gated AGENTS and peer-dependency validation by relevant inputs and generated workspace matrices from affected packages. Job-level conditions and validated matrices sit on top of trigger-level paths filters.
- Use it when
- A workflow serves several independent packages or checks with distinct inputs.
- Before applying it
- Workflow files, shared config, dependency changes and routing-tool changes can affect every target. Keep required status reporting and treat invalid routing artifacts as full coverage.
Skip independent jobs only for proven narrow edits
For modifications to existing top-level pytest modules, Codex Security keeps all five Python jobs and the checks that depend on them, and skips the independent SDK, Windows install, Action and eval jobs. A related change also skips container jobs once it proved those modules sit outside the image's build context.
Across seven earlier PRs that only edited existing pytest modules, the jobs this route skips used 17h29m53s of runner time (139 jobs) Source
- Use it when
- Edits to existing test modules launch unrelated SDK, packaging or container work.
- Before applying it
- Independence needs more than path names: added, deleted, renamed, shared-helper and mixed changes run the full set.
Run the platform checks a binding change affects
Checkout Kit replaced its single protocol signal with language-specific binding signals and a shared-input signal. A Kotlin-only binding change now starts the Android consumers, and a schema or generator change still starts every platform.
Merged 2026-08-03; the per-binding signals are on the default branch. Source
- Use it when
- A monorepo starts every platform job for changes confined to one generated binding.
- Before applying it
- Shared schemas, generators and lockfiles reach several languages, so they map to every consumer.
Skip platform jobs for documentation-only changes
Checkout Kit keeps one shared filter file with per-platform filters that exclude Markdown files and docs directories, so a docs-only pull request starts none of the platform suites while workflow and filter changes still trigger validation. The same exclusions now live in .ci/changed-file-filters.yml on the default branch. Shopify CLI tried the workflow-level form and restored the tests, because they are required to merge.
Merged 2026-06-26; the exclusions are on the default branch. Source
- Use it when
- Documentation edits start expensive platform suites that build and test only source code.
- Before applying it
- A workflow skipped at the trigger never reports its required checks; filter at the job level so the checks still report.
Cancel superseded pull request runs
Free runner capacity by canceling validation that a newer commit has replaced. Keep required main-branch and release runs intact.
How it works
Group validation runs by the pull request or branch they validate. When a newer commit starts another run in that group, cancel the older validation. Scope the cancellation rule to work that the new run replaces; deployment, release and migration workflows need their own completion rules.
Cancel superseded runs, never on main
Anthropic's Python and TypeScript SDKs added a workflow concurrency group keyed on the branch, so a newer push cancels the older run. On main the group is keyed on the commit, so main runs never cancel each other.
- Use it when
- Developers or agents push to the same branch several times while CI is still running.
- Before applying it
- Deploy, publish and migration workflows keep running to completion; only runs whose result a newer commit replaces are cancelled.
Cancel obsolete runs using the correct event identity
Mastra added workflow concurrency and cancellation to lint, docs and workspace workflows. Related changes handled workflow_run identity and separated comment-driven approval updates from PR updates.
- Use it when
- Repeated pushes leave obsolete verification jobs occupying runners.
- Before applying it
- Key the group on workflow plus the logical PR or ref, with safe fallbacks, so a new commit cancels the old run. Keep release and destructive side-effect jobs out of indiscriminate cancellation.
Cancel superseded pull request runs
Stripe iOS added a concurrency group to its dead-code workflow, keyed by pull request number, with cancel-in-progress so a new push stops the previous scan.
- Use it when
- Superseded pull request runs keep consuming runners.
- Before applying it
- Scope cancellation to replaceable pull request validation. Releases, deployments, migrations and default-branch runs that must finish keep cancel-in-progress off.
Cancel superseded pull request runs
Base Web added a concurrency group keyed on the workflow and pull request number to its PR Check workflow, with cancel-in-progress, so a new push stops the run it supersedes. NilAway did the same for its Golden Test workflow.
- Use it when
- Developers push several commits to a pull request in quick succession and each push starts a full run.
- Before applying it
- Key the group on the pull request so runs for different pull requests never cancel each other, and keep default-branch and release runs out of cancel-in-progress.
Run only the tests a change can affect
Use dependency or observed-file relationships to select affected tests, with a full-suite fallback for changes the selector cannot classify.
How it works
Start with the files changed in a pull request, then use the repository’s dependency graph or recorded file usage to identify the tests and build targets that depend on them. Run that affected set for scoped changes. Keep the full-suite path for changes the selector cannot safely classify. The examples below use different selectors, so choose the one that matches your repository’s tooling.
Run only affected tests
Translated to GitHub Actions
- What Anthropic built
- Anthropic's test selection service determines which tests run on each change.
- Your GitHub Actions version
- Scope the slowest build or test job to the changed packages and their dependents.
Anthropic picks which tests run on each pull request from package relevance and past results. In GitHub Actions, the same idea scopes the slowest build or test job to the changed packages and their dependents.
- Use it when
- The merge-gating build or test step dominates and existing tooling can compute affected targets.
- Before applying it
- The scoper falls back to the full suite on an error or an empty result, and the default branch and merge queue keep their full runs.
- GitHub Actions adaptation
- Compare the source system with your own workflow before applying this translation.
Route tests through the dependency graph
Mastra evolved from task hashes to a symbol-aware import graph. Runtime and type-test dependencies require separate treatment. This adapts changed-package filtering to test-file selection.
- Use it when
- A monorepo repeatedly tests unrelated packages for small changes.
- Before applying it
- Follow-ups repaired exported subpaths, merge-base comparisons, Vitest project discovery and JavaScript Vitest configs. Release PRs and alpha publishes run the full suite. Unknown inputs expand to the full suite.
Build only the data models a change modified
Ramp's data CI created an isolated database for each pull request and built every dbt model in it, a full duplicate of production. After each production run, Ramp now saves dbt's manifest.json, and CI builds with --select state:modified+1 against it. Heavy upstream models are cloned from production with zero-copy clones, so CI builds and tests only the modified models and their direct downstream dependencies. The OPT70 recipe scopes a build to changed targets; here the comparison is against the saved production state.
Ramp saw a dramatic dip in the rolling 7 day average cost of its CI warehouses, mostly sustained as pull request volume grew. Source
- Use it when
- CI builds or tests every model, package or target on each pull request, and the tool can select by state against a saved artifact.
- Before applying it
- The selection is only as current as the saved state, so save the artifact after every production run and keep a full build on the default branch or a schedule.
Run only the tests a change can reach
Translated to GitHub Actions
- What Stripe built
- Stripe instruments its Ruby test runner so every file a test opens is recorded against that test, then reruns the tests whose recorded files changed.
- Your GitHub Actions version
- Record the files each test reads, compare the change's file inventory with a recent completed build, and rerun only the tests whose files changed, always including previously failing tests.
Stripe instruments its Ruby test runner so every file a test opens is recorded against that test, including files opened by child processes. For a new change it hashes the file inventory, compares it with a recent completed build and reruns the tests whose recorded files changed. Guardrails always rerun previously failing tests and tests that discover files by globbing, and linters receive the changed-file list.
On average Stripe runs roughly 5% of its full Ruby test suite per build, with a median of 0.5%, using under 10% of the compute of running everything. Source
- Use it when
- A large suite runs in full on every change while most changes touch a small part of the code.
- Before applying it
- Selection is only as complete as the dependency signal. Keep full runs on the default branch, always run tests whose inputs are discovered at runtime, and rerun every test that was red last time.
- GitHub Actions adaptation
- Compare the source system with your own workflow before applying this translation.
Select changed modules and their dependents
Stripe iOS re-records snapshots only for modified modules and the modules that depend on them, walking Xcode target links including shared test utilities. SDK size reports check only affected SDKs and their dependents. When the selection is uncertain, both run in full.
In Stripe’s combined CI runs, framework tests went from about 22 to about 12 to 14 minutes, and snapshots and localization from about 15 to about 13 to 14 minutes, with the related build and job changes. Source
- Use it when
- A monorepo rebuilds and tests modules unrelated to a change.
- Before applying it
- Shared configuration, generated inputs and test utilities can affect untouched modules. Run the full set for anything the graph leaves unplaced, and keep full validation on the default branch.
Install and build only the workspace packages a job uses
Scope dependency installation and builds to the package or lane under test, including its actual dependencies.
Install or build only the workspace packages this step needs
Linear restricted its API test install to the API package and its dependencies, which cut pnpm install from 44-73 seconds to 16-18 seconds, then applied the same filter to API-adjacent jobs. In its public SDK, the release-preparation job builds only the package schema generation imports before updating the schema.
- Use it when
- A job’s dependency closure is smaller than the workspace.
- Before applying it
- Include transitive dependencies and generated prerequisites.
Build only the dependencies the test consumes
The memory test build replaced the entire stores wildcard with libsql, pg and upstash while retaining its other required packages. Another change moved a docs-server build out of the changed-test gate.
- Use it when
- A narrow test job pays for unrelated workspace builds.
- Before applying it
- Trace transitive dependencies and generated artifacts. Before moving a build out of a gate, confirm another required check still runs it.
Give each CI lane a defined package set
The app libraries ran everything in ci.yml in sequence, once per Node version. Shopify/shopify-app-js#2777 split that into a lint job and three package-family jobs that run in parallel; each builds its own dependency closure with pnpm filters and tests only its family. The pattern match is partial: the filters scope each lane, and the speedup comes from running the lanes in parallel.
The PR reports CI running in about 5 minutes against about 11 before. Source
- Use it when
- A monorepo runs one long sequential job that builds and tests every package on every run.
- Before applying it
- A new package has to be added to a lane, and each lane repeats install and setup.
Run CI once per pull request commit
Remove overlapping push and pull request triggers so each change receives one intended validation run.
Run CI once per change
Anthropic reports more CI jobs in a given day. As a StarSling pattern for GitHub Actions, removing duplicate push and pull_request runs and metadata-only triggers keeps each change to one run.
- Use it when
- Run history shows duplicate events for the same commit, or metadata events rerun checks that do not depend on that metadata.
- Before applying it
- Preview deployments, cache warming, release side effects and metadata-dependent checks keep the triggers they need.
Run each pull-request commit once
The Ruby extension's workflow ran on every push and every pull request, so each in-repo branch got two identical runs; for one commit they started three seconds apart. Shopify/vscode-shopify-ruby#808 limits push to main and adds a concurrency group whose cancellation applies only to pull_request events, so a main run that is already running is never cancelled, though a main run still waiting in the same group is replaced by a newer push.
Merged 2026-09-18; the workflow on the default branch is unchanged since. Source
- Use it when
- The same commit runs equivalent validation through both push and pull_request, and developers push several times in a row.
- Before applying it
- Check that the two runs test the same ref with the same permissions before treating them as duplicates.
Run push builds only on the default branch
target-determinator's presubmit workflow ran on every push and on every pull request, so a commit pushed to a pull request branch in the repository ran the test matrix twice. The change restricts the push trigger to main; pull requests still run through the pull_request event.
- Use it when
- A workflow triggers on both push and pull_request, and the push trigger covers every branch.
- Before applying it
- Branches without a pull request no longer get push builds; the workflow keeps workflow_dispatch for running them by hand.
Move selected heavy checks after merge
Keep a deliberately chosen pull request gate and schedule heavier validation after merge, with explicit ownership of the resulting risk.
Run the heavy matrix after merge
Codex kept Bazel build, test and clippy on pull requests, along with Cargo hygiene and its required aggregate, and moved the full Cargo matrix to pushes on main.
Full Cargo matrix moved to post-merge pushes on main Source
- Use it when
- A heavyweight matrix repeats a signal another required workflow already provides.
- Before applying it
- A regression only the deferred legs catch now surfaces after merge. Count those leg-unique failures first, and give the post-merge run an owner who is notified and can block a release.
Move heavyweight checks after merge, by decision
Shopify/tecnix#74 keeps the component tests and cheap flake checks on pull requests. VM tests, the remaining flake checks, the installer tests and flake regressions run on pushes to main, merge-queue runs and manual dispatches, and as optional jobs on pull requests. The required check names still report on every pull request. The pattern match is partial: the PR moves whole suites after merge with their tradeoff written down in the PR.
Merged 2026-10-01; the pull-request conditions are on the default branch. Source
- Use it when
- Heavy suites sit on the required path of every pull request, and the cheaper checks catch nearly every break first.
- Before applying it
- Failures in the moved suites show up on main after merge, so someone has to watch main and act on them.
Reject irrelevant events before allocating runners
Put event and change conditions before runner allocation so irrelevant comments or edits do not start expensive jobs.
Do not start a runner for irrelevant comments
Mastra gated the major-version job on the exact command in the new or previous comment body. This adapts the non-content-event recipe to issue_comment, preserving command removal as well as addition.
- Use it when
- Comment-triggered automation starts expensive checks for ordinary discussion.
- Before applying it
- Event matching and command parsing must agree. Preserve author authorization, edit/removal handling and fork trust, and keep permission checks separate from event filtering.
Keep metadata edits from restarting source validation
Codex Security skips its source checks on body-only PR edits, gives those events their own concurrency group, and renames the skipped aggregates. Title, base and combined edits still run full validation.
Body-only edits skip source validation in their own concurrency group Source
- Use it when
- Description-only edits rerun the same source checks or cancel an active run.
- Before applying it
- Title validators, retargets and metadata checks still need their events. A skipped run must never overwrite a failed, pending or missing required result.
Wait until a draft is ready for expensive checks
Avoid spending runner time on expensive validation until a draft pull request becomes ready for review.
Run expensive checks when a draft becomes ready
Checkout Kit skips its change-detection job, and so every platform job behind it, while a pull request is a draft. Adding ready_for_review to the trigger types starts a full run as soon as the draft is marked ready.
Merged 2026-06-16; both lines are on the default branch. Source
- Use it when
- Developers push to draft pull requests many times before asking for review.
- Before applying it
- Without ready_for_review, a draft promoted to ready keeps its skipped result on an untested commit.
Wait until a draft is ready for expensive checks
SubmitQueue added draft guards to every CI job and its required aggregate, then added ready_for_review to the workflow events. Pushes to main and merge-queue runs keep their validation.
Drafts skip the full suite, and it runs when the pull request is marked ready for review. Source
- Use it when
- Drafts receive frequent exploratory commits and your team agrees that expensive checks can wait.
- Before applying it
- Drafts get feedback later, so agree on it as a team policy. Test draft, ready, reopened, main and merge-queue events so every required check still reports.
After the first full agent scan, review only what changed
Run the full agent review once and keep its artifacts, then compare each new commit with the previous one and scan only the changes.
After the first full agent scan, review only what changed
Shopify's security harness, Dispatch, builds code partitions on its first run against an application and deploys hunting agents in parallel, one per partition. That run also produces reusable artifacts, such as documentation of the available APIs and data models, which are stored and kept up to date. On follow-up runs the pipeline compares the latest commit with the previous commit, then scans only the latest changes and updates the stored artifacts. Verifier agents run one after another, so tests that prove a finding do not collide on ports, databases and fixtures.
2026: A full application scan with publicly available frontier models costs between $50 and $300, and incremental diff scans cost roughly $5 to $50. Source
- Use it when
- A scheduled or per-commit agent scan covers a large repository, and most commits touch a small part of it.
- Before applying it
- Run the full scan again when the partitions or the stored artifacts change materially, so a diff scan always works from current context.
Package once, and build full installers after merge
Drop the build step that packaging repeats, and build the unpacked app pull requests test.
Package once, and build full installers after merge
The desktop job ran a Build step and then packaging, which runs the same electron-vite build again. The PR removed the standalone Build step. Pull requests now package the unpacked app that the end-to-end journeys consume; pushes to main still build and inspect the full installers.
- Use it when
- A job builds explicitly before a packaging or test command that builds again, or pull requests build release artifacts that no test reads.
- Before applying it
- Read the wrapper scripts before removing a build step. The full installers still need a post-merge run that someone watches.
Reduce the setup floor before increasing shard count
Each extra shard repeats checkout, dependencies and container setup.
Reduce the setup floor before increasing shard count
Each extra shard repeats checkout, dependencies and container setup. Before its setup work, Linear’s test job used 4 shards and spent 8.3 minutes on setup; afterward it ran 8 shards with 7.5 minutes of setup.
- Use it when
- More shards would repeat substantial fixed setup work.
- Before applying it
- Remeasure per-shard setup before changing shard count.
Company Playbooks
- Read Anthropic’s playbookRun only affected tests
- Read DoorDash’s playbookRun build and test in parallel, sharded across CI agents
- Read Linear’s playbookBenchmark the same job on faster hardware
- Read Mastra’s playbookStarSling runners for the slowest test workflow
- Read OpenAI’s playbookRank CI fixes by the minutes they save
- Read Ramp’s playbookMove a CPU-bound test suite to a faster CPU
- Read Shopify’s playbookLet an agent iterate against one CI metric
- Read Spotify’s playbookBuild arm64 on native arm64 runners
- Read Stripe’s playbookRun only the tests a change can reach
- Read Uber’s playbookBuild the merge-queue speculations most likely to be needed