After the first full agent scan, review only what changed
Run the full agent review once and keep its artifacts, then compare each new commit with the previous one and scan only the changes.
How Shopify used this technique
Read what Shopify changed and follow the original sources.
After the first full agent scan, review only what changed
Shopify's security harness, Dispatch, builds code partitions on its first run against an application and deploys hunting agents in parallel, one per partition. That run also produces reusable artifacts, such as documentation of the available APIs and data models, which are stored and kept up to date. On follow-up runs the pipeline compares the latest commit with the previous commit, then scans only the latest changes and updates the stored artifacts. Verifier agents run one after another, so tests that prove a finding do not collide on ports, databases and fixtures.
2026: A full application scan with publicly available frontier models costs between $50 and $300, and incremental diff scans cost roughly $5 to $50. Source
- Use it when
- A scheduled or per-commit agent scan covers a large repository, and most commits touch a small part of it.
- Before applying it
- Run the full scan again when the partitions or the stored artifacts change materially, so a diff scan always works from current context.
Company Playbooks
- Read Anthropic’s playbookRun only affected tests
- Read DoorDash’s playbookRun build and test in parallel, sharded across CI agents
- Read Linear’s playbookBenchmark the same job on faster hardware
- Read Mastra’s playbookStarSling runners for the slowest test workflow
- Read OpenAI’s playbookRank CI fixes by the minutes they save
- Read Ramp’s playbookMove a CPU-bound test suite to a faster CPU
- Read Shopify’s playbookLet an agent iterate against one CI metric
- Read Spotify’s playbookBuild arm64 on native arm64 runners
- Read Stripe’s playbookRun only the tests a change can reach
- Read Uber’s playbookBuild the merge-queue speculations most likely to be needed