--- title: "After the first full agent scan, review only what changed" description: "Run the full agent review once and keep its artifacts, then compare each new commit with the previous one and scan only the changes." url: https://starsling.dev/ci-playbooks/techniques/scan-only-the-diff canonicalUrl: https://starsling.dev/ci-playbooks/techniques/scan-only-the-diff --- # After the first full agent scan, review only what changed Run the full agent review once and keep its artifacts, then compare each new commit with the previous one and scan only the changes. ## How to choose an approach This technique currently has one documented company example. Validate its prerequisites, tradeoffs and original CI environment against your repository before adapting it. - **[Shopify](https://starsling.dev/ci-playbooks/shopify#scan-only-the-diff)**: Use it when A scheduled or per-commit agent scan covers a large repository, and most commits touch a small part of it. Check before adopting: Run the full scan again when the partitions or the stored artifacts change materially, so a diff scan always works from current context. ## Apply this technique ````text Evaluate and apply this CI technique to my repository: After the first full agent scan, review only what changed. Run the full agent review once and keep its artifacts, then compare each new commit with the previous one and scan only the changes. Read the technique and its source-backed company examples: https://starsling.dev/ci-playbooks/techniques/scan-only-the-diff.md Inspect this repository’s CI workflows, build and test commands, dependencies, and recent run timings. Identify where this technique applies and explain the expected benefit using this repository’s evidence. Measure a baseline from recent comparable runs: elapsed time, queue time, runner minutes and failures where available. If data is unavailable, say what cannot be measured. Compare relevant company approaches, checking prerequisites, platform differences and tradeoffs. Rank applicable options by evidence, fit and measurable opportunity; do not present company results as predictions for this repo. Recommend the smallest safe change, show what checks and fallback paths remain, and ASK ME WHICH CHANGE TO APPROVE BEFORE EDITING ANY FILES. Only after approval, work on a new feature branch, make one reversible change, and open a reviewable pull request. Never merge or push to a default or protected branch. Keep required checks, test coverage, deployment and release behavior intact. Validate the change and compare equivalent runs where possible; report limitations and what should be monitored after merge. ```` ## How Shopify used this technique Read what Shopify changed and follow the original sources. ### Shopify: After the first full agent scan, review only what changed Shopify's security harness, Dispatch, builds code partitions on its first run against an application and deploys hunting agents in parallel, one per partition. That run also produces reusable artifacts, such as documentation of the available APIs and data models, which are stored and kept up to date. On follow-up runs the pipeline compares the latest commit with the previous commit, then scans only the latest changes and updates the stored artifacts. Verifier agents run one after another, so tests that prove a finding do not collide on ports, databases and fixtures. Evidence: Company-reported result or observation 2026: A full application scan with publicly available frontier models costs between $50 and $300, and incremental diff scans cost roughly $5 to $50. ([Source](https://shopify.engineering/building-an-agentic-harness-that-outlasts-the-model)) Use it when: A scheduled or per-commit agent scan covers a large repository, and most commits touch a small part of it. Before applying it: Run the full scan again when the partitions or the stored artifacts change materially, so a diff scan always works from current context. [Read more in the Shopify playbook](https://starsling.dev/ci-playbooks/shopify#scan-only-the-diff) [Primary source](https://shopify.engineering/building-an-agentic-harness-that-outlasts-the-model) ## CI optimization is continuous work. Apply this technique yourself, or use StarSling Runners for faster GitHub Actions and optional agent-driven optimization PRs, which teams on eligible paid plans can turn on. [Get self-improving CI](https://starsling.dev/products/runners) ## Related techniques ### [Run only the tests a change can affect](https://starsling.dev/ci-playbooks/techniques/select-affected-tests) Use dependency or observed-file relationships to select affected tests, with a full-suite fallback for changes the selector cannot classify. 5 implementations · 4 companies (Anthropic, Mastra, Ramp, Stripe) ### [Install and build only the workspace packages a job uses](https://starsling.dev/ci-playbooks/techniques/scope-workspace-builds) Scope dependency installation and builds to the package or lane under test, including its actual dependencies. 3 implementations · 3 companies (Linear, Mastra, Shopify) ### [Move selected heavy checks after merge](https://starsling.dev/ci-playbooks/techniques/defer-heavy-validation) Keep a deliberately chosen pull request gate and schedule heavier validation after merge, with explicit ownership of the resulting risk. 2 implementations · 2 companies (OpenAI, Shopify) ## Company Playbooks ### [Anthropic](https://starsling.dev/ci-playbooks/anthropic) Run only affected tests ### [DoorDash](https://starsling.dev/ci-playbooks/doordash) Run build and test in parallel, sharded across CI agents ### [Linear](https://starsling.dev/ci-playbooks/linear) Benchmark the same job on faster hardware ### [Mastra](https://starsling.dev/ci-playbooks/mastra) StarSling runners for the slowest test workflow ### [OpenAI](https://starsling.dev/ci-playbooks/openai) Rank CI fixes by the minutes they save ### [Ramp](https://starsling.dev/ci-playbooks/ramp) Move a CPU-bound test suite to a faster CPU ### [Shopify](https://starsling.dev/ci-playbooks/shopify) Let an agent iterate against one CI metric ### [Spotify](https://starsling.dev/ci-playbooks/spotify) Build arm64 on native arm64 runners ### [Stripe](https://starsling.dev/ci-playbooks/stripe) Run only the tests a change can reach ### [Uber](https://starsling.dev/ci-playbooks/uber) Build the merge-queue speculations most likely to be needed ## Related guides Implementation guides for this technique. ### [Build and test only what changed](https://starsling.dev/best-practices/github-actions/build-only-affected) Scope work with your dependency graph and keep a full-run fallback. ### [Cancel superseded CI runs](https://starsling.dev/best-practices/github-actions/cancel-superseded-runs) Use concurrency groups to keep PR verification focused on the latest commit. ### [Cut CI queue time](https://starsling.dev/best-practices/github-actions/cut-queue-time) Measure waiting time and identify concurrency or capacity constraints. ### [Filter workflows by changed files](https://starsling.dev/best-practices/github-actions/path-filter-workflows) Distinguish workflow triggers from job-level conditions for required checks.