--- title: "Require verification before an agent hands over a pull request" description: "Have the agent complete the relevant build, test, and review checks before presenting its change for approval." url: https://starsling.dev/ci-playbooks/techniques/verify-before-agent-pr canonicalUrl: https://starsling.dev/ci-playbooks/techniques/verify-before-agent-pr --- # Require verification before an agent hands over a pull request Have the agent complete the relevant build, test, and review checks before presenting its change for approval. ## How to choose an approach Compare the examples by repository setup, prerequisites, and risks before adopting one. Each company implemented this in its own environment; measure each change against your own baseline. - **[DoorDash](https://starsling.dev/ci-playbooks/doordash#gate-agent-pull-requests)**: Use it when Agent pull requests go red in CI after they open, or parallel agents share one checkout. Check before adopting: Local gates must match the required checks exactly; a gate that runs only in the agent's sandbox can drift from CI. - **[OpenAI](https://starsling.dev/ci-playbooks/openai#verification-skill)**: Use it when Agents report a change done before running the commands your CI runs. Check before adopting: Keep the skill's command list identical to the workflow's, and update both in one change. - **[Ramp](https://starsling.dev/ci-playbooks/ramp#agent-opens-reviewed-pr)**: Use it when Changes can be described in a message, the development environment runs in the cloud, and reviewers want to see the result before they read the diff. Check before adopting: Every agent pull request still goes through the same review and required checks as any other, so the review queue is where the volume lands. Adapted for GitHub Actions; compare the original system with yours. - **[Spotify](https://starsling.dev/ci-playbooks/spotify#agent-verification-loop)**: Use it when Agents or bots open pull requests across many repositories, and reviewers spend time on changes that stray from the task. Check before adopting: The judge adds a model call to every session. Give it the diff and the original prompt, and run it after the deterministic verifiers pass. Adapted for GitHub Actions; compare the original system with yours. ## Apply this technique ````text Evaluate and apply this CI technique to my repository: Require verification before an agent hands over a pull request. Have the agent complete the relevant build, test, and review checks before presenting its change for approval. Read the technique and its source-backed company examples: https://starsling.dev/ci-playbooks/techniques/verify-before-agent-pr.md Inspect this repository’s CI workflows, build and test commands, dependencies, and recent run timings. Identify where this technique applies and explain the expected benefit using this repository’s evidence. Measure a baseline from recent comparable runs: elapsed time, queue time, runner minutes and failures where available. If data is unavailable, say what cannot be measured. Compare relevant company approaches, checking prerequisites, platform differences and tradeoffs. Rank applicable options by evidence, fit and measurable opportunity; do not present company results as predictions for this repo. Recommend the smallest safe change, show what checks and fallback paths remain, and ASK ME WHICH CHANGE TO APPROVE BEFORE EDITING ANY FILES. Only after approval, work on a new feature branch, make one reversible change, and open a reviewable pull request. Never merge or push to a default or protected branch. Keep required checks, test coverage, deployment and release behavior intact. Validate the change and compare equivalent runs where possible; report limitations and what should be monitored after merge. ```` ## How companies used this technique Read what each company changed and follow the original sources. ### DoorDash: Let an agent open a pull request only after build, tests, coverage and lint pass DoorDash's feature-flag cleanup system turns a stale-flag ticket into a merge-ready pull request. Removal agents run in parallel, up to four per repository, each in its own git worktree, and Gradle runs with --no-daemon so parallel worktrees share no state. Before an agent may open a pull request it must confirm the build passes, the tests pass, patch coverage is at least 95% on changed lines and the Detekt linter is clean, retrying until each gate passes. On the 50 most recent stale flags, the system produced usable pull requests for 45, at an average of $4.79 and 13.8 minutes per flag. Evidence: Company-reported result or observation the system produced usable PRs for 45 of them at an average cost of $4.79 and 13.8 minutes per flag ([Source](https://careersatdoordash.com/blog/automating-feature-flag-cleanup-at-scale-with-a-multi-agent-llm-system/)) Use it when: Agent pull requests go red in CI after they open, or parallel agents share one checkout. Before applying it: Local gates must match the required checks exactly; a gate that runs only in the agent's sandbox can drift from CI. [Read more in the DoorDash playbook](https://starsling.dev/ci-playbooks/doordash#gate-agent-pull-requests) [Primary source](https://careersatdoordash.com/blog/automating-feature-flag-cleanup-at-scale-with-a-multi-agent-llm-system/) ### OpenAI: Make the verification stack a skill the agent must pass In the Agents SDK repositories, a code-change-verification skill runs the formatting, lint, type-check and test stack whenever runtime code, tests, examples or build and test behavior changes, and AGENTS.md says the work is not complete until it passes. Docs-only changes skip it. Separately, the JavaScript repository's changeset-validation skill keeps its validation rules in a shared prompt, so local runs and GitHub Actions use the same logic. Evidence: Company-reported result or observation The skill encodes the repository’s definition of “verified,” and AGENTS.md makes that definition enforceable. ([Source](https://developers.openai.com/blog/skills-agents-sdk)) Use it when: Agents report a change done before running the commands your CI runs. Before applying it: Keep the skill's command list identical to the workflow's, and update both in one change. [Read more in the OpenAI playbook](https://starsling.dev/ci-playbooks/openai#verification-skill) [Primary source](https://developers.openai.com/blog/skills-agents-sdk) ### Ramp: Have the coding agent hand over a visually validated pull request **Translated to GitHub Actions** - What Ramp built: Ramp's Inspect takes a task from Slack, GitHub, Linear or the web and turns it into a visually validated and reviewed pull request. - Your GitHub Actions version: A coding agent that gathers context, fixes the issue, attaches visual proof and prepares the pull request for review. Ramp's Inspect takes a task from Slack, GitHub, Linear or the web and converts it into a visually validated and fully reviewed pull request, using the team's development environment in the cloud. For a mobile bug, a screenshot and a one-line message are enough: Inspect gathers context, pinpoints the issue, fixes it, provides visual proof and prepares the pull request for review. Ramp reports 99% of its engineering, design and product teams used Inspect in the past month, and that Inspect accounts for 74% of merged PRs. Evidence: Company-reported result or observation 2026: Inspect accounts for 74% of Ramp's merged PRs, and 99% of its engineering, design, and product teams used it in the past month. ([Source](https://ramp.com/blog/introducing-inspect)) Use it when: Changes can be described in a message, the development environment runs in the cloud, and reviewers want to see the result before they read the diff. Before applying it: Every agent pull request still goes through the same review and required checks as any other, so the review queue is where the volume lands. GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation. [Read more in the Ramp playbook](https://starsling.dev/ci-playbooks/ramp#agent-opens-reviewed-pr) [Primary source](https://ramp.com/blog/introducing-inspect) ### Spotify: Run verifiers and a judge before an agent opens a pull request **Translated to GitHub Actions** - What Spotify built: Spotify's background coding agent runs independent verifiers and an LLM judge before it opens a pull request. - Your GitHub Actions version: Verifiers that activate from the repository's contents, with a red verifier stopping the pull request from opening, then a judge that reads the diff against the original prompt. Spotify's background coding agent runs one or more independent verifiers before it opens a pull request. A verifier activates from the repository's contents, such as a Maven verifier when a pom.xml sits at the root, and returns a short error summary so the agent keeps its context window for the change. In Claude Code the checks run from the stop hook, and a red verifier stops the pull request from opening. An LLM judge then reads the diff and the original prompt and flags changes that go beyond the instructions, such as refactoring code or disabling flaky tests. Evidence: Company-reported result or observation Spotify reports that out of thousands of agent sessions, the judge vetoes about a quarter of them, and the agent is able to course correct half the time. ([Source](https://engineering.atspotify.com/2025/12/feedback-loops-background-coding-agents-part-3)) Use it when: Agents or bots open pull requests across many repositories, and reviewers spend time on changes that stray from the task. Before applying it: The judge adds a model call to every session. Give it the diff and the original prompt, and run it after the deterministic verifiers pass. GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation. [Read more in the Spotify playbook](https://starsling.dev/ci-playbooks/spotify#agent-verification-loop) [Primary source](https://engineering.atspotify.com/2025/12/feedback-loops-background-coding-agents-part-3) ## CI optimization is continuous work. Apply this technique yourself, or use StarSling Runners for faster GitHub Actions and optional agent-driven optimization PRs, which teams on eligible paid plans can turn on. [Get self-improving CI](https://starsling.dev/products/runners) ## Company Playbooks ### [Anthropic](https://starsling.dev/ci-playbooks/anthropic) Run only affected tests ### [DoorDash](https://starsling.dev/ci-playbooks/doordash) Run build and test in parallel, sharded across CI agents ### [Linear](https://starsling.dev/ci-playbooks/linear) Benchmark the same job on faster hardware ### [Mastra](https://starsling.dev/ci-playbooks/mastra) StarSling runners for the slowest test workflow ### [OpenAI](https://starsling.dev/ci-playbooks/openai) Rank CI fixes by the minutes they save ### [Ramp](https://starsling.dev/ci-playbooks/ramp) Move a CPU-bound test suite to a faster CPU ### [Shopify](https://starsling.dev/ci-playbooks/shopify) Let an agent iterate against one CI metric ### [Spotify](https://starsling.dev/ci-playbooks/spotify) Build arm64 on native arm64 runners ### [Stripe](https://starsling.dev/ci-playbooks/stripe) Run only the tests a change can reach ### [Uber](https://starsling.dev/ci-playbooks/uber) Build the merge-queue speculations most likely to be needed