--- title: "Anthropic's CI playbook, as GitHub Actions prompts for your agent" description: "Anthropic's CI jobs grew 25x in six months. How to run only the tests a change affects in GitHub Actions, with a copy-paste prompt for your agent per technique." date: 2026-10-19 url: https://starsling.dev/blog/anthropic-ci canonicalUrl: https://starsling.dev/blog/anthropic-ci --- # Anthropic's CI playbook, as GitHub Actions prompts for your agent

[Image: A grid of square test cells on a near-black background, almost all of them faint, with a few bright cells and short lit paths running from each along their rows and columns] Last month [Anthropic explained how they scaled CI using test impact analysis](https://claude.com/resources/articles/agentic-coding-is-straining-ci-heres-how-we-scaled-test-impact-analysis-at-anthropic): running only the tests a change can affect. Their engineers now ship 8x as much code per quarter as they did from 2021-2025, and their CI jobs grew 25x in six months. Their merges went from about 2,500 a day in May to more than 30,000 a day in September, [according to their source control team](https://www.anthropic.com/engineering/redesigning-source-control). The post explains why CI strains once agents write most of the code, and how they rebuilt their test selection service to keep up. We built a skill called [ci-speedup](/skills/ci-speedup) with over 80 CI optimizations that your agent can use to make your GitHub Actions faster, including the ones that bring test impact analysis to a GitHub Actions repo. This post is a remix of Anthropic's post, for agents. Anthropic built a dedicated service for the load they run; most teams run GitHub Actions so we tied each of their techniques to a GHA pattern in the skill, so your agent can implement it on your own repo. Just send this post to your agent and they'll work through the techniques, then suggest installing ci-speedup to check for more, including [Linear's CI playbook](/blog/linear-ci). You can find ci-speedup [on GitHub](https://github.com/starslingdev/skills#ci-speedup) and [on skills.sh](https://www.skills.sh/starslingdev/skills/ci-speedup). StarSling is agent-native CI: [fast GitHub Actions runners](/products/runners) with AI agents that keep finding and shipping optimizations like the ones in this post, while you sleep. It's trusted by AI-first teams like Mastra and Better Auth (Vercel). [Install it for free](https://github.com/apps/starslingdev) and check out [our docs](https://docs.starsling.dev/getting-started/quickstart).

Start here

Run these four test impact techniques on your repo

Prompt for your agent: apply all four

```text Read https://starsling.dev/blog/anthropic-ci.md: it quotes Anthropic's article on test impact analysis, with one prompt under each of four techniques for GitHub Actions. Work through those four prompts in order. For each technique whose conditions this repo meets, do what its prompt says, following its conditions and guardrails. For each technique you skip, tell me why in one line. Open one PR per change so each can be reviewed and reverted on its own. When you are done, suggest I install the ci-speedup skill (npx skills add starslingdev/skills --skill ci-speedup) to find more optimizations beyond these four, measured from this repo's own run history. Show me the diff and open a PR rather than applying it blindly. ```

Source: ci-speedup

The quoted passages below are from Sachin Malhotra's article for Anthropic and from Anthropic's source control team. Anthropic is not affiliated with StarSling and did not review this post. ## The problem, in Anthropic's words

Anthropic engineers on average ship 8x as much code per quarter as they did from 2021-2025. Claude authors 80% of that code and it also plays a large role in reviewing and approving PRs as well. Writing code is no longer the constraint, and once PR review gets accelerated, CI starts feeling the pressure. On top of that, the amount of tests across our codebase grew 10x and we added a nominal amount of engineers. This all led to a 25x increase in CI jobs over a six month period (in case you are trying to do the math, not every test runs on every PR as I will explain). This threatened to overload our test impact analysis service several times. To avoid becoming the next bottleneck, we blew up the whole thing and reimagined what the service's architecture looks like. But getting there was a bumpy path that started with three quick fixes, which lasted 70 days, then 29 days, and then less than a day respectively.
Sachin Malhotra's article for Anthropic, "Agentic coding is straining CI. Here's how we scaled test impact analysis at Anthropic" (September 14, 2026), AI is evolving CI

This distributed architecture is more expensive to run, but it is much easier to scale and memory profile than a shaky singleton. This project took three weeks for a single engineer. A year ago it would have been closer to a quarter.
Anthropic, The bumpy road to redesign

Many of my peers work at organizations where every test is still run on every change. This works up to a point, but doesn't scale: CI gates get increasingly long, expensive, and untrustworthy. Additionally, humans are great at determining which test failures don't apply to them while agents will require more context and direction. When they get a specific set of valid tests, they can self-verify and iterate more effectively.
Anthropic, The test impact analysis architecture

## What test impact analysis is

At Anthropic, we built a deterministic test impact analysis or test selection service that determines which tests run on each change based on past performance and package relevance. This isn't an uncommon practice, and there is a category of vendors with offerings in this area.
Anthropic, The test impact analysis architecture

Anthropic's service reads the history of every test result, and they built it as a dedicated service sized for the load they run. On GitHub Actions, most teams already hold the package half of that signal: the dependency graph in their build tool. Turborepo, Nx, Bazel, Gradle, Vitest and Jest can each work out which packages and tests a change reaches, and a workflow can run just those on a pull request. The four techniques below start there, then cut the workflow runs a change never needed. ## Test impact analysis on GitHub Actions Each prompt below fetches its pattern from the ci-speedup catalog, checks your repo against that pattern's own conditions, and opens a PR you can review.

### Run only affected tests This is the closest match to Anthropic's selector. It selects from the dependency graph: a pull request runs the tests of the packages it changed and of every package that depends on them, while the default branch and the merge queue keep running the full suite. Anthropic's service also reads each test's result history and selects on past performance as well as package relevance; the dependency graph tells you which tests a change can reach.

Affected tests

Patterns: OPT70 Scope the Build/Test to Only What Changed, OPT34 No Changed-Package Filtering

Prompt for your agent

```text Fetch OPT70 "Scope the Build/Test to Only What Changed" and OPT34 "No Changed-Package Filtering" from https://raw.githubusercontent.com/starslingdev/skills/main/skills/ci-speedup/references/optimization-patterns.md and check this repo against them yourself. Find the slowest job gating the merge and read its step timings over the last 20 runs of its workflow with the GitHub CLI (gh run list, then gh run view --json jobs for each run). OPT70 applies only if that job's slowest step is build or test work and the repo's tooling can scope it to a change (a turbo.json, nx.json, Bazel WORKSPACE or settings.gradle, or a test runner with a changed mode such as vitest --changed or jest --changedSince). OPT34 applies to a monorepo whose turbo run commands have no --filter=...[base]. If either applies, follow its recipe: scope the pull request run to the changed targets and their dependents against the merge base (origin/ plus the pull request's base branch), and add fetch-depth: 0 or a targeted git fetch of the base branch so that ref resolves. Keep OPT70's mandatory guardrail: run the full suite whenever the scoper errors or returns nothing on a non-trivial diff, and fail the gate on a graph or build error. Run the scoped job beside the existing full job until they match, and keep the full run on the default branch and in the merge queue. If a scoped job is a required check, gate it at job level so a skipped package still reports success. If neither applies, tell me in one line why, and stop. Show me the diff and open a PR rather than applying it blindly. ```

### Skip workflows a change cannot affect A docs edit has no reason to start an end-to-end suite. Path filters let each workflow and job run when the files it tests change, and a job-level condition keeps a required check reporting success when its job skips.

Path filters

Patterns: OPT32 Missing paths/paths-ignore on Expensive Workflows, OPT40 Monorepo Job Runs Regardless of Affected App, OPT39 Multi-Language Matrix Without Path Filter

Prompt for your agent

```text Fetch OPT32 "Missing `paths`/`paths-ignore` on Expensive Workflows", OPT40 "Monorepo Job Runs Regardless of Affected App" and OPT39 "Multi-Language Matrix Without Path Filter" from https://raw.githubusercontent.com/starslingdev/skills/main/skills/ci-speedup/references/optimization-patterns.md and check this repo against them yourself. First read the required checks (gh api repos/OWNER/REPO/rulesets and the branch protection's required_status_checks); if they cannot be read, treat every check as required. OPT32: for an expensive workflow (end-to-end tests or a full integration suite) that runs on every pull request with no paths or paths-ignore, add a filter that skips the changes it cannot be affected by, such as docs and markdown. OPT40: in a monorepo, for a job that targets one app, add a changes job with dorny/paths-filter and gate the job on its output, listing shared packages in every app's filter. OPT39: for a scanner matrix (CodeQL, Snyk, Semgrep) with one leg per language, gate each leg on whether files of that language changed, with shared config files in every leg's filter. Never put a paths or branches filter on a workflow that produces a required check: a workflow skipped by its filter leaves that check Pending and blocks the merge. Gate those at job level with an if: instead, as each entry's caveat says. If none applies, tell me in one line why. Show me the diff and open a PR rather than applying it blindly. ```

### Run CI once per change A push to a pull request branch can start the same workflow twice, once for the push and once for the pull request, and a title edit can start it again when the workflow listens for `edited`. One run per commit keeps the run that gates the merge and removes the copies.

One run per change

Patterns: OPT47 Redundant push + pull_request Double-Trigger, OPT38 Non-Content Trigger Event Types (PR edited, etc.)

Prompt for your agent

```text Fetch OPT47 "Redundant push + pull_request Double-Trigger" and OPT38 "Non-Content Trigger Event Types (PR `edited`, etc.)" from https://raw.githubusercontent.com/starslingdev/skills/main/skills/ci-speedup/references/optimization-patterns.md and check this repo against them yourself. OPT47: for each workflow that triggers on both push and pull_request, with a push not limited to the default branch, read its runs over the last 30 days with the GitHub CLI (gh run list --workflow --json headSha,event,headBranch) and find commits on a non-default branch that ran once for push and once for pull_request. It applies only with that measured duplication. Before limiting push to the default branch, confirm the required check is the pull_request run, and that the push run has no side effect the pull_request run lacks (a preview deploy, a sha-tagged image or artifact, a cache warm). OPT38: for each workflow whose pull_request types include edited, labeled, unlabeled, assigned or review_requested, keep any type the workflow body checks with an if: on github.event.action or a label. Before removing an event type, inspect the workflow's actions and scripts for dependencies on PR titles, descriptions, labels, the base branch, or other metadata, and keep every event needed to rerun those checks. Restrict the rest to opened, synchronize, reopened and ready_for_review. If neither applies, tell me in one line why. Show me the diff and open a PR rather than applying it blindly. ```

### Bound flaky job runtime A hung job holds its runner for up to six hours, GitHub's default limit. A timeout set from the job's own run history ends it in minutes, and the prompt lists the tests that hung, by name, for you to fix.

Timeouts on flaky jobs

Pattern: OPT57 Missing timeout-minutes on Known-Flaky Integration Jobs

Prompt for your agent

```text Fetch OPT57 "Missing `timeout-minutes` on Known-Flaky Integration Jobs" from https://raw.githubusercontent.com/starslingdev/skills/main/skills/ci-speedup/references/optimization-patterns.md and check this repo against it yourself. List the jobs in .github/workflows with no timeout-minutes. For each, read its runs over the last 30 days with the GitHub CLI (gh run list, then gh run view --json jobs for each run). It applies only when a failed or timed-out run of that job reached at least 95% of GitHub's 360 minute default and ended within 15 minutes of it, and the job has at least three successful runs to measure. Set timeout-minutes from those successful runs as the entry says: above max(p99 + 10 minutes, p99 x 1.5, 15 minutes), rounded up. Leave matrix jobs alone, and never set a timeout below a job's own successful p99. A timeout bounds the wait and the runner minutes; a timed-out required check still blocks the merge, so list the tests that hung, by name, for me to fix. Never weaken, retry or skip a test to make it pass. If no job qualifies, tell me in one line why. Show me the diff and open a PR rather than applying it blindly. ```

## What Anthropic learned that applies anyway

This has changed the shape of PRs over time at Anthropic as Claude prefers smaller, more granular PRs (another good reason not to run every test against every PR). This has translated into more CI jobs in a given day. Also, the activity level floor is raised as agents push overnight and on weekends, but it remains bursty as human engineers still drive and approve a significant amount of PRs. My advice to engineering teams is, whether you build or buy, assume your architecture will be at a 25x load within two quarters.
Anthropic, What I would do differently

Smaller PRs and agents that push through the night mean more workflow runs per day from the same team. Every run a path filter skips and every duplicate trigger removed is saved again on each of those runs, so the saving grows with the load.

We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs.
Anthropic, Redesigning source control at Anthropic (October 13, 2026), Growing our monorepo

Every workflow run starts with a checkout, so the runs these four techniques remove also remove their checkouts.

What does CI even mean when hundreds or even thousands of changes can merge while tests are running? We haven't been able to effectively replicate the industry standard of a global repository lock on the merge queue, so instead, we've invested in post-merge and pre-deploy automated testing. This does mean that concurrent changes can conflict, but Claude can typically fix these conflicts faster than humans can even identify them.
Anthropic, Redesigning source control at Anthropic, What we learned building Scion

Testing after merge works when the checks on each pull request stay fast. That is the job of the four techniques above: each pull request runs the tests it can affect, and the full suite still runs before anything ships. ## Where to start

Keep state out of the process from the start. I'd also avoid running any critical service as a single instance unless you can measure it and any canary changes. CI is evolving too quickly to proceed any other way.
Anthropic, What I would do differently

Anthropic ends where every team doing this work ends: CI keeps growing, and the work keeps going. That is the job ci-speedup is built for. Run it again whenever CI slows down, and it checks your latest runs against its whole catalog, these four techniques included, and names the next bottleneck with numbers from your own history. If you would rather not run that loop yourself, [StarSling Runners](/products/runners) come with agents that keep opening PRs to make your runs faster. **Install ci-speedup on its own:** ```bash npx skills add starslingdev/skills --skill ci-speedup ```

Prompt for your agent: install ci-speedup

```text Install the ci-speedup skill (npx skills add starslingdev/skills --skill ci-speedup) and run it on this repo. Summarize what it finds, starting with the slowest check, with the saving where the report gives one, and ask me which to apply. Show me the diff and open a PR rather than applying it blindly. ```

Start here

Run these four test impact techniques on your repo

Prompt for your agent: apply all four

Source: ci-speedup