Constrain agent migrations with repeatable validation

Give migration agents a bounded recipe and validation contract so changes can be checked and reviewed in manageable units.

View in Markdown

Prompt

Evaluate and apply this CI technique to my repository: Constrain agent migrations with repeatable validation.

Give migration agents a bounded recipe and validation contract so changes can be checked and reviewed in manageable units.

Read the technique and its source-backed company examples: https://starsling.dev/ci-playbooks/techniques/agent-migration-recipes.md

Inspect this repository’s CI workflows, build and test commands, dependencies, and recent run timings. Identify where this technique applies and explain the expected benefit using this repository’s evidence.
Measure a baseline from recent comparable runs: elapsed time, queue time, runner minutes and failures where available. If data is unavailable, say what cannot be measured.
Compare relevant company approaches, checking prerequisites, platform differences and tradeoffs. Rank applicable options by evidence, fit and measurable opportunity; do not present company results as predictions for this repo.
Recommend the smallest safe change, show what checks and fallback paths remain, and ASK ME WHICH CHANGE TO APPROVE BEFORE EDITING ANY FILES.
Only after approval, work on a new feature branch, make one reversible change, and open a reviewable pull request. Never merge or push to a default or protected branch.
Keep required checks, test coverage, deployment and release behavior intact. Validate the change and compare equivalent runs where possible; report limitations and what should be monitored after merge.

View all playbooks

How companies used this technique

Read what each company changed and follow the original sources.

Find every consumer from lineage data and open one agent pull request per repository

Translated to GitHub Actions

What Spotify built: Spotify used Backstage's endpoint lineage, Codesearch and Fleetshift to roll out 240 Honk migration pull requests across its BigQuery Runner and dbt pipelines.
Your GitHub Actions version: List every consumer from lineage data, then open one agent pull request per repository.

Spotify had to move about 1,800 downstream data pipelines off two deprecated datasets. Backstage's endpoint lineage listed each dataset's downstream consumers, and Codesearch queries marked the target repositories as in scope. Fleetshift then orchestrated the change on the BigQuery Runner and dbt pipelines, with Honk making the edit in each repository from a context file whose field mappings were written out in tables. Where a change needed a judgement call, Honk left the field unchanged and added a comment linking to the human migration guide for the reviewing team.

Spotify estimated about 10 engineering weeks of effort to migrate the pipelines manually, and rolled out 240 automated migration pull requests. Source

Use it when: A deprecation or upgrade touches many repositories, and a dependency or lineage graph can list who is affected.
Before applying it: Spell out every mapping in the context file, and have the agent hand judgement calls to the reviewing team.
GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation.

Upgrade the JDK in bounded pull requests computed from the build graph

Translated to GitHub Actions

What Stripe built: Stripe's AutoJDK analyzes the live build graph and generates JDK upgrades in bounded partitions, with minions fixing mechanical CI errors.
Your GitHub Actions version: Compute eligible targets from the build graph and open upgrades in bounded, rerunnable pull requests.

Stripe's AutoJDK analyzes the live build graph to decide which Java and Scala targets are eligible, weighing reverse dependencies, runtime JDK compatibility caps, dependency ordering and framework-specific guards. It generates target_jdk upgrades in bounded partitions as automated pull requests that are safe to rerun, and tracks adoption and blockers over time. Minions fix mechanical CI errors from an upgrade in sandboxes with fresh git checkouts, and an agentic skill runs upgrades across all partitions, monitors and fixes CI, and cherry-picks the commits into larger pull requests.

Since going live 3 months ago, Stripe merged 160+ PRs using the automated upgrader. Source

Use it when: A language or toolchain upgrade touches thousands of build targets, and large upgrade waves overwhelm reviewers or produce merge conflicts.
Before applying it: Keep the version policy in the platform so the upgrader has one place to change, and keep CI checks that enforce dependency ordering between targets.
GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation.

Migrate a test framework with a recipe that keeps only passing targets

Translated to GitHub Actions

What Uber built: Uber faced over 600,000 JUnit 4 tests and migrated over 75,000 test classes to JUnit 5 with OpenRewrite recipes applied per Bazel test target, with Shepherd generating the diffs.
Your GitHub Actions version: Apply a migration recipe per test target, run that target's tests, and keep the change only when they pass.

Uber faced over 600,000 JUnit 4 tests and migrated over 75,000 test classes to JUnit 5 with OpenRewrite recipes applied per Bazel test target. A precondition recipe limits each run to files whose test runners, rules and base classes the recipe supports. A script applies the patch, updates the build dependencies and runs the target's tests, and keeps the change when they pass. Shepherd then generates a diff for each migrated target, and each rollout round adds recipes for the patterns the previous round turned up.

Uber reports migrating over 75,000 test classes to JUnit 5, modifying over 1.25 million lines of code within 4 months, with over 5,000 diffs generated. Source

Use it when: A framework or API migration spans thousands of test targets, and a manual migration would take developer hours across many teams.
Before applying it: Start with the utilities used most often. Revert any target whose tests change outcome, and rerun after adding a recipe for the pattern behind it.
GitHub Actions adaptation: Compare the source system with your own workflow before applying this translation.

How companies used this technique

Find every consumer from lineage data and open one agent pull request per repository

Upgrade the JDK in bounded pull requests computed from the build graph

Migrate a test framework with a recipe that keeps only passing targets

Company Playbooks