{ "spec_version": "ci-score-v0.1.3", "scope_statement": "CI Score gauges best-practice adherence on CI speed and gate hygiene: a straightforward pass/fail rubric of best-practice checks computed from a repo's workflow configuration.", "scope_note": "The score reads CONFIGURATION FACTS a maintainer can verify in their own YAML in seconds. The measured wall-clock report renders beside it and enriches check evidence; measured magnitudes never gate a score in v1. Security stays out of v1 except SHA pinning (owner-named); runner-minute efficiency is reported alongside, not scored.", "bands": [ { "grade": "A", "min": 85, "max": 100, "minus": [ 85, 89 ], "bare": [ 90, 94 ], "plus": [ 95, 100 ] }, { "grade": "B", "min": 70, "max": 84, "minus": [ 70, 74 ], "bare": [ 75, 79 ], "plus": [ 80, 84 ] }, { "grade": "C", "min": 55, "max": 69, "minus": [ 55, 59 ], "bare": [ 60, 64 ], "plus": [ 65, 69 ] }, { "grade": "D", "min": 40, "max": 54, "minus": [ 40, 44 ], "bare": [ 45, 49 ], "plus": [ 50, 54 ] }, { "grade": "F", "min": 0, "max": 39, "minus": null, "bare": [ 0, 39 ], "plus": null } ], "bands_note": "Suffix ranges are PINNED, not computed. The A band is 16 points wide and does not divide evenly into three, so an implementer who divides by three gets A- 85-90 while this spec says A- 85-89 - a repo scoring 90 would be published as 'A-' by one implementation and 'A' by another. All bounds are inclusive. F is never suffixed (there is no F+ / F-), matching the convention every reader already holds.", "refusals": [ { "reason_code": "no_workflow_yaml", "human_reason": "No score: no GitHub Actions workflow files were found, so there is nothing to check.", "trigger": "scanned_workflows == 0 (an integer count)" }, { "reason_code": "facts_unavailable", "human_reason": "No score: this document does not carry a complete practice-facts stamp for this spec version - re-run the skill to score this repo. This is a data-vintage limit, not a finding against the repo.", "trigger": "practice_facts is absent, or does not cover every check in this spec's registry (a partial stamp must never publish a score computed from the subset)" }, { "reason_code": "automation_only", "human_reason": "Not scored: this repository's workflows show no build or test activity - what is visible is automation (bots, releases, triage), not the project's CI.", "trigger": "workflows exist (scanned_workflows > 0) but NONE do project build or test: no TEST signal (a test-like job id/name, OR a test-runner command in a step - pytest/go test/npm test/cargo test/rspec/... - so a real test-only repo whose job is named `ci` still scores), no BUILD signal (build tool / build-or-lint job / build command / build action), and no dependency-install COMMAND in any workflow. A setup-* action does NOT count (issue-triage bots use setup-node/setup-bun to run their scripts) and manifest presence does NOT count (an automation-only repo may carry a package.json its CI never installs). A repo that delegates its CI to a cross-repo reusable workflow is NEVER refused (invisible, not absent). Conservative: any one signal scores the repo. Added OD-CS20 after anthropics/claude-code, an npm package whose visible workflows are all issue-triage bots + release automation, graded a real-looking number off pure automation." } ], "refusals_note": "THREE refusals, all 'nothing to score honestly' shapes: no workflows at all; a document whose facts stamp is missing/incomplete for this spec version; or an automation-only repo whose workflows do no build or test (OD-CS20). Config facts need no run history, no branch-protection access, and no merge-queue calibration - every repo with REAL CI scores on a fresh run (owner directive, OD-CS15; OD-CS20 refuses only pure automation).", "practice_coverage_note": "Every published /best-practices/github-actions/ page maps to a scored check EXCEPT keep-advisory-checks-non-blocking: whether an advisory check blocks merges lives in branch-protection semantics, not workflow YAML, so no configuration fact can decide it honestly - deferred to v2 with the required-checks data (recorded here so 'every best practice' is accounted for, not silently missing).", "formula": { "check_states": [ "pass", "fail", "not_applicable" ], "score": "round_half_up(100 * checks_passed / checks_applicable) - not_applicable leaves the denominator", "rounding": "ROUND HALF UP - floor(x + 0.5). Python's round() is banker's rounding; any reimplementation must agree.", "rules": { "evidence_display_only": "Where the measured pipeline has evidence for a failed check (e.g. the uncached install's observed cost on the gating job), the card MAY display it under the check. Display only - it never changes a state.", "purity": "the scorer is a pure function of (findings.json minus the stamp, this spec.json): no network, no clock, no LLM. Same inputs, byte-identical stamp." } }, "checks": [ { "check_id": "ci.cache.dependency-cache", "label": "Dependency caching", "fact": "at least one workflow or local composite action configures dependency caching (a cache action - actions/cache incl. restore/save, rust-cache, buildjet - or a setup-* action's cache input). Caching housed in cross-repo reusable workflows is not visible; run the score in that repo.", "practice_slug": "cache-dependencies", "not_applicable_when": "the repo shows NO dependency-install signal at all - no dependency manifest at the repo root (package.json / lockfiles / requirements*.txt / pyproject.toml / go.mod / Cargo.toml / build.gradle / pom.xml / *.csproj / mix.exs / build.sbt / Gemfile / composer.json / pubspec.yaml and the like), no dependency-install command in any workflow or composite step (pip / npm / pnpm / yarn / bun / poetry / uv / composer / bundle / gem / conda / mvn / gradle / dotnet / mix / cabal / stack with an install-style verb, or `go mod download` / `cargo fetch`), and no language setup action (actions/setup-node / setup-python / setup-java / setup-dotnet / setup-go / setup-ruby and the like) - and no cache is already configured; with nothing installed there is nothing to cache, so a missing cache is never a fail (OD-CS19). Any one of those three signals makes the check applicable. OR CI is delegated to cross-repo reusable workflows and no local caching is visible - a mechanism this fact cannot see is never failed" }, { "check_id": "ci.cache.build-cache", "label": "Build caching", "fact": "a build-tool cache is configured (turbo/nx/gradle/sccache/bazel remote or actions-cache-backed)", "practice_slug": null, "not_applicable_when": "no detectable build tool configuration in the repo tree (turbo.json / nx.json / settings.gradle(.kts) / .bazelrc)" }, { "check_id": "ci.checkout.shallow-clone", "label": "Shallow checkout", "fact": "no checkout step on a PR-GATING workflow sets fetch-depth: 0 (structure-walked; comments never count; post-merge automation - a pull_request trigger whose types are only [closed], e.g. backport/changelog jobs - is exempt: it needs history and gates nobody)", "practice_slug": "shallow-checkout" }, { "check_id": "ci.parallel.test-sharding", "label": "Test sharding / matrix", "fact": "a test-like job (test/spec/e2e/integration/unit in its id or name) runs a matrix, or any job's matrix axis is shard-like (shard/chunk/split/partition)", "practice_slug": "shard-tests", "not_applicable_when": "no test-like job (test/spec/e2e/integration/unit in an id or name) and no shard-like matrix axis exists in any parsed workflow - you cannot shard tests you do not have, so a missing shard is never a fail (OD-CS19); OR CI is delegated to cross-repo reusable workflows and no local test job is visible" }, { "check_id": "ci.build.change-scoped", "label": "Change-scoped builds", "fact": "CI scopes work to what changed: --filter=...[base], nx affected, --changed/--onlyChanged/--affected run modes, turbo-ignore, or a changed-files step (dorny/paths-filter, tj-actions/changed-files)", "practice_slug": "build-only-affected", "not_applicable_when": "no turbo/nx task graph is detected - gradle/bazel scoping is not checkable from workflow YAML and is never failed for a mechanism this fact cannot see" }, { "check_id": "ci.trigger.concurrency-groups", "label": "Concurrency groups", "fact": "at least one PR-triggered workflow declares a concurrency group (the config lever against self-stampede queue time)", "practice_slug": "cut-queue-time" }, { "check_id": "ci.trigger.cancel-superseded", "label": "Superseded runs cancelled", "fact": "at least one PR-triggered workflow sets concurrency with cancel-in-progress (true or a templated expression - conditional cancellation is the practice too; an explicit false is not)", "practice_slug": "cancel-superseded-runs" }, { "check_id": "ci.trigger.path-filter", "label": "Path filters", "fact": "at least one PR-triggered workflow scopes itself with paths / paths-ignore", "practice_slug": "path-filter-workflows" }, { "check_id": "ci.hygiene.job-timeouts", "label": "Job timeouts", "fact": "workflow jobs set timeout-minutes (any usage; GitHub's default is 360 minutes)", "practice_slug": null }, { "check_id": "ci.security.scoped-id-token", "label": "Scoped OIDC id-token", "fact": "id-token: write is granted at job level, never workflow-wide (an explicit grant or a workflow-level permissions: write-all both count as workflow-wide)", "practice_slug": "scope-id-token-per-job", "not_applicable_when": "no workflow requests an OIDC id-token" }, { "check_id": "ci.security.pinned-action-shas", "label": "Pinned action SHAs", "fact": "at least 95% of remote action references are pinned to a full commit SHA, across workflow files AND local composite actions (first-party actions/* included - the pinning practice does not exempt them; the evidence always shows pinned-of-total). The threshold asks 'is pinning an adopted, automated practice?': >=95% admits Renovate-style stragglers (at most 1 in 20) while rejecting partial adoption, and is gaming-resistant - reaching it from a low base means actually pinning the fleet (OD-CS17)", "practice_slug": "pin-action-shas" } ], "band_population": { "repo_count": 31, "note": "Grade bands were read against a calibration dry run over public open-source repositories. Private and monorepo calibration is pending." } }