---
title: "CI Score methodology: the published formula"
description: "The exact CI Score formula: eleven pass/fail configuration checks, how not-applicable leaves the denominator, the A to F bands, and the refusals."
url: https://starsling.dev/ci-score/methodology
canonicalUrl: https://starsling.dev/ci-score/methodology
---

# How the CI Score is computed

CI Score gauges best-practice adherence on CI speed and gate hygiene: a straightforward pass/fail rubric of best-practice checks computed from a repo's workflow configuration.

Spec version: ci-score-v0.1.3. The machine-readable form of everything below is at https://starsling.dev/ci-score/spec.json.

## The formula

```
round_half_up(100 * checks_passed / checks_applicable) - not_applicable leaves the denominator
```

ROUND HALF UP - floor(x + 0.5). Python's round() is banker's rounding; any reimplementation must agree.

The score reads CONFIGURATION FACTS a maintainer can verify in their own YAML in seconds. The measured wall-clock report renders beside it and enriches check evidence; measured magnitudes never gate a score in v1. Security stays out of v1 except SHA pinning (owner-named); runner-minute efficiency is reported alongside, not scored.

the scorer is a pure function of (findings.json minus the stamp, this spec.json): no network, no clock, no LLM. Same inputs, byte-identical stamp.

Where the measured pipeline has evidence for a failed check (e.g. the uncached install's observed cost on the gating job), the card MAY display it under the check. Display only - it never changes a state.

## The bands

Published pages render the numeric score only. The bands stay defined in the spec and stamped beside every value, but no page prints a letter: with at most eleven applicable checks the instrument cannot separate adjacent values, and the number says that honestly where a report-card letter would not.

| Grade | Range | Minus | Bare | Plus |
| --- | --- | --- | --- | --- |
| A | 85 to 100 | 85 to 89 | 90 to 94 | 95 to 100 |
| B | 70 to 84 | 70 to 74 | 75 to 79 | 80 to 84 |
| C | 55 to 69 | 55 to 59 | 60 to 64 | 65 to 69 |
| D | 40 to 54 | 40 to 44 | 45 to 49 | 50 to 54 |
| F | 0 to 39 | n/a | 0 to 39 | n/a |

Suffix ranges are PINNED, not computed. The A band is 16 points wide and does not divide evenly into three, so an implementer who divides by three gets A- 85-90 while this spec says A- 85-89 - a repo scoring 90 would be published as 'A-' by one implementation and 'A' by another. All bounds are inclusive. F is never suffixed (there is no F+ / F-), matching the convention every reader already holds.

Calibrated against 31 public open-source repositories. Private and monorepo calibration is pending.

## The 11 checks

### Dependency caching

`ci.cache.dependency-cache`

at least one workflow or local composite action configures dependency caching (a cache action - actions/cache incl. restore/save, rust-cache, buildjet - or a setup-* action's cache input). Caching housed in cross-repo reusable workflows is not visible; run the score in that repo.

Not applicable when: the repo shows NO dependency-install signal at all - no dependency manifest at the repo root (package.json / lockfiles / requirements*.txt / pyproject.toml / go.mod / Cargo.toml / build.gradle / pom.xml / *.csproj / mix.exs / build.sbt / Gemfile / composer.json / pubspec.yaml and the like), no dependency-install command in any workflow or composite step (pip / npm / pnpm / yarn / bun / poetry / uv / composer / bundle / gem / conda / mvn / gradle / dotnet / mix / cabal / stack with an install-style verb, or `go mod download` / `cargo fetch`), and no language setup action (actions/setup-node / setup-python / setup-java / setup-dotnet / setup-go / setup-ruby and the like) - and no cache is already configured; with nothing installed there is nothing to cache, so a missing cache is never a fail (OD-CS19). Any one of those three signals makes the check applicable. OR CI is delegated to cross-repo reusable workflows and no local caching is visible - a mechanism this fact cannot see is never failed

[Practice guide](https://starsling.dev/best-practices/github-actions/cache-dependencies)

### Build caching

`ci.cache.build-cache`

a build-tool cache is configured (turbo/nx/gradle/sccache/bazel remote or actions-cache-backed)

Not applicable when: no detectable build tool configuration in the repo tree (turbo.json / nx.json / settings.gradle(.kts) / .bazelrc)

### Shallow checkout

`ci.checkout.shallow-clone`

no checkout step on a PR-GATING workflow sets fetch-depth: 0 (structure-walked; comments never count; post-merge automation - a pull_request trigger whose types are only [closed], e.g. backport/changelog jobs - is exempt: it needs history and gates nobody)

[Practice guide](https://starsling.dev/best-practices/github-actions/shallow-checkout)

### Test sharding / matrix

`ci.parallel.test-sharding`

a test-like job (test/spec/e2e/integration/unit in its id or name) runs a matrix, or any job's matrix axis is shard-like (shard/chunk/split/partition)

Not applicable when: no test-like job (test/spec/e2e/integration/unit in an id or name) and no shard-like matrix axis exists in any parsed workflow - you cannot shard tests you do not have, so a missing shard is never a fail (OD-CS19); OR CI is delegated to cross-repo reusable workflows and no local test job is visible

[Practice guide](https://starsling.dev/best-practices/github-actions/shard-tests)

### Change-scoped builds

`ci.build.change-scoped`

CI scopes work to what changed: --filter=...[base], nx affected, --changed/--onlyChanged/--affected run modes, turbo-ignore, or a changed-files step (dorny/paths-filter, tj-actions/changed-files)

Not applicable when: no turbo/nx task graph is detected - gradle/bazel scoping is not checkable from workflow YAML and is never failed for a mechanism this fact cannot see

[Practice guide](https://starsling.dev/best-practices/github-actions/build-only-affected)

### Concurrency groups

`ci.trigger.concurrency-groups`

at least one PR-triggered workflow declares a concurrency group (the config lever against self-stampede queue time)

[Practice guide](https://starsling.dev/best-practices/github-actions/cut-queue-time)

### Superseded runs cancelled

`ci.trigger.cancel-superseded`

at least one PR-triggered workflow sets concurrency with cancel-in-progress (true or a templated expression - conditional cancellation is the practice too; an explicit false is not)

[Practice guide](https://starsling.dev/best-practices/github-actions/cancel-superseded-runs)

### Path filters

`ci.trigger.path-filter`

at least one PR-triggered workflow scopes itself with paths / paths-ignore

[Practice guide](https://starsling.dev/best-practices/github-actions/path-filter-workflows)

### Job timeouts

`ci.hygiene.job-timeouts`

workflow jobs set timeout-minutes (any usage; GitHub's default is 360 minutes)

### Scoped OIDC id-token

`ci.security.scoped-id-token`

id-token: write is granted at job level, never workflow-wide (an explicit grant or a workflow-level permissions: write-all both count as workflow-wide)

Not applicable when: no workflow requests an OIDC id-token

[Practice guide](https://starsling.dev/best-practices/github-actions/scope-id-token-per-job)

### Pinned action SHAs

`ci.security.pinned-action-shas`

at least 95% of remote action references are pinned to a full commit SHA, across workflow files AND local composite actions (first-party actions/* included - the pinning practice does not exempt them; the evidence always shows pinned-of-total). The threshold asks 'is pinning an adopted, automated practice?': >=95% admits Renovate-style stragglers (at most 1 in 20) while rejecting partial adoption, and is gaming-resistant - reaching it from a low base means actually pinning the fleet (OD-CS17)

[Practice guide](https://starsling.dev/best-practices/github-actions/pin-action-shas)

Each check's exact configuration fact, and the rule that takes it out of the denominator, is published verbatim in [the scoring spec](https://starsling.dev/ci-score/spec.json).

Every published /best-practices/github-actions/ page maps to a scored check EXCEPT keep-advisory-checks-non-blocking: whether an advisory check blocks merges lives in branch-protection semantics, not workflow YAML, so no configuration fact can decide it honestly - deferred to v2 with the required-checks data (recorded here so 'every best practice' is accounted for, not silently missing).

## Refusals

- **no_workflow_yaml**: No score: no GitHub Actions workflow files were found, so there is nothing to check.
- **facts_unavailable**: No score: this document does not carry a complete practice-facts stamp for this spec version - re-run the skill to score this repo. This is a data-vintage limit, not a finding against the repo.
- **automation_only**: Not scored: this repository's workflows show no build or test activity - what is visible is automation (bots, releases, triage), not the project's CI.

THREE refusals, all 'nothing to score honestly' shapes: no workflows at all; a document whose facts stamp is missing/incomplete for this spec version; or an automation-only repo whose workflows do no build or test (OD-CS20). Config facts need no run history, no branch-protection access, and no merge-queue calibration - every repo with REAL CI scores on a fresh run (owner directive, OD-CS15; OD-CS20 refuses only pure automation).

## Who publishes this, and the conflict stated plainly

StarSling builds CI runners. This score deliberately grades configuration hygiene, not speed or runner choice. No check depends on any StarSling product, runner-minute economics are reported beside a score and never graded, and the scorer is a pure local function that anyone can re-run to reproduce any number we publish.

The grade does not predict speed, and measured, it runs the other way: large monorepos adopt every practice and also carry enormous test volume, while small repositories are fast because there is little to run. Among the repositories we measured end to end, the lowest-scoring one has the fastest merge gate. That is why the measured wall-clock report renders beside a score and is never blended into it.

If a published page about your repository is wrong, the page carries a contact address. Factual errors are corrected and re-scored, and the collection date tells you which run you are reading.

## How these scores get published

Every score here is collected by running the skill against a full, clean checkout of the repository at one commit, and the resulting document is committed to this site as data. Nothing is scored on demand, and nothing recomputes when a page renders: the board shows committed stamps or it shows nothing.

Five checks run in our CI on every change. Parity asserts that every value a page exposes equals the committed stamp. Recompute re-runs the real scorer over the committed facts and requires the stamp back bit for bit. Provenance re-derives the scorer's own file hashes and requires them to match the pinned commit we vendored from. The fourth re-derives the published artifacts, so a committed cache cannot drift from the data it came from. The fifth rejects any local filesystem path in the committed data, so nothing about the machine that ran the collection ships with the score.

The skill that computes this score documents the full methodology, check by check: what each one looks for, and the reasoning behind it. https://github.com/starslingdev/skills/blob/main/skills/ci-score/references/ci-score-methodology.md

See an error? Email founders@starsling.dev