How the CI Score is computed
CI Score gauges best-practice adherence on CI speed and gate hygiene: a straightforward pass/fail rubric of best-practice checks computed from a repo's workflow configuration.
- spec
- ci-score-v0.1.3
- checks
- 11
- refusals
- 3
Passed over applicable, rounded half up.
round_half_up(100 * checks_passed / checks_applicable) - not_applicable leaves the denominator
ROUND HALF UP - floor(x + 0.5). Python's round() is banker's rounding; any reimplementation must agree.
- scope
- The score reads CONFIGURATION FACTS a maintainer can verify in their own YAML in seconds. The measured wall-clock report renders beside it and enriches check evidence; measured magnitudes never gate a score in v1. Security stays out of v1 except SHA pinning (owner-named); runner-minute efficiency is reported alongside, not scored.
- purity
- the scorer is a pure function of (findings.json minus the stamp, this spec.json): no network, no clock, no LLM. Same inputs, byte-identical stamp.
- evidence
- Where the measured pipeline has evidence for a failed check (e.g. the uncached install's observed cost on the gating job), the card MAY display it under the check. Display only - it never changes a state.
Pinned ranges, never rendered.
Published pages render the numeric score only. The bands stay defined in the spec and stamped beside every value, but no page prints a letter: with at most eleven applicable checks the instrument cannot separate adjacent values, and the number says that honestly where a report-card letter would not.
| Grade | Range | Minus | Bare | Plus |
|---|---|---|---|---|
| A | 85 to 100 | 85 to 89 | 90 to 94 | 95 to 100 |
| B | 70 to 84 | 70 to 74 | 75 to 79 | 80 to 84 |
| C | 55 to 69 | 55 to 59 | 60 to 64 | 65 to 69 |
| D | 40 to 54 | 40 to 44 | 45 to 49 | 50 to 54 |
| F | 0 to 39 | n/a | 0 to 39 | n/a |
Suffix ranges are PINNED, not computed. The A band is 16 points wide and does not divide evenly into three, so an implementer who divides by three gets A- 85-90 while this spec says A- 85-89 - a repo scoring 90 would be published as 'A-' by one implementation and 'A' by another. All bounds are inclusive. F is never suffixed (there is no F+ / F-), matching the convention every reader already holds.
Calibrated against 31 repositories. Grade bands were read against a calibration dry run over public open-source repositories. Private and monorepo calibration is pending.
The 11 checks, grouped by namespace.
- cache
- Dependency cachingci.cache.dependency-cache
- Build cachingci.cache.build-cache
- checkout
- Shallow checkoutci.checkout.shallow-clone
- parallel
- Test sharding / matrixci.parallel.test-sharding
- build
- Change-scoped buildsci.build.change-scoped
- trigger
- Concurrency groupsci.trigger.concurrency-groups
- Superseded runs cancelledci.trigger.cancel-superseded
- Path filtersci.trigger.path-filter
- hygiene
- Job timeoutsci.hygiene.job-timeouts
- security
- Scoped OIDC id-tokenci.security.scoped-id-token
- Pinned action SHAsci.security.pinned-action-shas
Each check's exact configuration fact, and the rule that takes it out of the denominator, is published verbatim in the scoring spec.
The skill that computes this score documents the full methodology (opens in new tab), check by check: what each one looks for, and the reasoning behind it.
When there is no score, we say so.
no_workflow_yaml
No score: no GitHub Actions workflow files were found, so there is nothing to check.
facts_unavailable
No score: this document does not carry a complete practice-facts stamp for this spec version - re-run the skill to score this repo. This is a data-vintage limit, not a finding against the repo.
automation_only
Not scored: this repository's workflows show no build or test activity - what is visible is automation (bots, releases, triage), not the project's CI.
THREE refusals, all 'nothing to score honestly' shapes: no workflows at all; a document whose facts stamp is missing/incomplete for this spec version; or an automation-only repo whose workflows do no build or test (OD-CS20). Config facts need no run history, no branch-protection access, and no merge-queue calibration - every repo with REAL CI scores on a fresh run (owner directive, OD-CS15; OD-CS20 refuses only pure automation).
Who publishes this, and the conflict stated plainly
StarSling builds CI runners. This score deliberately grades configuration hygiene, not speed or runner choice. No check depends on any StarSling product, runner-minute economics are reported beside a score and never graded, and the scorer is a pure local function that anyone can re-run to reproduce any number we publish.
The grade does not predict speed, and measured, it runs the other way: large monorepos adopt every practice and also carry enormous test volume, while small repositories are fast because there is little to run. Among the repositories we measured end to end, the lowest-scoring one has the fastest merge gate. That is why the measured wall-clock report renders beside a score and is never blended into it.
If a published page about your repository is wrong, the page carries a contact address. Factual errors are corrected and re-scored, and the collection date tells you which run you are reading.
How these scores get published
Every score here is collected by running the skill against a full, clean checkout of the repository at one commit, and the resulting document is committed to this site as data. Nothing is scored on demand, and nothing recomputes when a page renders: the board shows committed stamps or it shows nothing.
Five checks run in our CI on every change. Parity asserts that every value a page exposes equals the committed stamp. Recompute re-runs the real scorer over the committed facts and requires the stamp back bit for bit. Provenance re-derives the scorer's own file hashes and requires them to match the pinned commit we vendored from. The fourth re-derives the published artifacts, so a committed cache cannot drift from the data it came from. The fifth rejects any local filesystem path in the committed data, so nothing about the machine that ran the collection ships with the score.