Run
9 emulators 1054 tests
Results as of this run. The arrow shows each target's movement since the previous run it was tested in.
What changed in the suite this run
Suite on
The board now measures the most recent release tag rather than main. Merging
a test still runs it against real AWS, which is what validates it, but the
published figures no longer move until a release moves them, so a dated
changelog entry always sits behind a change in the denominator. Expect the
figures to shift on this release: the board is switching from measuring main
to measuring a tag, and those are different trees today.
Every board now says what produced it. results/summary.json carries a suite
block naming the ref measured, its commit, the suite version at that ref, the
region it ran against and when. It is additive, so schemaVersion stays at 1.
The same block reaches /data/latest.json, /data/index.json and
/data/runs.json, where each historical run carries its own copy, so a
denominator that moved between two runs can be attributed to the release that
moved it. Branch on kind: only tag is a released board.
A board is graded against the suite manifest and split registry as they stood at the ref it measured. Region health is the exception and is read live, so a region dropped since the tag still counts against today's cohorts. That is the one input allowed to move under a board without a new measurement, and the board carries a health date beside its measurement date to say so.
Releases are cut by one workflow dispatch: it bumps the version, dates this section, installs against the bumped tree, tags, and opens a draft release, then starts the measurement. The draft publishes itself when the board carrying that version lands, which takes about three hours.
eu-west-2 has crossed to the validation framework's generic constraint message
for BatchGetItem with an empty RequestItems map, so the split row recording
that behaviour is re-characterised against a 34-region capture taken on
2026-08-17. Eleven of the thirty-three regions that answer now return the
generic message; twenty-two still return the bespoke required-parameter
sentence, and me-central-1 joins the row.
The matching validation-ordering row is retired. Both wordings refuse an empty
RequestItems before the map is read, which is all that tier asserts, so the
assertion now matches the parameter name case-insensitively and spans both. The
BatchWriteItem case beside it, which no region has moved yet, is written the
same way.
A probe absent from the baseline is no longer reported as drift. Adding a probe to the capture script leaves every older baseline without it, and a scheduled red then named the new probe as the thing that had moved.
The weekly cross-region capture now includes eu-west-2, so the drift lens reads its baseline and the candidate regions from one capture taken at one moment rather than comparing today's candidates against an older baseline file. A scheduled red also keeps the eu-west-2 capture its drift verdict was read from, which was previously discarded with the runner.
-
Grade baselineground truth
no divergence in all 33 regions · covers 100.0% of the suite
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges100.0% coveredTier 3 · Strict 0.0% diverges100.0% covered -
Runs via: npx, Docker, Homebrew, binary, npm, cargo, embedded, GitHub Action, source
Disclosure: maintained by this board's author
Grade Aunchangedlow divergence (0.9%) in 2 regions · up to 1.3% in the other 31 · covers 94.7% of the suite
Tier breakdown
Tier 1 · Core 2.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges82.2% covered 40 unsupportedTier 3 · Strict 0.0% diverges95.3% covered 16 unsupportedAlso built for WebAssembly / OPFS
-
Grade B
low divergence (0.9%) in 2 regions · up to 1.3% in the other 31 · covers 83.4% · coverage lowers this row to B
-
-
Grade Bunchanged
low divergence (1.9%) in 21 regions · up to 2.0% in the other 12 · covers 87.8% of the suite · coverage lowers this row to B
Tier breakdown
Tier 1 · Core 0.8% diverges100.0% coveredTier 2 · Complete 1.8% diverges51.6% covered 109 unsupportedTier 3 · Strict 3.5% diverges94.1% covered 20 unsupportedAlso built for SQLite
-
Grade B
low divergence (1.8%) in 21 regions · up to 1.9% in the other 12 · covers 87.8% · coverage lowers this row to B
-
-
Grade Bunchanged
moderate divergence (11.9%) in 21 regions · up to 12.0% in the other 12 · covers 96.0% of the suite
Tier breakdown
Tier 1 · Core 5.7% diverges100.0% coveredTier 2 · Complete 15.1% diverges88.4% covered 26 unsupportedTier 3 · Strict 18.5% diverges95.3% covered 16 unsupported -
Grade Cunchanged
moderate divergence (12.8%) in 25 regions · up to 13.0% in the other 8 · covers 77.0% of the suite · coverage lowers this row to C
Tier breakdown
Tier 1 · Core 10.8% diverges100.0% coveredTier 2 · Complete 12.4% diverges17.8% covered 185 unsupportedTier 3 · Strict 15.9% diverges83.2% covered 57 unsupported -
Grade Cunchanged
moderate divergence (14.8%) in 25 regions · up to 14.9% in the other 8 · covers 95.3% of the suite · coverage lowers this row to C
Tier breakdown
Tier 1 · Core 6.3% diverges100.0% coveredTier 2 · Complete 16.0% diverges84.9% covered 34 unsupportedTier 3 · Strict 26.2% diverges95.3% covered 16 unsupported -
Grade Cunchanged
high divergence (15.1%) in 25 regions · up to 15.2% in the other 8 · covers 94.0% of the suite
Tier breakdown
Tier 1 · Core 7.6% diverges100.0% coveredTier 2 · Complete 14.2% diverges79.1% covered 47 unsupportedTier 3 · Strict 26.5% diverges95.3% covered 16 unsupported -
bec9b9f74932 51 unsupported ; covers 95.2% of the suite
Runs via: Docker, Homebrew, install script, Scoop, binary, JAR
Grade Cunchangedhigh divergence (21.0%) in 21 regions · up to 21.1% in the other 12 · covers 95.2% of the suite
Tier breakdown
Tier 1 · Core 10.8% diverges100.0% coveredTier 2 · Complete 32.4% diverges84.4% covered 35 unsupportedTier 3 · Strict 27.9% diverges95.3% covered 16 unsupported