Run
7 emulators 684 tests
Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so an upward arrow can be the new tests biting rather than a target getting worse.
When this run was published the board led with a single correctness percentage. The figures here are that run's own results restated as divergence and coverage, so they read differently from what this page showed at the time. The changelog has the change.
Suite grew from 655 to 684 tests this run.
That's 29 new tests measured against every target. Movement below compares to the previous run, so a rise here is as likely to be the stricter suite as a real regression.
What changed in the suite this run
Suite on
Grew to 684 tests with a control-plane and table-configuration sweep: the CreateTable/UpdateTable config parameters and the secondary control-plane operations (limits, backups and PITR, exports and imports, Kinesis, resource policies, contributor insights), each characterised against real AWS and probe-skipped where a target doesn't implement it.
The published percentage changed with it. It now measures correctness over implemented operations, Pass / (Pass + Fail), so skips no longer count against the score. A skip is honest scope; a fail is a bug.
-
Grade baselineground truth0.0%Divergence100.0%Coverage
no divergence in all 33 regions
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges100.0% coveredTier 3 · Strict 0.0% diverges100.0% covered -
Grade Bdiverged 0.6 percentage points more4.4%Divergence98.0%Coverage
coverage lowers this row to B
Tier breakdown
Tier 1 · Core 5.2% diverges100.0% coveredTier 2 · Complete 4.8% diverges88.7% covered 14 unsupportedTier 3 · Strict 2.8% diverges100.0% covered -
e981a5afd790 9 unsupported ; covers 98.7% of the suite
Runs via: Docker, Homebrew, install script, Scoop, binary, JAR
Grade Bdiverged 1.2 percentage points more6.4%Divergence98.7%CoverageTier breakdown
Tier 1 · Core 6.1% diverges100.0% coveredTier 2 · Complete 8.1% diverges92.7% covered 9 unsupportedTier 3 · Strict 6.1% diverges100.0% covered -
Grade Bdiverged 1.0 percentage points more6.0%Divergence95.0%Coverage
Tier breakdown
Tier 1 · Core 6.9% diverges100.0% coveredTier 2 · Complete 4.8% diverges72.6% covered 34 unsupportedTier 3 · Strict 5.1% diverges100.0% covered -
Grade Bdiverged 0.2 percentage points less11.8%Divergence98.8%Coverage
Tier breakdown
Tier 1 · Core 1.4% diverges100.0% coveredTier 2 · Complete 7.3% diverges93.5% covered 8 unsupportedTier 3 · Strict 31.3% diverges100.0% covered -
Grade Bdiverged 0.1 percentage points less12.7%Divergence98.1%Coverage
Tier breakdown
Tier 1 · Core 2.3% diverges100.0% coveredTier 2 · Complete 10.5% diverges89.5% covered 13 unsupportedTier 3 · Strict 30.8% diverges100.0% covered -
Grade Cdiverged 0.8 percentage points more15.6%Divergence91.4%Coverage
Tier breakdown
Tier 1 · Core 4.6% diverges100.0% coveredTier 2 · Complete 43.5% diverges52.4% covered 59 unsupportedTier 3 · Strict 17.3% diverges100.0% covered -
Grade Ddiverged 0.1 percentage points less34.4%Divergence98.5%Coverage
Tier breakdown
Tier 1 · Core 19.1% diverges100.0% coveredTier 2 · Complete 24.2% diverges91.9% covered 10 unsupportedTier 3 · Strict 65.0% diverges100.0% covered