Run
7 emulators 762 tests
Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so an upward arrow can be the new tests biting rather than a target getting worse.
When this run was published the board led with a single correctness percentage. The figures here are that run's own results restated as divergence and coverage, so they read differently from what this page showed at the time. The changelog has the change.
Suite grew from 744 to 762 tests this run.
That's 18 new tests measured against every target. Movement below compares to the previous run, so a rise here is as likely to be the stricter suite as a real regression.
What changed in the suite this run
Suite on
Grew to 762 tests, up 18, covering a malformed value in the lookup Key of a TransactWriteItems Update, Delete, or ConditionCheck - the path a Put item key does not take. Captured across four regions (eu-west-2, us-east-1, ap-southeast-2, eu-central-1), where every string was identical, so they pin exactly. An empty-string Key surfaces as a top-level ValidationException with the same message a Put item key gives; a wrong-typed or non-scalar Key cancels with a ValidationError reason carrying "The provided key element does not match the schema" - the key-only form, not the "Type mismatch for key" message the item-key path returns. The same run confirmed the BatchWriteItem table-key schema-mismatch message is region-invariant.
-
Grade baselineground truth0.0%Divergence100.0%Coverage
no divergence in all 33 regions
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges100.0% coveredTier 3 · Strict 0.0% diverges100.0% covered -
Grade Adiverged 5.4 percentage points less0.3%Divergence98.2%Coverage
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges90.7% covered 14 unsupportedTier 3 · Strict 0.8% diverges100.0% covered -
Grade Bdiverged 1.4 percentage points more7.5%Divergence94.5%Coverage
Tier breakdown
Tier 1 · Core 6.4% diverges100.0% coveredTier 2 · Complete 7.3% diverges72.0% covered 42 unsupportedTier 3 · Strict 9.2% diverges100.0% covered -
Grade Bunchanged10.0%Divergence100.0%Coverage
Tier breakdown
Tier 1 · Core 6.7% diverges100.0% coveredTier 2 · Complete 17.3% diverges100.0% coveredTier 3 · Strict 10.5% diverges100.0% covered -
78ee619e895a 9 unsupported ; covers 98.8% of the suite
Runs via: Docker, Homebrew, install script, Scoop, binary, JAR
Grade Bdiverged 1.8 percentage points more10.8%Divergence98.8%CoverageTier breakdown
Tier 1 · Core 5.6% diverges100.0% coveredTier 2 · Complete 19.3% diverges94.0% covered 9 unsupportedTier 3 · Strict 13.4% diverges100.0% covered -
Grade Bdiverged 0.7 percentage points more14.0%Divergence99.0%Coverage
Tier breakdown
Tier 1 · Core 2.1% diverges100.0% coveredTier 2 · Complete 10.0% diverges94.7% covered 8 unsupportedTier 3 · Strict 35.3% diverges100.0% covered -
Grade Cdiverged 0.8 percentage points more15.2%Divergence98.3%Coverage
Tier breakdown
Tier 1 · Core 3.5% diverges100.0% coveredTier 2 · Complete 12.7% diverges91.3% covered 13 unsupportedTier 3 · Strict 35.3% diverges100.0% covered -
Grade Cdiverged 1.9 percentage points more21.0%Divergence91.2%Coverage
Tier breakdown
Tier 1 · Core 7.0% diverges100.0% coveredTier 2 · Complete 48.0% diverges55.3% covered 67 unsupportedTier 3 · Strict 26.1% diverges100.0% covered