Run
10 emulators 1,266 tests
Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so an upward arrow can be the new tests biting rather than a target getting worse.
Suite grew from 1,251 to 1,266 tests this run.
That's 15 new tests measured against every target. Movement below compares to the previous run, so a rise here is as likely to be the stricter suite as a real regression.
What changed in the suite this run
Suite on
Five Scan error messages are now regional splits. The validation-framework
rollout reached Scan's parameter checks in September, and eu-west-2,
eu-central-1, us-west-1 and ap-northeast-2 now word the errors for Segment >=
TotalSegments, a Limit of 0, a negative Segment, a mismatched
ExclusiveStartKey and SPECIFIC_ATTRIBUTES without a projection differently
from the other 29 regions. Four passes across all 33 regions agreed. The
assertions follow eu-west-2, the pinned region, onto the new wording, and an
engine giving the old wording is credited in the regions that still use it. The
negative Segment message had no exact-wording test, so it gains one, taking
the suite to 1252 tests.
The negative Segment ordering test now accepts either wording, since it only
checks which validation fires. The capture harness gains the matching Scan
probes and a --table-prefix option for taking them locally.
The nesting-depth row moves af-south-1, ap-southeast-3, ap-southeast-4, ap-southeast-5 and il-central-1 to the strict side, leaving 22 regions strict and 11 lax. ap-northeast-1 answers both ways and stays lax for now.
UpdateItem's item size is now a regional split, across eleven tests. Twenty-two regions, eu-west-2 among them, size an update by what the statement writes, as the suite already pinned, while ten cap the finished item at 409,600 like any other write. Engines that apply the plain cap are credited in those ten.
Fourteen cases from @exoego for the 4096-byte expression cap and the 32-level nesting cap on the batch and transact write surfaces, captured against eu-west-2 and us-east-1.
- Expression size in a transaction. A
TransactWriteItemsmember'sConditionExpressionorUpdateExpressionover 4096 bytes is a top-levelValidationException, not a cancellation, for all four member types and even when an earlier member is fine. - Nesting depth on batch and transact writes. A Put item nested beyond the
32-level cap is refused up front by
BatchWriteItemandTransactWriteItems. UpdateItem checks the depth of anExpressionAttributeValuesentry in most regions, butTransactWriteItemsdoes not check it for any action in any region. An Update that writes a too deep value into the item still cancels with aValidationErrorreason on the stored-item cap.
-
Grade baselineground truth0.0%Divergence100.0%Coverage
no divergence in all 33 regions
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges100.0% coveredTier 3 · Strict 0.0% diverges100.0% covered -
Grade Adiverged 0.6 percentage points more0.8%Divergence98.9%Coverage
low divergence (0.8%) in 17 regions · up to 1.7% in the other 16
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges95.9% covered 14 unsupportedTier 3 · Strict 2.4% diverges100.0% coveredAlso built for WebAssembly / OPFS
-
Grade A
low divergence (0.2%) in 17 regions · up to 1.0% in the other 16 · covers 88.0%
-
-
Grade Bdiverged 0.1 percentage points more1.9%Divergence78.9%Coverage
low divergence (1.9%) in 17 regions · up to 2.3% in the other 16
coverage lowers this row to B
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges33.6% covered 229 unsupportedTier 3 · Strict 5.6% diverges91.1% covered 38 unsupportedAlso built for SQLite
-
Grade B
low divergence (2.1%) in 17 regions · up to 2.5% in the other 16 · covers 83.0% · coverage lowers this row to B
-
-
Grade Bdiverged 0.3 percentage points less6.8%Divergence95.9%Coverage
moderate divergence (6.8%) in 1 region · up to 8.1% in the other 32
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.3% diverges89.6% covered 36 unsupportedTier 3 · Strict 20.0% diverges96.2% covered 16 unsupported -
Grade Bdiverged 0.6 percentage points less7.5%Divergence95.3%Coverage
moderate divergence (7.5%) in 1 region · up to 8.9% in the other 32
Tier breakdown
Tier 1 · Core 2.8% diverges100.0% coveredTier 2 · Complete 12.5% diverges87.5% covered 43 unsupportedTier 3 · Strict 8.9% diverges96.2% covered 16 unsupported -
Grade Cdiverged 0.8 percentage points less10.7%Divergence68.6%Coverage
moderate divergence (10.7%) in 10 regions · up to 11.9% in the other 23
coverage lowers this row to C
Tier breakdown
Tier 1 · Core 10.7% diverges100.0% coveredTier 2 · Complete 8.1% diverges11.6% covered 305 unsupportedTier 3 · Strict 12.9% diverges78.1% covered 93 unsupported -
f5aa8c18302c 59 unsupported ; covers 95.3% of the suite
Runs via: Docker, Homebrew, install script, Scoop, binary, JAR
Grade Cdiverged 0.7 percentage points less14.3%Divergence95.3%Coveragemoderate divergence (14.3%) in 1 region · up to 15.3% in the other 32
coverage lowers this row to C
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 27.5% diverges87.5% covered 43 unsupportedTier 3 · Strict 20.2% diverges96.2% covered 16 unsupported -
Grade Cdiverged 0.9 percentage points less14.8%Divergence94.2%Coverage
moderate divergence (14.8%) in 10 regions · up to 15.7% in the other 23
coverage lowers this row to C
Tier breakdown
Tier 1 · Core 7.5% diverges100.0% coveredTier 2 · Complete 15.4% diverges83.5% covered 57 unsupportedTier 3 · Strict 22.8% diverges96.2% covered 16 unsupported -
Grade Fdiverged 0.4 percentage points less44.9%Divergence78.5%Coverage
severe divergence (44.9%) in 10 regions · up to 45.6% in the other 23
Tier breakdown
Tier 1 · Core 47.2% diverges100.0% coveredTier 2 · Complete 13.9% diverges32.2% covered 234 unsupportedTier 3 · Strict 67.3% diverges91.1% covered 38 unsupported