Skip to content

Run

10 emulators 1,266 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in.

What changed in the suite this run

Suite on

3.6.0 was cut but never published. eu-west-2 changed how it validates TransactWriteItems while the 3.6.0 measurement was running, so five tests failed against DynamoDB itself and no board was produced. This release carries the fix, and the changes listed under 3.6.0 are first measured here.

eu-north-1 and ap-northeast-2 made the change first, and eu-west-2 followed on 2026-10-08, between two runs on main 30 minutes apart. The three now check a transaction's request before running it. An empty TransactItems gets the validation framework's generic constraint message. A 32-level value in the request is refused with a top-level ValidationException, where the transaction used to cancel on it or go on to evaluate the condition. A transacted Update that UpdateItem's statement-sized rule refuses is refused the same way, where it used to be stored. The other 30 regions still answer as before.

The five tests accept either answer, exactly, until the regions settle. No engine loses a test over it, and on the 3.5.0 board's results the engines that already refuse a 32-level value up front pass tests they used to fail: three for LocalStack, and one each for DynamoDB Local, ExtendDB and ExtendDB (SQLite). The tests keep their names, which describe what the 30 regions do, until one answer is pinned, because a rename needs every committed results file re-run.

Scored against 33 real regions. 1 dropped, out of scoring until it returns: me-south-1
  1. live (AWS)

    Runs via: AWS service

    Grade baseline
    0.0%Divergence
    100.0%Coverage

    no divergence in all 33 regions

    ground truth
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    100.0% covered
    Tier 3 · Strict 0.0% diverges
    100.0% covered
  2. Runs via: Docker, Homebrew, install script, Scoop, binary, JAR

    Grade A
    0.1%Divergence
    99.4%Coverage

    low divergence (0.1%) in 26 regions · up to 1.3% in the other 7

    diverged 14.2 percentage points less, grade C to A
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    98.0% covered 7 unsupported
    Tier 3 · Strict 0.2% diverges
    100.0% covered
  3. Runs via: npx, Docker, Homebrew, binary, npm, cargo, embedded, GitHub Action, source

    Maintained by the board’s author

    Grade A
    0.8%Divergence
    98.9%Coverage

    low divergence (0.8%) in 26 regions · up to 1.7% in the other 7

    unchanged
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    95.9% covered 14 unsupported
    Tier 3 · Strict 2.4% diverges
    100.0% covered
    Also built for WebAssembly / OPFS
    • Grade A

      low divergence (0.2%) in 26 regions · up to 1.0% in the other 7 · covers 88.0%

  4. Grade B
    1.8%Divergence
    78.9%Coverage

    low divergence (1.8%) in 26 regions · up to 2.2% in the other 7

    coverage lowers this row to B

    diverged 0.1 percentage points less
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    33.6% covered 229 unsupported
    Tier 3 · Strict 5.4% diverges
    91.1% covered 38 unsupported
    Also built for SQLite
    • SQLite
      Grade B

      low divergence (2.1%) in 26 regions · up to 2.4% in the other 7 · covers 83.0% · coverage lowers this row to B

  5. Runs via: Docker, pip, Homebrew, binary

    Grade A
    2.4%Divergence
    95.7%Coverage

    low divergence (2.4%) in 26 regions · up to 3.6% in the other 7

    diverged 5.1 percentage points less, grade B to A
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 6.1% diverges
    89.0% covered 38 unsupported
    Tier 3 · Strict 2.4% diverges
    96.2% covered 16 unsupported
  6. Runs via: pip, Docker, source

    Grade B
    6.6%Divergence
    95.9%Coverage

    moderate divergence (6.6%) in 1 region · up to 7.9% in the other 32

    diverged 0.2 percentage points less
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.3% diverges
    89.6% covered 36 unsupported
    Tier 3 · Strict 19.5% diverges
    96.2% covered 16 unsupported
  7. Grade C
    10.7%Divergence
    68.6%Coverage

    moderate divergence (10.7%) in 2 regions · up to 11.9% in the other 31

    coverage lowers this row to C

    unchanged
    Tier breakdown
    Tier 1 · Core 10.7% diverges
    100.0% covered
    Tier 2 · Complete 8.1% diverges
    11.6% covered 305 unsupported
    Tier 3 · Strict 12.9% diverges
    78.1% covered 93 unsupported
  8. Grade C
    14.7%Divergence
    94.2%Coverage

    moderate divergence (14.7%) in 2 regions · up to 15.6% in the other 31

    coverage lowers this row to C

    diverged 0.1 percentage points less
    Tier breakdown
    Tier 1 · Core 7.5% diverges
    100.0% covered
    Tier 2 · Complete 15.4% diverges
    83.5% covered 57 unsupported
    Tier 3 · Strict 22.6% diverges
    96.2% covered 16 unsupported
  9. Runs via: Docker, binary, source

    Grade F
    44.9%Divergence
    78.6%Coverage

    severe divergence (44.9%) in 2 regions · up to 45.6% in the other 31

    unchanged
    Tier breakdown
    Tier 1 · Core 47.2% diverges
    100.0% covered
    Tier 2 · Complete 13.9% diverges
    32.5% covered 233 unsupported
    Tier 3 · Strict 67.3% diverges
    91.1% covered 38 unsupported