Skip to content

Run

10 emulators 1,678 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so an upward arrow can be the new tests biting rather than a target getting worse.

Suite grew from 1,266 to 1,678 tests this run.

That's 412 new tests measured against every target. Movement below compares to the previous run, so a rise here is as likely to be the stricter suite as a real regression.

What changed in the suite this run

Suite on

The suite grows from 1,266 tests to 1,678, so every target's figures move. The new tests come from captures against real DynamoDB, most of them taken in October, and mostly cover rejection paths and the order errors are reported in, where a wrong answer used to score as a pass.

  • PartiQL grammar and functions. Number literals with an exponent or no leading digit, comments, ORDER BY DESC, table aliases, unfinished statements and tokens left over after one, functions in WHERE and SET, set_add and set_delete, the FROM-first form of UPDATE, and what each write returns.
  • Numbers. Exponents outside the range, including zero, on attribute values, expression values and every key path, and in PartiQL parameters, along with which error wins when a parameter is both nested too deep and not a valid number.
  • Query key conditions. Operand types for begins_with, the ordering operators and BETWEEN, condition counts, and which problem is reported when a key condition has more than one.
  • Multi-attribute GSI keys. A GSI with one HASH and four RANGE attributes, queried and refused, and PartiQL refusing to read one while the base table and other indexes keep working. These sit in Tier 2, since the feature is new.
  • Batch and transaction validation. The enums, table-name member paths, empty and malformed conditions and unused values that were only tested on single-item operations, ClientRequestToken length, and a validation-ordering file for transactions.
  • Deep requests. Which error wins when a value nested 40, 63 or 100 levels sits beside another problem, and the order TransactWriteItems checks its actions in.
  • The two validation layers. How malformed numbers and duplicate set members are worded on each operation, and which are checked before a transaction runs.
  • Sizes and billing. Exact expression-size messages, an UpdateItem ceiling test per clause kind, and vector billing: a vector attribute is charged 4 bytes per dimension on a write and, since some time between 9 and 22 October, on a read too, while the 400KB limit still counts its numbers.

DynamoDB is still rolling out its newer request validation, so some regions answer these requests differently, and eu-west-2 gave two answers to one of them on the same day. Where that happens a test accepts the two answers seen and names the regions on each side. Those tests are listed on #243 to be pinned once the regions settle.

Twenty existing tests were tightened along the way: nineteen in the batch and transaction work, mostly exact strings where a phrase or pattern was checked, and the 1E-130 precision test, which now compares the stored value rather than accepting anything that came back (#276). The UpdateItem size notes also stop counting the regions that size an update like any other write, a count that had gone stale, and point at their registry rows instead (#244).

The board, each target's page and the results table now say when a target's letter would not hold in every region. Scored against a region that answers some tests differently, a target can earn a lower letter than the one its row publishes, and the row now names those regions and that letter, for example "B against eu-south-2 and me-central-1, where real DynamoDB answers some tests differently". The grading criteria do not change and no letter moves, so the grading version stays at 1. A note can come and go between runs as regions converge and diverge, with no change in the target (#271).

A BatchExecuteStatement test described ReturnValuesOnConditionCheckFailure as inert on a batch member, the opposite of what DynamoDB does, and its only check could not tell a target that ignores the option from one that honours it. DynamoDB returns the existing item with ALL_OLD, on the failed member's Error, and none with NONE or the option absent. The three tests now check that item exactly, so a target that ignores the option, or returns the item whatever the setting, now fails one or more of them. The TransactWriteItems test beside them checks the item on the cancellation reason exactly too, where it checked only that one was there. The batch tests keep their names, which still hold for the member itself, because a rename needs every committed results file re-run.

Fifteen regions move to the new wording on the five Scan rows. af-south-1, ap-east-1, ap-northeast-3, ap-south-1, ap-southeast-1 to ap-southeast-5, ca-west-1, eu-north-1, eu-west-3, il-central-1, mx-central-1 and sa-east-1 joined eu-west-2 by the 2026-10-17 sweep, which leaves 13 regions on the old wording, and an engine that gives it is now credited in those 13 alone. Three runs against all 33 regions on 2026-10-22 agreed with the sweep on every move; where they disagreed, a region keeps its recorded answer. That holds back us-east-1, which gave the new Scan wording on all three runs, and us-east-2 and eu-south-2, which gave the pinned answer on the other fourteen rows. me-central-1 gave both answers to BatchGetItem's empty RequestItems across the runs and keeps the one its row records.

Scored against 33 real regions. 1 dropped, out of scoring until it returns: me-south-1
  1. DynamoDB
    live (AWS)

    Runs via: AWS service

    Grade baseline
    0.0%Divergence
    100.0%Coverage

    no divergence in all 33 regions

    ground truth
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    100.0% covered
    Tier 3 · Strict 0.0% diverges
    100.0% covered
  2. Dynoxide
    1.4.0 14 unsupported ; covers 99.2% of the suite

    Runs via: npx, Docker, Homebrew, binary, npm, cargo, embedded, GitHub Action, source

    Maintained by the board’s author

    Grade A
    2.2%Divergence
    99.2%Coverage

    low divergence (2.2%) in 20 regions · up to 3.4% in the other 13

    diverged 2.2 percentage points more
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.2% diverges
    96.9% covered 14 unsupported
    Tier 3 · Strict 4.9% diverges
    100.0% covered
    Also built for WebAssembly / OPFS
    • WebAssembly / OPFS
      Grade B

      low divergence (1.4%) in 20 regions · up to 2.6% in the other 13 · covers 87.8% · coverage lowers this row to B

  3. Floci
    e0b292df4dc9 7 unsupported ; covers 99.6% of the suite

    Runs via: Docker, Homebrew, install script, Scoop, binary, JAR

    Grade B
    14.4%Divergence
    99.6%Coverage

    moderate divergence (14.4%) in 20 regions · up to 15.6% in the other 13

    C against eu-south-2 and me-central-1, where real DynamoDB answers some tests differently

    diverged 14.4 percentage points more, grade A to B
    Tier breakdown
    Tier 1 · Core 0.2% diverges
    100.0% covered
    Tier 2 · Complete 6.2% diverges
    98.5% covered 7 unsupported
    Tier 3 · Strict 29.2% diverges
    100.0% covered
  4. ExtendDB
    v0.1.13 504 unsupported ; covers 70.0% of the suite

    Runs via: source

    Grade C
    10.3%Divergence
    70.0%Coverage

    moderate divergence (10.3%) in 11 regions · up to 10.5% in the other 22

    coverage lowers this row to C

    diverged 8.4 percentage points more, grade B to C
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 2.7% diverges
    28.8% covered 322 unsupported
    Tier 3 · Strict 21.9% diverges
    75.0% covered 182 unsupported
    Also built for SQLite
    • SQLite
      Grade C

      moderate divergence (10.7%) in 11 regions · up to 11.0% in the other 22 · covers 73.4% · coverage lowers this row to C

  5. LocalStack
    2026.10.1 73 unsupported ; covers 95.6% of the suite

    Runs via: Docker, pip, Homebrew, binary

    Grade C
    19.5%Divergence
    95.6%Coverage

    high divergence (19.5%) in 11 regions · up to 20.4% in the other 22

    diverged 17.1 percentage points more, grade A to C
    Tier breakdown
    Tier 1 · Core 0.2% diverges
    100.0% covered
    Tier 2 · Complete 13.5% diverges
    89.6% covered 47 unsupported
    Tier 3 · Strict 36.5% diverges
    96.4% covered 26 unsupported
  6. Ministack
    e47e70067479

    Runs via: pip, Docker, source

    Grade C
    21.6%Divergence
    100.0%Coverage

    high divergence (21.6%) in 20 regions · up to 22.8% in the other 13

    diverged 21.5 percentage points more, grade A to C
    Tier breakdown
    Tier 1 · Core 0.2% diverges
    100.0% covered
    Tier 2 · Complete 16.2% diverges
    100.0% covered
    Tier 3 · Strict 39.5% diverges
    100.0% covered
  7. DynamoDB Local
    6ada631ec7aa 79 unsupported ; covers 95.3% of the suite

    Runs via: Docker, JAR, Maven

    Grade D
    24.1%Divergence
    95.3%Coverage

    high divergence (24.1%) in 2 regions · up to 24.8% in the other 31

    coverage lowers this row to D

    diverged 9.4 percentage points more, grade C to D
    Tier breakdown
    Tier 1 · Core 7.6% diverges
    100.0% covered
    Tier 2 · Complete 11.9% diverges
    86.5% covered 61 unsupported
    Tier 3 · Strict 42.8% diverges
    97.5% covered 18 unsupported
  8. Dynalite
    4.0.0 707 unsupported ; covers 57.9% of the suite

    Runs via: npm, npx, embedded

    Grade D
    11.9%Divergence
    57.9%Coverage

    moderate divergence (11.9%) in 2 regions · up to 12.8% in the other 31

    coverage lowers this row to D

    diverged 1.2 percentage points more, grade C to D
    Tier breakdown
    Tier 1 · Core 11.1% diverges
    100.0% covered
    Tier 2 · Complete 6.2% diverges
    8.8% covered 412 unsupported
    Tier 3 · Strict 16.0% diverges
    59.5% covered 295 unsupported
  9. Kumo
    76e97ef55735 508 unsupported ; covers 69.7% of the suite

    Runs via: Docker, binary, source

    Grade F
    43.9%Divergence
    69.7%Coverage

    severe divergence (43.9%) in 2 regions · up to 44.4% in the other 31

    diverged 0.9 percentage points less
    Tier breakdown
    Tier 1 · Core 47.3% diverges
    100.0% covered
    Tier 2 · Complete 13.1% diverges
    27.9% covered 326 unsupported
    Tier 3 · Strict 60.6% diverges
    75.0% covered 182 unsupported