Skip to content

Run

9 emulators 1251 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so an upward arrow can be the new tests biting rather than a target getting worse.

Suite grew from 1056 to 1251 tests this run.

That's 195 new tests measured against every target. Movement below compares to the previous run, so a rise here is as likely to be the stricter suite as a real regression.

What changed in the suite this run

Suite on

Thirteen captures against eu-west-2 turned into coverage, and the suite grew from 1056 tests to 1251. All of it was ground truth nothing asserted, so an engine could discard a PartiQL index qualifier or store an item AWS would refuse and still score a clean pass.

  • PartiQL index qualifier. What a qualified SELECT returns, when an unprojected attribute is rejected and when it is served from the base table, how the reach-back is charged, that a qualified write is rejected, and the exact wording of each rejection.
  • PartiQL comparison on non-scalar types. Equality and inequality on every set, map and list, each checked against the same predicate written as a ConditionExpression.
  • BatchExecuteStatement member fields. Per-member ConsistentRead, the primary-key requirement, and which failures echo TableName.
  • The 400KB gate, to the byte. 409,600 accepted and 409,601 refused across seven write surfaces. UpdateItem is the outlier: it charges what the statement writes plus a fixed cost per clause, so the key and every untouched attribute stay out of the figure.
  • A number's byte cost. Twenty-three literals, each measured by an accept and a refusal either side of the gate. The rule the developer guide documents is wrong in two ways, and both have a case.
  • Reads are sized before the filter. Scan, Query and PartiQL all charge before the WHERE clause and the projection, so discarding rows saves nothing.
  • Vector write capacity. VectorWriteRequestBytes pinned term by term, charged on the change to the index's own stored view like a GSI. VectorSearchRequestBytes stays shape-only: five identical searches against an unchanged index reported 14214, 13903, 14214, 14214 and 14518.
  • Vector index creation has two phases. Resource allocation, where the table sits in UPDATING, Backfilling reads false and a cancel is refused, then backfilling, which takes one. The one-online-index limit binds the table rather than the call.

Two fixes came with it. A local run against real DynamoDB takes its own table namespace, so it can no longer delete a CI run's tables. And a TransactWriteItems control, plus the whole PartiQL validation-ordering file, were feeding an unsupported fault into an assertion about wording and scoring a declared gap as a disagreement; both now skip.

BatchWriteItem with an empty RequestItems map is now a recorded regional split. eu-north-1 answers the validation framework's generic constraint message where the other 32 answering regions answer the bespoke required-parameter sentence. A capture on 2026-08-17 found every answering region on the bespoke wording, which dates the crossing to that week rather than guessing at it.

The over-25-requests assertion beside it spans both cohorts instead of becoming a second row. Neither wording is byte-stable: the table name carries a per-run suffix, the old cohort echoes all 26 requests back, and seven regions echo them as a JVM object identity rather than expanded fields. A row records one verbatim answer per region and there is nothing verbatim here to record, so the anchored pattern grew a second branch instead. It still refuses a wrong limit and a wrong table, and it matches every one of the 33 answering regions.

The capture harness can take that evidence without creating tables. --probes selects named probes and --no-tables skips the fixtures, which is all these three need: DynamoDB refuses each of them before it looks at a table, so the answers do not depend on one existing, and read-only credentials are enough. The 2026-08-17 capture was taken with a scoped script that was never committed; this is the committed way to reproduce it.

Scored against 33 real regions. 1 dropped, out of scoring until it returns: me-south-1
  1. DynamoDB
    live (AWS)

    Runs via: AWS service

    Grade baseline

    no divergence in all 33 regions · covers 100.0% of the suite

    ground truth
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    100.0% covered
    Tier 3 · Strict 0.0% diverges
    100.0% covered
  2. Dynoxide
    1.1.0 14 unsupported ; covers 98.9% of the suite

    Runs via: npx, Docker, Homebrew, binary, npm, cargo, embedded, GitHub Action, source

    Disclosure: maintained by this board's author

    Grade A

    low divergence (0.2%) in 22 regions · up to 0.5% in the other 11 · covers 98.9% of the suite

    diverged 0.2 percentage points more
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    95.9% covered 14 unsupported
    Tier 3 · Strict 0.5% diverges
    100.0% covered
    Also built for WebAssembly / OPFS
    • WebAssembly / OPFS
      Grade A

      low divergence (0.2%) in 16 regions · up to 0.5% in the other 17 · covers 88.8%

  3. ExtendDB
    v0.1.10 267 unsupported ; covers 78.7% of the suite

    Runs via: source

    Grade B

    low divergence (1.8%) in 16 regions · up to 2.2% in the other 17 · covers 78.7% of the suite · coverage lowers this row to B

    diverged 1.7 percentage points more, grade A to B
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    33.6% covered 229 unsupported
    Tier 3 · Strict 5.6% diverges
    90.7% covered 38 unsupported
    Also built for SQLite
    • SQLite
      Grade B

      low divergence (2.1%) in 16 regions · up to 2.4% in the other 17 · covers 82.8% · coverage lowers this row to B

  4. LocalStack
    2026.8.1 59 unsupported ; covers 95.3% of the suite

    Runs via: Docker, pip, Homebrew, binary

    Grade B

    moderate divergence (8.1%) in 1 regions · up to 8.3% in the other 32 · covers 95.3% of the suite

    diverged 4.6 percentage points more
    Tier breakdown
    Tier 1 · Core 2.8% diverges
    100.0% covered
    Tier 2 · Complete 12.5% diverges
    87.5% covered 43 unsupported
    Tier 3 · Strict 10.7% diverges
    96.1% covered 16 unsupported
  5. Dynalite
    4.0.0 386 unsupported ; covers 69.1% of the suite

    Runs via: npm, npx, embedded

    Grade C

    moderate divergence (11.5%) in 11 regions · up to 11.7% in the other 22 · covers 69.1% of the suite · coverage lowers this row to C

    diverged 1.3 percentage points less
    Tier breakdown
    Tier 1 · Core 10.7% diverges
    100.0% covered
    Tier 2 · Complete 8.1% diverges
    11.6% covered 305 unsupported
    Tier 3 · Strict 15.4% diverges
    80.2% covered 81 unsupported
  6. DynamoDB Local
    ff89bd48ff32 73 unsupported ; covers 94.2% of the suite

    Runs via: Docker, JAR, Maven

    Grade C

    high divergence (15.7%) in 16 regions · up to 15.7% in the other 17 · covers 94.2% of the suite

    diverged 0.6 percentage points more
    Tier breakdown
    Tier 1 · Core 7.5% diverges
    100.0% covered
    Tier 2 · Complete 15.4% diverges
    83.5% covered 57 unsupported
    Tier 3 · Strict 25.9% diverges
    96.1% covered 16 unsupported
  7. Ministack
    9b67e6f8e53d 52 unsupported ; covers 95.8% of the suite

    Runs via: pip, Docker, source

    Grade C

    high divergence (17.6%) in 6 regions · up to 17.7% in the other 27 · covers 95.8% of the suite

    diverged 5.7 percentage points more, grade B to C
    Tier breakdown
    Tier 1 · Core 5.6% diverges
    100.0% covered
    Tier 2 · Complete 25.5% diverges
    89.6% covered 36 unsupported
    Tier 3 · Strict 25.4% diverges
    96.1% covered 16 unsupported
  8. Floci
    4e451c39c7bb 61 unsupported ; covers 95.1% of the suite

    Runs via: Docker, Homebrew, install script, Scoop, binary, JAR

    Grade D

    very high divergence (26.9%) in 6 regions · up to 27.1% in the other 27 · covers 95.1% of the suite

    diverged 6.0 percentage points more, grade C to D
    Tier breakdown
    Tier 1 · Core 10.1% diverges
    100.0% covered
    Tier 2 · Complete 44.6% diverges
    87.0% covered 45 unsupported
    Tier 3 · Strict 32.4% diverges
    96.1% covered 16 unsupported