Skip to content

Run

7 emulators 982 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so an upward arrow can be the new tests biting rather than a target getting worse.

When this run was published the board led with a single correctness percentage. The figures here are that run's own results restated as divergence and coverage, so they read differently from what this page showed at the time. The changelog has the change.

Suite grew from 954 to 982 tests this run.

That's 28 new tests measured against every target. Movement below compares to the previous run, so a rise here is as likely to be the stricter suite as a real regression.

What changed in the suite this run

Suite on

Grew to 982 tests, up 28, all characterised against real DynamoDB across the per-region ground truth 2.0.0 put in place. New coverage is PartiQL's RETURNING clause; the GSI lifecycle also joins the ground truth, and the sweep's drift classifier gains a converged case.

  • PartiQL RETURNING (#103, #105): the clause pinned across every PartiQL surface. DELETE accepts only RETURNING ALL OLD * and rejects the other three variants with a 400, the message pinned; UPDATE accepts all four ALL/MODIFIED x OLD/NEW forms, where MODIFIED returns just the changed attribute and drops the key. The empty-projection edges are pinned too: a MODIFIED projection that resolves to nothing - a nested leaf, a list index, a batch statement - returns Items: [] rather than a keyed row. BatchExecuteStatement honours RETURNING; ExecuteTransaction rejects it. The non-upsert paths round it out: INSERT on an existing item, and UPDATE or DELETE behind a false predicate, fail ConditionalCheckFailed and leave the item untouched.
  • PartiQL RETURNING over list indices (#102): the MODIFIED projection shapes for a list-index SET or REMOVE, pinned against real AWS. The projection reads the literal index against the resulting list (MODIFIED NEW *) or the prior list (MODIFIED OLD *): an in-range index returns its element, an out-of-range one returns nothing, and multiple changed indices pack into a dense list in ascending index order. So a non-zero index collapses to one element, an append projects only when the index lands inside the new list, and removing the last index yields nothing under NEW while a middle REMOVE returns the shifted element. Setting an index on an absent attribute is rejected, not auto-created. In BatchExecuteStatement a member that fails to parse surfaces per-statement with the short Code: ValidationError, the same Code as an execution failure, and does not fail the batch.
  • GSI lifecycle in the ground truth (#100): the 14 UpdateTable GSI lifecycle tests are now observed by the weekly sweep and recorded as per-region ground truth. They are the one slice the gating run drops for runtime, so they were the one slice without a real-AWS answer; the sweep records them without slowing the gate.
  • Drift classification (#104): a drift where every region moves off the pinned answer at once is now classified as converged rather than moved, with the converged path covered end to end.
Scored against 33 real regions. 1 dropped, out of scoring until it returns: me-south-1
  1. DynamoDB
    live (AWS)

    Runs via: AWS service

    Grade baseline

    no divergence in all 33 regions · covers 100.0% of the suite

    ground truth
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    100.0% covered
    Tier 3 · Strict 0.0% diverges
    100.0% covered
  2. Dynoxide
    0.11.4 14 unsupported ; covers 98.6% of the suite

    Runs via: npx, Docker, Homebrew, binary, npm, cargo, embedded, GitHub Action, source

    Disclosure: maintained by this board's author

    Grade A

    low divergence (3.7%) in 6 regions · up to 4.0% in the other 27 · covers 98.6% of the suite

    diverged 2.9 percentage points more
    Tier breakdown
    Tier 1 · Core 0.2% diverges
    100.0% covered
    Tier 2 · Complete 13.7% diverges
    92.3% covered 14 unsupported
    Tier 3 · Strict 3.1% diverges
    100.0% covered
  3. LocalStack
    2026.6.3 8 unsupported ; covers 99.2% of the suite

    Runs via: Docker, pip, Homebrew, binary

    Grade C

    moderate divergence (14.8%) in 27 regions · up to 14.9% in the other 6 · covers 99.2% of the suite · coverage lowers this row to C

    unchanged
    Tier breakdown
    Tier 1 · Core 6.5% diverges
    100.0% covered
    Tier 2 · Complete 13.7% diverges
    95.6% covered 8 unsupported
    Tier 3 · Strict 27.5% diverges
    100.0% covered
  4. ExtendDB
    v0.1.1 71 unsupported ; covers 92.8% of the suite

    Runs via: source

    Grade C

    high divergence (15.6%) in all 33 regions · covers 92.8% of the suite

    diverged 0.5 percentage points less
    Tier breakdown
    Tier 1 · Core 14.5% diverges
    100.0% covered
    Tier 2 · Complete 8.7% diverges
    63.4% covered 67 unsupported
    Tier 3 · Strict 21.0% diverges
    98.8% covered 4 unsupported
  5. DynamoDB Local
    d89f8fcc6b1a 13 unsupported ; covers 98.7% of the suite

    Runs via: Docker, JAR, Maven

    Grade C

    high divergence (15.9%) in 27 regions · up to 16.0% in the other 6 · covers 98.7% of the suite

    unchanged
    Tier breakdown
    Tier 1 · Core 7.8% diverges
    100.0% covered
    Tier 2 · Complete 15.8% diverges
    92.9% covered 13 unsupported
    Tier 3 · Strict 27.8% diverges
    100.0% covered
  6. Ministack
    22a278f078f5

    Runs via: pip, Docker, source

    Grade C

    high divergence (16.7%) in all 33 regions · covers 100.0% of the suite

    diverged 2.3 percentage points more
    Tier breakdown
    Tier 1 · Core 10.9% diverges
    100.0% covered
    Tier 2 · Complete 23.5% diverges
    100.0% covered
    Tier 3 · Strict 21.3% diverges
    100.0% covered
  7. Floci
    d2ecc8035822 9 unsupported ; covers 99.1% of the suite

    Runs via: Docker, Homebrew, install script, Scoop, binary, JAR

    Grade C

    high divergence (19.7%) in all 33 regions · covers 99.1% of the suite

    diverged 2.0 percentage points more
    Tier breakdown
    Tier 1 · Core 8.6% diverges
    100.0% covered
    Tier 2 · Complete 31.1% diverges
    95.1% covered 9 unsupported
    Tier 3 · Strict 29.3% diverges
    100.0% covered
  8. Dynalite
    4.0.0 96 unsupported ; covers 90.2% of the suite

    Runs via: npm, npx, embedded

    Grade C

    high divergence (21.5%) in 27 regions · up to 21.7% in the other 6 · covers 90.2% of the suite

    diverged 0.6 percentage points less
    Tier breakdown
    Tier 1 · Core 8.6% diverges
    100.0% covered
    Tier 2 · Complete 43.2% diverges
    49.7% covered 92 unsupported
    Tier 3 · Strict 28.1% diverges
    98.8% covered 4 unsupported