Skip to content

Run

7 emulators 736 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so a downward arrow can be the new tests biting rather than a target regressing.

Suite grew from 706 to 736 tests this run.

That's 30 new tests measured against every target. Movement below compares to the previous run, so a fall here is as likely to be the stricter suite as a real regression.

What changed in the suite this run

Suite on

Grew to 736 tests, up 30, covering what TransactWriteItems and BatchWriteItem do with an item whose key value is malformed - the wrong type, non-scalar, or an empty string - across both table and index keys. PutItem already covered this; the transactional and batch paths covered none of it.

Characterising it against real AWS turned up a split worth pinning. An empty-string key value is rejected by up-front input validation, so even inside a transaction it surfaces as a top-level ValidationException. A wrong-typed or non-scalar key value is caught while the transaction runs, so it comes back as a TransactionCanceledException carrying a ValidationError reason rather than a top-level error. BatchWriteItem has no cancellation path, so every variant there is a plain ValidationException. The tests pin both halves, which catches two opposite mistakes: an engine that wraps the empty-string case as a cancellation, and one that surfaces the type-mismatch case as a top-level error.

  1. DynamoDB
    live (AWS) full coverage
    100% ground truth
    Tier 1 100%
    Tier 2 100%
    Tier 3 100%
  2. Dynoxide
    0.10.0 14 unsupported ; covers 98.1% of the suite
    95.3% -3.6pp fell 3.6 percentage points
    Tier 1 97.3%
    Tier 2 93.7%
    Tier 3 93.0%
  3. ExtendDB
    v0.1.1 42 unsupported ; covers 94.3% of the suite
    94.4% -3.1pp fell 3.1 percentage points
    Tier 1 95.1%
    Tier 2 91.9%
    Tier 3 94.3%
  4. Ministack
    a64f3c3bebff full coverage
    91.0% -5.6pp fell 5.6 percentage points
    Tier 1 91.0%
    Tier 2 87.9%
    Tier 3 93.0%
  5. Floci
    18c54f4a774a 9 unsupported ; covers 98.8% of the suite
    88.4% -3.5pp fell 3.5 percentage points
    Tier 1 91.5%
    Tier 2 82.6%
    Tier 3 86.9%
  6. LocalStack
    2026.5.4 8 unsupported ; covers 98.9% of the suite
    86.4% -2.0pp fell 2.0 percentage points
    Tier 1 97.8%
    Tier 2 91.0%
    Tier 3 65.5%
  7. DynamoDB Local
    d89f8fcc6b1a 13 unsupported ; covers 98.2% of the suite
    85.2% -2.0pp fell 2.0 percentage points
    Tier 1 96.4%
    Tier 2 87.5%
    Tier 3 65.9%
  8. Dynalite
    4.0.0 67 unsupported ; covers 90.9% of the suite
    80.0% -2.2pp fell 2.2 percentage points
    Tier 1 95.1%
    Tier 2 14.9%
    Tier 3 76.9%