Skip to content

Run

7 emulators 699 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in.

When this run was published the board led with a single correctness percentage. The figures here are that run's own results restated as divergence and coverage, so they read differently from what this page showed at the time. The changelog has the change.

What changed in the suite this run

Suite on

Real DynamoDB in eu-west-2 reworded a chunk of its validation errors, and the Tier 3 error-message tests moved with it. They now assert the contract the error carries - its type, the field it objects to, and the constraint - rather than the exact prose, because AWS varies the wrapper, the echoed input value and the field casing from one region to the next. A four-region capture in June found eu-west-2 and eu-central-1 on the new wording and us-east-1 and ap-southeast-2 still on the old, so the line between contract and cosmetic is drawn from what is invariant across all four.

This moves some Tier 3 numbers. Targets that were only ever marked down for wording DynamoDB itself renders inconsistently now pass those checks, so their Tier 3 scores rise: the suite has stopped counting a cosmetic difference as a behavioural one. Genuine behavioural divergences are still pinned exactly, for example PutItem with a { NULL: false } attribute, which DynamoDB now accepts in eu-west-2 and normalises to { NULL: true }.

  1. live (AWS)

    Runs via: AWS service

    Grade baseline

    no divergence in all 33 regions · covers 100.0% of the suite

    ground truth
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    100.0% covered
    Tier 3 · Strict 0.0% diverges
    100.0% covered
  2. Runs via: npx, Docker, Homebrew, binary, npm, cargo, embedded, GitHub Action, source

    Disclosure: maintained by this board's author

    Grade A

    low divergence (0.4%) · covers 98.0% of the suite

    diverged 0.4 percentage points more
    Tier breakdown
    Tier 1 · Core 0.0% diverges
    100.0% covered
    Tier 2 · Complete 0.0% diverges
    89.4% covered 14 unsupported
    Tier 3 · Strict 1.4% diverges
    100.0% covered
  3. Grade A

    low divergence (2.0%) · covers 94.0% of the suite

    diverged 3.9 percentage points less
    Tier breakdown
    Tier 1 · Core 2.8% diverges
    100.0% covered
    Tier 2 · Complete 1.5% diverges
    68.2% covered 42 unsupported
    Tier 3 · Strict 0.9% diverges
    100.0% covered
  4. 176654570cf2

    Runs via: pip, Docker, source

    Grade A

    low divergence (2.6%) · covers 100.0% of the suite

    diverged 10.4 percentage points less
    Tier breakdown
    Tier 1 · Core 1.7% diverges
    100.0% covered
    Tier 2 · Complete 6.1% diverges
    100.0% covered
    Tier 3 · Strict 1.9% diverges
    100.0% covered
  5. Runs via: Docker, Homebrew, install script, Scoop, binary, JAR

    Grade B

    moderate divergence (7.3%) · covers 98.7% of the suite

    unchanged
    Tier breakdown
    Tier 1 · Core 5.9% diverges
    100.0% covered
    Tier 2 · Complete 11.4% diverges
    93.2% covered 9 unsupported
    Tier 3 · Strict 7.0% diverges
    100.0% covered
  6. Runs via: Docker, pip, Homebrew, binary

    Grade B

    moderate divergence (11.6%) · covers 98.9% of the suite

    unchanged
    Tier breakdown
    Tier 1 · Core 1.4% diverges
    100.0% covered
    Tier 2 · Complete 6.8% diverges
    93.9% covered 8 unsupported
    Tier 3 · Strict 31.3% diverges
    100.0% covered
  7. Grade B

    moderate divergence (12.7%) · covers 98.1% of the suite

    unchanged
    Tier breakdown
    Tier 1 · Core 2.8% diverges
    100.0% covered
    Tier 2 · Complete 9.8% diverges
    90.2% covered 13 unsupported
    Tier 3 · Strict 30.8% diverges
    100.0% covered
  8. Grade C

    high divergence (16.2%) · covers 90.4% of the suite

    diverged 0.9 percentage points more
    Tier breakdown
    Tier 1 · Core 4.5% diverges
    100.0% covered
    Tier 2 · Complete 40.9% diverges
    49.2% covered 67 unsupported
    Tier 3 · Strict 20.1% diverges
    100.0% covered