Skip to content

ExtendDB (SQLite)

github.com/ExtendDB/extenddb

v0.1.11 9 recorded runs

Grade B
low divergence · coverage lowers this row to B
answers 2.1% of the suite differently from real DynamoDB
up to 2.4% in the others
82.8% of the suite implemented · 215 tests it doesn't attempt
Tier 1 · Core 0.0% diverges
100.0% covered
Tier 2 · Complete 1.2% diverges
44.1% covered 193 unsupported
4 of 152 attempted
Tier 3 · Strict 5.4% diverges
94.6% covered 22 unsupported
22 of 388 attempted

How to run it

Every way ExtendDB (SQLite) is distributed, each linking to the project's own documentation for it. These are the project's claims, not the suite's measurements.

Needs: a Rust toolchain to build it, and no external database

The PostgreSQL build's server over a single SQLite file rather than an external database. TLS and SigV4 stay mandatory.

Per-region divergence

The headline is eu-west-2 + 15 regions, measured against 16 of 33 observed regions, and scored in ap-east-1. Real DynamoDB doesn't behave identically in every region, so a target can match some regions more closely than others. Each figure below is divergence in those regions, best first. eu-west-2 is marked as the historical baseline, and a region that couldn't be resolved this sweep is flagged rather than counted as a disagreement.

2.1% 16 regions
ap-east-1ap-east-2ap-northeast-2ap-northeast-3ap-south-1ap-southeast-1ap-southeast-2ca-west-1eu-central-1eu-west-2 · baselineeu-west-3mx-central-1sa-east-1us-east-1us-west-1us-west-2
2.2% 6 regions
af-south-1ap-southeast-3ap-southeast-4ap-southeast-5eu-north-1il-central-1
2.3% 10 regions
ap-northeast-1ap-south-2ap-southeast-6ap-southeast-7ca-central-1eu-central-2eu-south-1eu-south-2eu-west-1us-east-2
2.4% 1 region
me-central-1

Fully supports

Operation areas this target implements completely - every test passes, nothing skipped. Areas it only partly gets wrong are in “Where it falls short” below; the ones it only partly attempts are in “What it doesn’t attempt”.

  • batchGetItem tier 1
  • batchWriteItem tier 1
  • createTable tier 1
  • deleteItem tier 1
  • deleteTable tier 1
  • describeTable tier 1
  • getItem tier 1
  • listTables tier 1
  • putItem tier 1
  • query tier 1
  • scan tier 1
  • updateItem tier 1
  • updateTable tier 1
  • account tier 2
  • backups tier 2
  • streams tier 2
  • tags tier 2
  • transactions tier 2
  • ttl tier 2
  • updateTable tier 2
  • legacy-api tier 3

Divergence and coverage over time

Both are plotted because neither reads correctly alone. Divergence falls when a target stops attempting an operation it used to get wrong, so a fall in the first plot is only an improvement if the second one holds.

Divergence

ExtendDB (SQLite) divergence, 2026-08-15 to 2026-09-13 0% 2% 4% divergence - lower is better 15 Aug 17 Aug 18 Aug 21 Aug 23 Aug 25 Aug 30 Aug 3 Sep 13 Sep 2.1%
Diverging on 2.1% of the suite as of 13 Sep, at worst 2.6% on 23 Aug. Lower is better, so a falling line is a target getting closer to real DynamoDB.

Coverage

ExtendDB (SQLite) coverage, 2026-08-15 to 2026-09-13 80% 90% 100% coverage - higher is better 15 Aug 17 Aug 18 Aug 21 Aug 23 Aug 25 Aug 30 Aug 3 Sep 13 Sep 82.8%
Implementing 82.8% of the suite as of 13 Sep. Higher is better. This is the share of the suite's tests a target implements at all. A fail becoming a skip leaves both numerators over the same fixed denominator, so a fall here is matched point for point by a fall in divergence: withdrawal costs exactly as much coverage as it gains divergence.

By operation

Every operation this target implements, with how much of it diverges and how much of it it covers, on the same two axes as the figures above. The counts are fails over the operation's whole size. full, partial, failing, unsupported. The matrix compares operations across targets.

Tier 1 - Core

  • batchGetItem 0/15 0.0% 100.0% batchGetItem: supported, diverges on 0.0% of it, covers 100.0% (15 pass)
  • batchWriteItem 0/18 0.0% 100.0% batchWriteItem: supported, diverges on 0.0% of it, covers 100.0% (18 pass)
  • createTable 0/30 0.0% 100.0% createTable: supported, diverges on 0.0% of it, covers 100.0% (30 pass)
  • deleteItem 0/13 0.0% 100.0% deleteItem: supported, diverges on 0.0% of it, covers 100.0% (13 pass)
  • deleteTable 0/3 0.0% 100.0% deleteTable: supported, diverges on 0.0% of it, covers 100.0% (3 pass)
  • describeTable 0/4 0.0% 100.0% describeTable: supported, diverges on 0.0% of it, covers 100.0% (4 pass)
  • getItem 0/39 0.0% 100.0% getItem: supported, diverges on 0.0% of it, covers 100.0% (39 pass)
  • listTables 0/5 0.0% 100.0% listTables: supported, diverges on 0.0% of it, covers 100.0% (5 pass)
  • putItem 0/129 0.0% 100.0% putItem: supported, diverges on 0.0% of it, covers 100.0% (129 pass)
  • query 0/95 0.0% 100.0% query: supported, diverges on 0.0% of it, covers 100.0% (95 pass)
  • scan 0/60 0.0% 100.0% scan: supported, diverges on 0.0% of it, covers 100.0% (60 pass)
  • updateItem 0/70 0.0% 100.0% updateItem: supported, diverges on 0.0% of it, covers 100.0% (70 pass)
  • updateTable 0/15 0.0% 100.0% updateTable: supported, diverges on 0.0% of it, covers 100.0% (15 pass)

Tier 2 - Complete

  • account 0/2 0.0% 100.0% account: supported, diverges on 0.0% of it, covers 100.0% (2 pass)
  • backups 0/5 0.0% 100.0% backups: supported, diverges on 0.0% of it, covers 100.0% (5 pass)
  • contributorInsights 0/2 · 2 skip n/a 0.0% contributorInsights: unsupported, implements none of it (2 skip)
  • export 0/2 · 2 skip n/a 0.0% export: unsupported, implements none of it (2 skip)
  • kinesis 0/1 · 1 skip n/a 0.0% kinesis: unsupported, implements none of it (1 skip)
  • partiql 0/186 · 186 skip n/a 0.0% partiql: unsupported, implements none of it (186 skip)
  • resourcePolicy 0/2 · 2 skip n/a 0.0% resourcePolicy: unsupported, implements none of it (2 skip)
  • streams 0/18 0.0% 100.0% streams: supported, diverges on 0.0% of it, covers 100.0% (18 pass)
  • tags 0/8 0.0% 100.0% tags: supported, diverges on 0.0% of it, covers 100.0% (8 pass)
  • transactions 0/62 0.0% 100.0% transactions: supported, diverges on 0.0% of it, covers 100.0% (62 pass)
  • ttl 0/7 0.0% 100.0% ttl: supported, diverges on 0.0% of it, covers 100.0% (7 pass)
  • updateTable 0/14 0.0% 100.0% updateTable: supported, diverges on 0.0% of it, covers 100.0% (14 pass)
  • vectorSearch 4/36 11.1% 100.0% vectorSearch: partially supported, diverges on 11.1% of it, covers 100.0% (32 pass, 4 fail)

Tier 3 - Strict

  • error-messages 0/184 · 14 skip 0.0% 92.4% error-messages: partially supported, diverges on 0.0% of it, covers 92.4% (170 pass, 14 skip)
  • legacy-api 0/42 0.0% 100.0% legacy-api: supported, diverges on 0.0% of it, covers 100.0% (42 pass)
  • limits 22/150 · 5 skip 14.7% 96.7% limits: partially supported, diverges on 14.7% of it, covers 96.7% (123 pass, 22 fail, 5 skip)
  • validation-ordering 0/34 · 3 skip 0.0% 91.2% validation-ordering: partially supported, diverges on 0.0% of it, covers 91.2% (31 pass, 3 skip)

Where it falls short

Operations ExtendDB (SQLite) implements and answers differently from real DynamoDB, biggest gap first. This is what its divergence figure counts. Open one for the exact tests, or see the conformance suite.

  • limits Tier 3 22 diverging 5 not attempted
    View these tests in the suite →
    • Item size limit by surface - TransactWriteItems an Update accepts a finished item of exactly 409,600 bytes source:198
    • Item size limit by surface - TransactWriteItems an Update one byte over surfaces as a cancellation, with the update wording source:220
    • Item size limit by surface - TransactWriteItems answers the same way for a fresh key and for an existing row source:220
    • Item size limit by surface - UpdateItem does not charge for the key sits below 409,600 at a one-byte key, so the exclusion binds source:362
    • Item size limit by surface - UpdateItem does not charge for the key caps at exactly 409,600 with a 100-byte key value source:362
    • Item size limit by surface - UpdateItem does not charge for the key caps at exactly 409,600 with a 1024-byte key value source:362
    • Item size limit by surface - UpdateItem does not charge for the key buys back the same headroom from the key name as from the key value source:362
    • Item size limit by surface - UpdateItem does not charge for the key buys back a sort key’s own bytes as well source:362
    • Item size limit by surface - UpdateItem charges per action charges a second SET clause exactly 19 bytes source:362
    • Item size limit by surface - UpdateItem charges per action charges a REMOVE alongside a SET exactly 2 bytes source:362
    • Item size limit by surface - UpdateItem charges per action charges the clause, not the value it writes source:362
    • Item size limit by surface - UpdateItem charges per action does not charge for an attribute the statement leaves alone source:362
    • Item size limit by surface - UpdateItem charges per action charges the clause, not the attribute name or its alias source:362
    • Item size limit by surface - UpdateItem through a document path charges a SET through a nested path the same as a plain SET source:570
    • Item size limit by surface - UpdateItem through a document path charges a SET through a list index exactly one byte more source:570
    • Item size limit by surface - a transacted Update does not inherit the exclusion writes what UpdateItem refuses at the same key source:646
    • A number's byte cost sizes '1.5' at '3 bytes' source:102
    • A number's byte cost sizes '1.2' at '3 bytes' source:102
    • A number's byte cost sizes '1.200' at '3 bytes' source:102
    • A number's byte cost sizes '1.234' at '4 bytes' source:102
    • A number's byte cost sizes '3.14159' at '5 bytes' source:102
    • A number's byte cost sizes '100.5' at '4 bytes' source:102
    • Item size limit by surface - PartiQL INSERT accepts an item of exactly 409,600 bytes
    • Item size limit by surface - PartiQL INSERT refuses it one byte over, with the put wording
    • Item size limit by surface - PartiQL UPDATE accepts a finished item of exactly 409,600 bytes
    • Item size limit by surface - PartiQL UPDATE refuses it one byte over, with the update wording
    • Item size limit by surface - a PartiQL UPDATE does not inherit the exclusion writes what UpdateItem refuses at the same key
  • vectorSearch Tier 2 4 diverging
    View these tests in the suite →
    • PartiQL - vector indexes are out of reach rejects a PartiQL read of a vector index source:56
    • PartiQL - vector indexes are out of reach reads the vector attribute off the base table like any other item source:69
    • UpdateTable - vector index lifecycle refuses to delete the table while a vector index is still being created source:317
    • UpdateTable - vector index lifecycle reports UPDATING with Backfilling false while allocating, and takes a cancel only once backfilling starts source:434

What it doesn't attempt

Operations ExtendDB (SQLite) declines rather than gets wrong, so none of this counts towards its divergence. It is what the gap in its 82.8% coverage is made of. Each test here skipped itself because the target's own feature probe said the operation isn't implemented, which is often a deliberate choice rather than a defect.

  • partiql Tier 2 none of it 186 not attempted
    View these tests in the suite →
    • BatchExecuteStatement - PartiQL batch of multiple SELECT statements
    • BatchExecuteStatement - PartiQL batch of INSERT and UPDATE statements
    • BatchExecuteStatement - PartiQL partial failure - one valid and one invalid statement
    • BatchExecuteStatement - PartiQL honours a RETURNING clause on a member statement
    • BatchExecuteStatement - PartiQL honours a RETURNING ALL NEW * clause on a member UPDATE
    • BatchExecuteStatement - PartiQL honours a RETURNING MODIFIED NEW * clause on a member UPDATE (only the changed attr)
    • BatchExecuteStatement - PartiQL omits Item when a member UPDATE produces an empty MODIFIED projection
    • BatchExecuteStatement - PartiQL surfaces an invalid RETURNING variant on a member DELETE as a per-statement error
    • BatchExecuteStatement - PartiQL surfaces a malformed member statement as a per-statement ValidationError without failing the batch
    • BatchExecuteStatement - PartiQL rejects an empty Statements array
    • BatchExecuteStatement - PartiQL ConsistentRead is honoured per member charges a small eventually-consistent read half a unit
    • BatchExecuteStatement - PartiQL ConsistentRead is honoured per member charges exactly twice that when the member asks for consistency
    • BatchExecuteStatement - PartiQL ConsistentRead is honoured per member treats an explicit false the same as omitting it
    • BatchExecuteStatement - PartiQL ConsistentRead is honoured per member rates each member of a mixed batch on its own setting
    • BatchExecuteStatement - PartiQL a batch SELECT must name the table primary key rejects a member that filters on a non-key attribute
    • BatchExecuteStatement - PartiQL a batch SELECT must name the table primary key rejects an index-qualified member naming the index key
    • BatchExecuteStatement - PartiQL a batch SELECT must name the table primary key rejects one naming both the table primary key and the index key
    • BatchExecuteStatement - PartiQL a batch SELECT must name the table primary key leaves the rest of the batch running
    • BatchExecuteStatement - PartiQL a failed member echoes its table only if it ran carries TableName on a conditional failure
    • BatchExecuteStatement - PartiQL a failed member echoes its table only if it ran carries TableName on a duplicate insert
    • BatchExecuteStatement - PartiQL a failed member echoes its table only if it ran omits TableName on a member rejected before it ran
    • BatchExecuteStatement - PartiQL ReturnValuesOnConditionCheckFailure on a batch member returns no Item with the option set to ALL_OLD
    • BatchExecuteStatement - PartiQL ReturnValuesOnConditionCheckFailure on a batch member returns no Item with the option set to NONE
    • BatchExecuteStatement - PartiQL ReturnValuesOnConditionCheckFailure on a batch member returns no Item with the option set to undefined
    • BatchExecuteStatement - PartiQL ReturnValuesOnConditionCheckFailure on a batch member does return the item on a TransactWriteItems ConditionCheck
    • ExecuteStatement - comparison on non-scalar types 'SS' matches an equal value
    • ExecuteStatement - comparison on non-scalar types 'SS' does not match a different value
    • ExecuteStatement - comparison on non-scalar types 'SS' answers <> as the complement of =
    • ExecuteStatement - comparison on non-scalar types 'SS' matches a permuted value, because a string set is unordered
    • ExecuteStatement - comparison on non-scalar types 'SS' agrees with the same predicate as a ConditionExpression
    • ExecuteStatement - comparison on non-scalar types 'NS' matches an equal value
    • ExecuteStatement - comparison on non-scalar types 'NS' does not match a different value
    • ExecuteStatement - comparison on non-scalar types 'NS' answers <> as the complement of =
    • ExecuteStatement - comparison on non-scalar types 'NS' matches a permuted value, because a number set is unordered
    • ExecuteStatement - comparison on non-scalar types 'NS' agrees with the same predicate as a ConditionExpression
    • ExecuteStatement - comparison on non-scalar types 'BS' matches an equal value
    • ExecuteStatement - comparison on non-scalar types 'BS' does not match a different value
    • ExecuteStatement - comparison on non-scalar types 'BS' answers <> as the complement of =
    • ExecuteStatement - comparison on non-scalar types 'BS' matches a permuted value, because a binary set is unordered
    • ExecuteStatement - comparison on non-scalar types 'BS' agrees with the same predicate as a ConditionExpression
    • ExecuteStatement - comparison on non-scalar types 'L' matches an equal value
    • ExecuteStatement - comparison on non-scalar types 'L' does not match a different value
    • ExecuteStatement - comparison on non-scalar types 'L' answers <> as the complement of =
    • ExecuteStatement - comparison on non-scalar types 'L' does not match a permuted value, because a list is ordered, so a permutation is a different value
    • ExecuteStatement - comparison on non-scalar types 'L' agrees with the same predicate as a ConditionExpression
    • ExecuteStatement - comparison on non-scalar types 'M' matches an equal value
    • ExecuteStatement - comparison on non-scalar types 'M' does not match a different value
    • ExecuteStatement - comparison on non-scalar types 'M' answers <> as the complement of =
    • ExecuteStatement - comparison on non-scalar types 'M' matches a permuted value, because a map compares by content, not by the order its keys arrived in
    • ExecuteStatement - comparison on non-scalar types 'M' agrees with the same predicate as a ConditionExpression
    • ExecuteStatement - comparison on non-scalar types 'B' matches an equal value
    • ExecuteStatement - comparison on non-scalar types 'B' does not match a different value
    • ExecuteStatement - comparison on non-scalar types 'B' answers <> as the complement of =
    • ExecuteStatement - comparison on non-scalar types 'B' agrees with the same predicate as a ConditionExpression
    • ExecuteStatement - comparison on non-scalar types 'NULL' matches an equal value
    • ExecuteStatement - comparison on non-scalar types 'NULL' does not match a different value
    • ExecuteStatement - comparison on non-scalar types 'NULL' answers <> as the complement of =
    • ExecuteStatement - comparison on non-scalar types 'NULL' agrees with the same predicate as a ConditionExpression
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering rejects < against BOOL
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering rejects < against NULL
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering rejects < against L
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering rejects < against M
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering rejects < against SS
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering rejects < against NS
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering rejects < against BS
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering names the operator < as written
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering names the operator <= as written
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering names the operator > as written
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering names the operator >= as written
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering names BETWEEN as written too
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering accepts = and <> against BOOL
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering accepts = and <> against NULL
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering accepts = and <> against L
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering accepts = and <> against M
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering accepts = and <> against SS
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering accepts = and <> against NS
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering accepts = and <> against BS
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering accepts an ordering comparison between S and N
    • ExecuteStatement - comparison on non-scalar types ordering operators reject an operand type with no ordering orders binary by its bytes
    • ExecuteStatement - comparison on non-scalar types a write conditioned on a non-scalar predicate fires when the predicate holds
    • ExecuteStatement - comparison on non-scalar types a write conditioned on a non-scalar predicate raises ConditionalCheckFailed and leaves the item when it does not
    • ExecuteStatement - PartiQL INSERTs a new item
    • ExecuteStatement - PartiQL SELECTs an item by primary key
    • ExecuteStatement - PartiQL SELECTs with WHERE clause using comparison
    • ExecuteStatement - PartiQL UPDATEs an existing item
    • ExecuteStatement - PartiQL DELETEs an item
    • ExecuteStatement - PartiQL rejects INSERT on an existing item (INSERT is not upsert)
    • ExecuteStatement - PartiQL INSERT succeeds after DELETE of same key
    • ExecuteStatement - PartiQL SELECT returns empty results for non-matching WHERE
    • ExecuteStatement - PartiQL parameterized INSERT with ? placeholders
    • ExecuteStatement - PartiQL parameterized SELECT with ? placeholder
    • ExecuteStatement - PartiQL SELECT with nested map path
    • ExecuteStatement - PartiQL SELECT with specific attributes
    • ExecuteStatement - PartiQL SELECT with begins_with in WHERE clause
    • ExecuteStatement - PartiQL PartiQL UPDATE with SET on attribute
    • ExecuteStatement - PartiQL PartiQL UPDATE with REMOVE
    • ExecuteStatement - PartiQL returns a populated ConsumedCapacity block when requested
    • ExecuteStatement - PartiQL evaluates negated predicates (NOT begins_with, IS NOT MISSING)
    • ExecuteStatement - PartiQL DELETE with a false non-key predicate fails ConditionalCheckFailed and leaves the item
    • ExecuteStatement - PartiQL DELETE with a true non-key predicate removes the item
    • ExecuteStatement - PartiQL UPDATE with a false non-key predicate fails ConditionalCheckFailed and leaves the item
    • ExecuteStatement - PartiQL UPDATE with a true non-key predicate mutates the item
    • ExecuteStatement - PartiQL DELETE with a false NOT begins_with predicate fails ConditionalCheckFailed
    • ExecuteStatement - PartiQL DELETE with a true NOT begins_with predicate removes the item
    • ExecuteStatement - PartiQL rejects a write WHERE clause that omits the primary key
    • ExecuteStatement - PartiQL DELETE on a missing key with a non-key predicate is a silent no-op
    • ExecuteStatement - PartiQL DELETE RETURNING ALL OLD * returns the deleted item
    • ExecuteStatement - PartiQL DELETE RETURNING ALL OLD * on a missing item returns an empty Items list
    • ExecuteStatement - PartiQL DELETE rejects RETURNING MODIFIED OLD * (only ALL OLD * is valid on DELETE)
    • ExecuteStatement - PartiQL DELETE rejects RETURNING MODIFIED NEW * (only ALL OLD * is valid on DELETE)
    • ExecuteStatement - PartiQL DELETE rejects RETURNING ALL NEW * with a 400, not a 500
    • ExecuteStatement - PartiQL UPDATE RETURNING ALL OLD * returns the full prior item
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * returns only the changed attribute (old value, no key)
    • ExecuteStatement - PartiQL UPDATE RETURNING ALL NEW * returns the full new item
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * returns only the changed attribute (new value, no key)
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * over a nested path returns only the changed leaf
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * over a nested path returns only the changed leaf at its old value
    • ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED OLD * returns the removed attribute at its old value
    • ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED NEW * returns an empty Items list
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * on a newly-set attribute returns an empty Items list
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * on a newly-set attribute returns the new value
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * on a list index returns only the changed element
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * on a list index returns only the prior element
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * on a non-zero list index returns only the changed element
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * on a non-zero list index returns the prior element
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * over multiple list indices returns a dense pack of the changed elements
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * over multiple list indices returns the prior elements densely
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * packs changed indices in ascending index order, not statement order
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * packs changed indices in ascending index order, not statement order
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * on an out-of-range list index (append) returns an empty Items list
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * on an out-of-range list index (append) returns an empty Items list
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * appending at exactly the list length returns the appended element
    • ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * appending at exactly the list length returns an empty Items list
    • ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED OLD * on a list index returns the removed element at its old value
    • ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED NEW * on a list index returns the shifted element, not an empty list
    • ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED NEW * on the last list index returns an empty Items list
    • ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED OLD * on the last list index returns the removed element
    • ExecuteStatement - PartiQL UPDATE SET on a list index of an absent attribute is rejected, not auto-created
    • ExecuteStatement - PartiQL UPDATE on a non-existent key fails ConditionalCheckFailed (PartiQL UPDATE is not an upsert)
    • ExecuteStatement - PartiQL rejects a statement with syntax error
    • ExecuteStatement - PartiQL rejects a reference to a non-existent table
    • ExecuteTransaction - PartiQL transactional INSERT and UPDATE both succeed atomically
    • ExecuteTransaction - PartiQL transaction rolls back on duplicate key INSERT
    • ExecuteTransaction - PartiQL multiple INSERTs in one transaction
    • ExecuteTransaction - PartiQL idempotent replay under the same ClientRequestToken does not double-apply
    • ExecuteTransaction - PartiQL rejects a RETURNING clause inside a transaction statement
    • ExecuteTransaction - PartiQL rejects empty TransactStatements
    • ExecuteStatement - index qualifier membership a qualified SELECT omits an item the index does not hold
    • ExecuteStatement - index qualifier projection follows the index, not the item a KEYS_ONLY GSI returns the index key and the table key and nothing else
    • ExecuteStatement - index qualifier projection follows the index, not the item an INCLUDE GSI adds the named attribute and no others
    • ExecuteStatement - index qualifier projection follows the index, not the item a KEYS_ONLY LSI returns the index key and the table key and nothing else
    • ExecuteStatement - index qualifier projection follows the index, not the item SELECT * against a non-ALL index does not reach back
    • ExecuteStatement - index qualifier the reach-back splits by index kind an LSI serves an unprojected attribute from the base table
    • ExecuteStatement - index qualifier the reach-back splits by index kind a GSI rejects an unprojected attribute rather than reaching back
    • ExecuteStatement - index qualifier the reach-back splits by index kind naming only projected attributes on an LSI charges no table arm
    • ExecuteStatement - index qualifier the reach-back splits by index kind a filtered reach-back is charged on rows walked, not rows kept
    • ExecuteStatement - index qualifier the reach-back splits by index kind a bounded reach-back walks fewer rows and is charged less
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition rejects a keyed GSI read filtering on an unprojected attribute
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition rejects the LSI mirror, so the rule is not about index kind
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition accepts the same GSI filter unkeyed, matching nothing
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition accepts the same LSI filter unkeyed, matching nothing
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition accepts two unprojected filters with no index key
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition accepts a keyed read filtering on an attribute the index does project
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition rejects a keyed read with IS MISSING on an unprojected attribute
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition rejects a keyed read with IS NOT MISSING on an unprojected attribute
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition rejects a keyed read with begins_with on an unprojected attribute
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition accepts the same predicate on a projected attribute
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition rejects a keyed read with a negated filter on an unprojected attribute
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition matches every index row when the negated filter names an absent attribute
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition counts IN on the index partition key as keyed
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition accepts an unprojected filter inside an OR branch
    • ExecuteStatement - index qualifier the unprojected-filter rejection turns on the key condition counts BETWEEN on the index sort key as keyed
    • ExecuteStatement - index qualifier pagination across a qualified read mints a token when a bounded read leaves rows behind
    • ExecuteStatement - index qualifier pagination across a qualified read rejects a token minted against one index and replayed against another
    • ExecuteStatement - index qualifier write statements reject the qualifier, and not all the same way rejects an INSERT carrying a qualifier at parse position
    • ExecuteStatement - index qualifier write statements reject the qualifier, and not all the same way rejects an INSERT whose qualifier names no index with the same message
    • ExecuteStatement - index qualifier write statements reject the qualifier, and not all the same way rejects an UPDATE carrying a qualifier semantically
    • ExecuteStatement - index qualifier write statements reject the qualifier, and not all the same way rejects a DELETE carrying a qualifier semantically
    • ExecuteStatement - index qualifier write statements reject the qualifier, and not all the same way leaves the rows untouched after the rejected writes
    • ExecuteStatement - index qualifier a transaction refuses an index-qualified read rejects it outright, not as a cancellation
    • ExecuteStatement - index qualifier a transaction refuses an index-qualified read accepts the same read unqualified
    • ExecuteStatement - index qualifier capacity lands on the index arm a keyed GSI read charges the index and a zero table arm
    • ExecuteStatement - index qualifier capacity lands on the index arm a keyed LSI read charges the index and a zero table arm
    • ExecuteStatement - index qualifier capacity lands on the index arm ConsistentRead doubles the LSI arm
    • ExecuteStatement - index qualifier capacity lands on the index arm ConsistentRead doubles both arms of a reach-back
    • ExecuteStatement - index qualifier capacity lands on the index arm rejects ConsistentRead against a GSI qualifier
  • error-messages Tier 3 partly 14 not attempted
    View these tests in the suite →
    • PartiQL - exact error messages DELETE RETURNING MODIFIED OLD * - exact message
    • PartiQL - exact error messages DELETE RETURNING ALL NEW * - exact message
    • PartiQL - exact error messages DELETE RETURNING MODIFIED NEW * - exact message
    • PartiQL - exact error messages ExecuteTransaction with a RETURNING member - exact message
    • PartiQL - exact error messages an ordering comparison on an unordered operand - exact message
    • PartiQL index qualifier - exact error messages unprojected filter attribute on a GSI - exact message
    • PartiQL index qualifier - exact error messages unprojected filter attribute on an LSI - same wording, no Local prefix
    • PartiQL index qualifier - exact error messages unprojected projection attribute on a GSI - exact message
    • PartiQL index qualifier - exact error messages a qualifier naming no index - exact message, with no name appended
    • PartiQL index qualifier - exact error messages a qualifier naming the table itself - same message
    • PartiQL index qualifier - exact error messages a three-component path - exact message
    • PartiQL index qualifier - exact error messages an empty index component - exact message
    • PartiQL index qualifier - exact error messages ConsistentRead against a GSI qualifier - exact message
    • PartiQL index qualifier - exact error messages an index-qualified read inside a transaction - exact message
  • validation-ordering Tier 3 partly 3 not attempted
    View these tests in the suite →
    • PartiQL - validation ordering the operand-type check fires ahead of table resolution
    • PartiQL - validation ordering a well-formed statement on the same table reports it missing
    • PartiQL - validation ordering an empty path component fires ahead of table resolution
  • contributorInsights Tier 2 none of it 2 not attempted
    View these tests in the suite →
    • Contributor insights - enable/describe/list reports DISABLED by default
    • Contributor insights - enable/describe/list enabling transitions the status and lists the table
  • export Tier 2 none of it 2 not attempted
    View these tests in the suite →
    • Export and import - S3 ExportTableToPointInTime initiates an export and reports it
    • Export and import - S3 ImportTable ingests S3 data into a new table
  • resourcePolicy Tier 2 none of it 2 not attempted
    View these tests in the suite →
    • Resource policies - Put/Get/Delete GetResourcePolicy on a table with no policy throws PolicyNotFoundException
    • Resource policies - Put/Get/Delete Put then Get round-trips the policy, and Delete removes it
  • kinesis Tier 2 none of it 1 not attempted
    View these tests in the suite →
    • Kinesis streaming destination enables a streaming destination and reports it via Describe

Run history

Every percentage here is divergence, per tier and over the whole suite, so lower is better in each column. Coverage is the exception it is named as.

Run Gradecurrent criteria Divergence Movement
Grade B 2.1% unchanged

Suite on

Thirteen captures against eu-west-2 turned into coverage, and the suite grew from 1056 tests to 1251. All of it was ground truth nothing asserted, so an engine could discard a PartiQL index qualifier or store an item AWS would refuse and still score a clean pass.

  • PartiQL index qualifier. What a qualified SELECT returns, when an unprojected attribute is rejected and when it is served from the base table, how the reach-back is charged, that a qualified write is rejected, and the exact wording of each rejection.
  • PartiQL comparison on non-scalar types. Equality and inequality on every set, map and list, each checked against the same predicate written as a ConditionExpression.
  • BatchExecuteStatement member fields. Per-member ConsistentRead, the primary-key requirement, and which failures echo TableName.
  • The 400KB gate, to the byte. 409,600 accepted and 409,601 refused across seven write surfaces. UpdateItem is the outlier: it charges what the statement writes plus a fixed cost per clause, so the key and every untouched attribute stay out of the figure.
  • A number's byte cost. Twenty-three literals, each measured by an accept and a refusal either side of the gate. The rule the developer guide documents is wrong in two ways, and both have a case.
  • Reads are sized before the filter. Scan, Query and PartiQL all charge before the WHERE clause and the projection, so discarding rows saves nothing.
  • Vector write capacity. VectorWriteRequestBytes pinned term by term, charged on the change to the index's own stored view like a GSI. VectorSearchRequestBytes stays shape-only: five identical searches against an unchanged index reported 14214, 13903, 14214, 14214 and 14518.
  • Vector index creation has two phases. Resource allocation, where the table sits in UPDATING, Backfilling reads false and a cancel is refused, then backfilling, which takes one. The one-online-index limit binds the table rather than the call.

Two fixes came with it. A local run against real DynamoDB takes its own table namespace, so it can no longer delete a CI run's tables. And a TransactWriteItems control, plus the whole PartiQL validation-ordering file, were feeding an unsupported fault into an assertion about wording and scoring a declared gap as a disagreement; both now skip.

BatchWriteItem with an empty RequestItems map is now a recorded regional split. eu-north-1 answers the validation framework's generic constraint message where the other 32 answering regions answer the bespoke required-parameter sentence. A capture on 2026-08-17 found every answering region on the bespoke wording, which dates the crossing to that week rather than guessing at it.

The over-25-requests assertion beside it spans both cohorts instead of becoming a second row. Neither wording is byte-stable: the table name carries a per-run suffix, the old cohort echoes all 26 requests back, and seven regions echo them as a JVM object identity rather than expanded fields. A row records one verbatim answer per region and there is nothing verbatim here to record, so the anchored pattern grew a second branch instead. It still refuses a wrong limit and a wrong table, and it matches every one of the 33 answering regions.

The capture harness can take that evidence without creating tables. --probes selects named probes and --no-tables skips the fixtures, which is all these three need: DynamoDB refuses each of them before it looks at a table, so the answers do not depend on one existing, and read-only credentials are enough. The 2026-08-17 capture was taken with a scoped script that was never committed; this is the committed way to reproduce it.

 		Grade B 2.1% diverged 1.9 percentage points more
Grade A 0.2% diverged 2.4 percentage points less
Grade B 2.6% unchanged
Grade B 2.6% diverged 0.9 percentage points more

Suite on

AWS corrected the vector index readiness documentation, prompted by a write-up of the earlier guidance that drew on the suite's measurements. The ACTIVE-plus-backfilling state the old advice was built around, and which no index ever occupies, is gone from the three pages that described it. The wait now reads "Backfilling is not true" rather than "is false", so a check written literally from it fires on both creation paths instead of neither. The tutorial no longer says a search during backfill can return incomplete results. Two things the suite had measured but nobody had written down are documented as well: that DescribeTable reporting ACTIVE leads the dedicated search endpoint, and that the readiness check depending on neither status field is a real search in a retry loop.

That contract is now pinned rather than described. The UpdateTable walk asserts that Backfilling true is only ever reported alongside CREATING, that an ACTIVE index reports no Backfilling field at all, and that the base table goes ACTIVE while the index is still building, which is what makes a table waiter the wrong gate for a search. The first search that succeeds has to carry every seeded item, since the backfill window answers with an error rather than a partial view. On the CreateTable path a new test runs the documented check the way an application would, and every rejection before the first served response has to be the retryable ValidationException rather than a not-found.

The suite's own search wait now absorbs those two rejections and rethrows every other answer, so a fixture waiting on an index that is ACTIVE but not yet served no longer fails on the lag it was waiting out. Two files asserting exact rejection messages wait for a served search rather than for ACTIVE: "does not have the specified index" is also what a freshly ACTIVE index says, and it would otherwise stand in for whichever message the case asked for.

The tutorial's other new claim, that a table cannot be deleted while a vector index is being created, has a test of its own. Only the UpdateTable path can ask it. Across three runs an index created with its table reached ACTIVE in the same 250ms poll as the table, so the table is never ACTIVE with the index still building there, and a DeleteTable during creation is refused for the table's own status rather than with the documented index wording. Adding an index to a live table opens that window about thirty seconds in. The test cancels the index afterwards instead of waiting out the backfill, which a still-creating vector index turns out to accept the way a backfilling GSI does, so it costs a minute rather than seventeen.

A release now dispatches its measurement as the results bot rather than with the workflow's own token. GitHub raises no workflow_run event when a run started by GITHUB_TOKEN finishes, so 3.2.0 measured green for three hours and its board only landed once the results table was dispatched by hand.

 		Grade B 1.6% diverged 0.2 percentage points less

Suite on

The board now measures the most recent release tag rather than main. Merging a test still runs it against real AWS, which is what validates it, but the published figures no longer move until a release moves them, so a dated changelog entry always sits behind a change in the denominator. Expect the figures to shift on this release: the board is switching from measuring main to measuring a tag, and those are different trees today.

Every board now says what produced it. results/summary.json carries a suite block naming the ref measured, its commit, the suite version at that ref, the region it ran against and when. It is additive, so schemaVersion stays at 1. The same block reaches /data/latest.json, /data/index.json and /data/runs.json, where each historical run carries its own copy, so a denominator that moved between two runs can be attributed to the release that moved it. Branch on kind: only tag is a released board.

A board is graded against the suite manifest and split registry as they stood at the ref it measured. Region health is the exception and is read live, so a region dropped since the tag still counts against today's cohorts. That is the one input allowed to move under a board without a new measurement, and the board carries a health date beside its measurement date to say so.

Releases are cut by one workflow dispatch: it bumps the version, dates this section, installs against the bumped tree, tags, and opens a draft release, then starts the measurement. The draft publishes itself when the board carrying that version lands, which takes about three hours.

eu-west-2 has crossed to the validation framework's generic constraint message for BatchGetItem with an empty RequestItems map, so the split row recording that behaviour is re-characterised against a 34-region capture taken on 2026-08-17. Eleven of the thirty-three regions that answer now return the generic message; twenty-two still return the bespoke required-parameter sentence, and me-central-1 joins the row.

The matching validation-ordering row is retired. Both wordings refuse an empty RequestItems before the map is read, which is all that tier asserts, so the assertion now matches the parameter name case-insensitively and spans both. The BatchWriteItem case beside it, which no region has moved yet, is written the same way.

A probe absent from the baseline is no longer reported as drift. Adding a probe to the capture script leaves every older baseline without it, and a scheduled red then named the new probe as the thing that had moved.

The weekly cross-region capture now includes eu-west-2, so the drift lens reads its baseline and the candidate regions from one capture taken at one moment rather than comparing today's candidates against an older baseline file. A scheduled red also keeps the eu-west-2 capture its drift verdict was read from, which was previously discarded with the runner.

 		Grade B 1.8% unchanged
Grade B 1.8% unchanged

Suite on

ExtendDB's SQLite backend joins the run, built from the same release as the PostgreSQL one and held to the same TLS, SigV4 and IAM posture, so the storage engine is the only thing that differs between them.

A project's other builds now sit behind a disclosure on its row. Every build is measured in full and has a row of its own with its own figures; the disclosure starts closed only when every build under it reads the same grade, divergence and coverage as the row above, and only when each of them was measured in that run: a carried row on either side opens it, and so does a run the suite declined to score. It is read from each run, so a build can start closed on one and open on the next. The README table has no disclosure to offer, so it lists every build outright.

Every target in the data endpoints gains two fields. collapsedIntoProject is true when the board starts that build's row closed. standsForProject says which row the board treats as a project's own, which isVariant cannot answer: on a run where a project's reference build recorded nothing, a build is promoted to stand for it and every row of that project reads isVariant: true. Both are additive, so schemaVersion is unchanged, and /data/index.json gains a projects block documenting them.

/data/index.json also gains a schema block saying what the version number entitles a consumer to: a field you already read will not change type or meaning while schemaVersion stays put, and new fields can appear at any version. The grading criteria are named there as a separate axis, since a change to the bands changes what a letter means without the schema moving.

The coverage description said a partial run was carried forward on its last clean measurement. It is not: the row stays in the run, publishes null for both figures and reads carried: false, because the target did report. Carrying forward is what happens to the baseline's unobserved lanes. Corrected, along with Dynoxide's build label, which now names its storage (native SQLite) like every other build in the registry rather than its compile target.

 		Grade B 1.8% first run