How to run it
Every way ExtendDB (SQLite) is distributed, each linking to the project's own documentation for it. These are the project's claims, not the suite's measurements.
Needs: a Rust toolchain to build it, and no external database
The PostgreSQL build's server over a single SQLite file rather than an external database. TLS and SigV4 stay mandatory.
Per-region divergence
The headline is 21 regions, measured against 21 of 33 observed regions, and scored in ap-east-1. Real DynamoDB doesn't behave identically in every region, so a target can match some regions more closely than others. Each figure below is divergence in those regions, best first. eu-west-2 is marked as the historical baseline, and a region that couldn't be resolved this sweep is flagged rather than counted as a disagreement.
Fully supports
Operation areas this target implements completely - every test passes, nothing skipped. Areas it only partly gets wrong are in “Where it falls short” below; the ones it only partly attempts are in “What it doesn’t attempt”.
- batchGetItem tier 1
- batchWriteItem tier 1
- createTable tier 1
- deleteItem tier 1
- deleteTable tier 1
- describeTable tier 1
- getItem tier 1
- listTables tier 1
- scan tier 1
- updateItem tier 1
- updateTable tier 1
- account tier 2
- streams tier 2
- tags tier 2
- ttl tier 2
- legacy-api tier 3
Divergence and coverage over time
Both are plotted because neither reads correctly alone. Divergence falls when a target stops attempting an operation it used to get wrong, so a fall in the first plot is only an improvement if the second one holds.
Divergence
Coverage
By operation
Every operation this target implements, with how much of it diverges and how much of it it covers, on the same two axes as the figures above. The counts are fails over the operation's whole size. full, partial, failing, unsupported. The matrix compares operations across targets.
Tier 1 - Core
- batchGetItem 0/15 0.0% 100.0% batchGetItem: supported, diverges on 0.0% of it, covers 100.0% (15 pass)
- batchWriteItem 0/18 0.0% 100.0% batchWriteItem: supported, diverges on 0.0% of it, covers 100.0% (18 pass)
- createTable 0/30 0.0% 100.0% createTable: supported, diverges on 0.0% of it, covers 100.0% (30 pass)
- deleteItem 0/13 0.0% 100.0% deleteItem: supported, diverges on 0.0% of it, covers 100.0% (13 pass)
- deleteTable 0/3 0.0% 100.0% deleteTable: supported, diverges on 0.0% of it, covers 100.0% (3 pass)
- describeTable 0/4 0.0% 100.0% describeTable: supported, diverges on 0.0% of it, covers 100.0% (4 pass)
- getItem 0/39 0.0% 100.0% getItem: supported, diverges on 0.0% of it, covers 100.0% (39 pass)
- listTables 0/5 0.0% 100.0% listTables: supported, diverges on 0.0% of it, covers 100.0% (5 pass)
- putItem 2/129 1.6% 100.0% putItem: partially supported, diverges on 1.6% of it, covers 100.0% (127 pass, 2 fail)
- query 2/92 2.2% 100.0% query: partially supported, diverges on 2.2% of it, covers 100.0% (90 pass, 2 fail)
- scan 0/56 0.0% 100.0% scan: supported, diverges on 0.0% of it, covers 100.0% (56 pass)
- updateItem 0/70 0.0% 100.0% updateItem: supported, diverges on 0.0% of it, covers 100.0% (70 pass)
- updateTable 0/15 0.0% 100.0% updateTable: supported, diverges on 0.0% of it, covers 100.0% (15 pass)
Tier 2 - Complete
- account 0/2 0.0% 100.0% account: supported, diverges on 0.0% of it, covers 100.0% (2 pass)
- backups 1/5 20.0% 100.0% backups: partially supported, diverges on 20.0% of it, covers 100.0% (4 pass, 1 fail)
- contributorInsights 0/2 · 2 skip n/a 0.0% contributorInsights: unsupported, implements none of it (2 skip)
- export 0/2 · 2 skip n/a 0.0% export: unsupported, implements none of it (2 skip)
- kinesis 0/1 · 1 skip n/a 0.0% kinesis: unsupported, implements none of it (1 skip)
- partiql 0/76 · 76 skip n/a 0.0% partiql: unsupported, implements none of it (76 skip)
- resourcePolicy 0/2 · 2 skip n/a 0.0% resourcePolicy: unsupported, implements none of it (2 skip)
- streams 0/18 0.0% 100.0% streams: supported, diverges on 0.0% of it, covers 100.0% (18 pass)
- tags 0/8 0.0% 100.0% tags: supported, diverges on 0.0% of it, covers 100.0% (8 pass)
- transactions 1/62 1.6% 100.0% transactions: partially supported, diverges on 1.6% of it, covers 100.0% (61 pass, 1 fail)
- ttl 0/7 0.0% 100.0% ttl: supported, diverges on 0.0% of it, covers 100.0% (7 pass)
- updateTable 2/14 14.3% 100.0% updateTable: partially supported, diverges on 14.3% of it, covers 100.0% (12 pass, 2 fail)
- vectorSearch 0/26 · 26 skip n/a 0.0% vectorSearch: unsupported, implements none of it (26 skip)
Tier 3 - Strict
- error-messages 8/174 · 20 skip 4.6% 88.5% error-messages: partially supported, diverges on 4.6% of it, covers 88.5% (146 pass, 8 fail, 20 skip)
- legacy-api 0/42 0.0% 100.0% legacy-api: supported, diverges on 0.0% of it, covers 100.0% (42 pass)
- limits 1/93 1.1% 100.0% limits: partially supported, diverges on 1.1% of it, covers 100.0% (92 pass, 1 fail)
- validation-ordering 3/31 9.7% 100.0% validation-ordering: partially supported, diverges on 9.7% of it, covers 100.0% (28 pass, 3 fail)
Where it falls short
Operations ExtendDB (SQLite) implements and answers differently from real DynamoDB, biggest gap first. This is what its divergence figure counts. Open one for the exact tests, or see the conformance suite.
-
error-messages Tier 3 8 diverging 20 not attempted
View these tests in the suite →- BatchGetItem - exact error messages empty RequestItems: full required-parameter error source:26
- BatchWriteItem - index key error messages empty-binary index key: full secondary-index-key message source:54
- PutItem - index key error messages empty-binary index key value: full secondary-index-key message source:126
- UpdateItem - index key error messages empty-binary index key value: full secondary-index-key message source:150
- PutItem - exact error messages duplicate zero-length members in BS: full duplicates error source:246
- Query - exact error messages Select SPECIFIC_ATTRIBUTES without ProjectionExpression: full required-projection message source:214
- TransactWriteItems - index key error messages Put empty-binary index key: top-level ValidationException source:77
- TransactWriteItems - index key error messages Update empty-binary index key: top-level ValidationException source:77
- PartiQL - exact error messages DELETE RETURNING MODIFIED OLD * - exact message
- PartiQL - exact error messages DELETE RETURNING ALL NEW * - exact message
- PartiQL - exact error messages DELETE RETURNING MODIFIED NEW * - exact message
- PartiQL - exact error messages ExecuteTransaction with a RETURNING member - exact message
- SearchVectors - exact error messages TopK above the maximum
- SearchVectors - exact error messages missing SearchConditionExpression against a HASH-schema index
- SearchVectors - exact error messages non-equality comparator on the HASH element
- SearchVectors - exact error messages non-equality comparator on an INLINE_FILTER element
- SearchVectors - exact error messages query vector dimension mismatch
- SearchVectors - exact error messages condition attribute outside the SearchSchema
- SearchVectors - exact error messages L-wrapped search vector
- CreateTable vector indexes - exact error messages vector index on a provisioned table
- CreateTable vector indexes - exact error messages SearchSchema attribute missing from AttributeDefinitions
- CreateTable vector indexes - exact error messages Dimensions above the 4096 maximum
- CreateTable vector indexes - exact error messages more vector indexes than the per-table limit
- CreateTable vector indexes - exact error messages two indexes on one attribute with different Dimensions
- PutItem vector writes - exact error messages vector with the wrong dimension count
- PutItem vector writes - exact error messages vector element that is not a number, naming the element
- PutItem vector writes - exact error messages vector attribute that is not a list
- PutItem vector writes - exact error messages empty string in the SearchSchema HASH attribute
-
validation-ordering Tier 3 3 diverging
View these tests in the suite →- DeleteItem - validation ordering reports invalid ReturnValues and invalid ReturnConsumedCapacity together source:47
- PutItem - validation ordering reports invalid ReturnConsumedCapacity, ReturnItemCollectionMetrics, and ReturnValues together source:78
- UpdateItem - validation ordering rejects invalid ReturnValues (UpdateItem reports the first enum error) source:53
-
putItem Tier 1 2 diverging
-
query Tier 1 2 diverging
-
updateTable Tier 2 2 diverging
View these tests in the suite →- UpdateTable - add GSI drops an unused AttributeDefinition supplied with a GSI add source:546
- UpdateTable - remove GSI removes a GSI from a table source:607
-
backups Tier 2 1 diverging
View these tests in the suite →- On-demand backups - lifecycle and restore RestoreTableFromBackup initiates a restore into a new table source:114
-
transactions Tier 2 1 diverging
View these tests in the suite →- TransactWriteItems - ConsumedCapacity: conditional, check, replay, cancel reports write capacity on the first call and read capacity on a same-token replay source:1420
-
limits Tier 3 1 diverging
View these tests in the suite →- Nesting depth - 32-level document limit rejects a 32-level ExpressionAttributeValue with ValidationException source:99
What it doesn't attempt
Operations ExtendDB (SQLite) declines rather than gets wrong, so none of this counts towards its divergence. It is what the gap in its 87.8% coverage is made of. Each test here skipped itself because the target's own feature probe said the operation isn't implemented, which is often a deliberate choice rather than a defect.
-
partiql Tier 2 none of it 76 not attempted
View these tests in the suite →- BatchExecuteStatement - PartiQL batch of multiple SELECT statements
- BatchExecuteStatement - PartiQL batch of INSERT and UPDATE statements
- BatchExecuteStatement - PartiQL partial failure - one valid and one invalid statement
- BatchExecuteStatement - PartiQL honours a RETURNING clause on a member statement
- BatchExecuteStatement - PartiQL honours a RETURNING ALL NEW * clause on a member UPDATE
- BatchExecuteStatement - PartiQL honours a RETURNING MODIFIED NEW * clause on a member UPDATE (only the changed attr)
- BatchExecuteStatement - PartiQL omits Item when a member UPDATE produces an empty MODIFIED projection
- BatchExecuteStatement - PartiQL surfaces an invalid RETURNING variant on a member DELETE as a per-statement error
- BatchExecuteStatement - PartiQL surfaces a malformed member statement as a per-statement ValidationError without failing the batch
- BatchExecuteStatement - PartiQL rejects an empty Statements array
- ExecuteStatement - PartiQL INSERTs a new item
- ExecuteStatement - PartiQL SELECTs an item by primary key
- ExecuteStatement - PartiQL SELECTs with WHERE clause using comparison
- ExecuteStatement - PartiQL UPDATEs an existing item
- ExecuteStatement - PartiQL DELETEs an item
- ExecuteStatement - PartiQL rejects INSERT on an existing item (INSERT is not upsert)
- ExecuteStatement - PartiQL INSERT succeeds after DELETE of same key
- ExecuteStatement - PartiQL SELECT returns empty results for non-matching WHERE
- ExecuteStatement - PartiQL parameterized INSERT with ? placeholders
- ExecuteStatement - PartiQL parameterized SELECT with ? placeholder
- ExecuteStatement - PartiQL SELECT with nested map path
- ExecuteStatement - PartiQL SELECT with specific attributes
- ExecuteStatement - PartiQL SELECT with begins_with in WHERE clause
- ExecuteStatement - PartiQL PartiQL UPDATE with SET on attribute
- ExecuteStatement - PartiQL PartiQL UPDATE with REMOVE
- ExecuteStatement - PartiQL returns a populated ConsumedCapacity block when requested
- ExecuteStatement - PartiQL evaluates negated predicates (NOT begins_with, IS NOT MISSING)
- ExecuteStatement - PartiQL DELETE with a false non-key predicate fails ConditionalCheckFailed and leaves the item
- ExecuteStatement - PartiQL DELETE with a true non-key predicate removes the item
- ExecuteStatement - PartiQL UPDATE with a false non-key predicate fails ConditionalCheckFailed and leaves the item
- ExecuteStatement - PartiQL UPDATE with a true non-key predicate mutates the item
- ExecuteStatement - PartiQL DELETE with a false NOT begins_with predicate fails ConditionalCheckFailed
- ExecuteStatement - PartiQL DELETE with a true NOT begins_with predicate removes the item
- ExecuteStatement - PartiQL rejects a write WHERE clause that omits the primary key
- ExecuteStatement - PartiQL DELETE on a missing key with a non-key predicate is a silent no-op
- ExecuteStatement - PartiQL DELETE RETURNING ALL OLD * returns the deleted item
- ExecuteStatement - PartiQL DELETE RETURNING ALL OLD * on a missing item returns an empty Items list
- ExecuteStatement - PartiQL DELETE rejects RETURNING MODIFIED OLD * (only ALL OLD * is valid on DELETE)
- ExecuteStatement - PartiQL DELETE rejects RETURNING MODIFIED NEW * (only ALL OLD * is valid on DELETE)
- ExecuteStatement - PartiQL DELETE rejects RETURNING ALL NEW * with a 400, not a 500
- ExecuteStatement - PartiQL UPDATE RETURNING ALL OLD * returns the full prior item
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * returns only the changed attribute (old value, no key)
- ExecuteStatement - PartiQL UPDATE RETURNING ALL NEW * returns the full new item
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * returns only the changed attribute (new value, no key)
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * over a nested path returns only the changed leaf
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * over a nested path returns only the changed leaf at its old value
- ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED OLD * returns the removed attribute at its old value
- ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED NEW * returns an empty Items list
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * on a newly-set attribute returns an empty Items list
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * on a newly-set attribute returns the new value
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * on a list index returns only the changed element
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * on a list index returns only the prior element
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * on a non-zero list index returns only the changed element
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * on a non-zero list index returns the prior element
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * over multiple list indices returns a dense pack of the changed elements
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * over multiple list indices returns the prior elements densely
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * packs changed indices in ascending index order, not statement order
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * packs changed indices in ascending index order, not statement order
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * on an out-of-range list index (append) returns an empty Items list
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * on an out-of-range list index (append) returns an empty Items list
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED NEW * appending at exactly the list length returns the appended element
- ExecuteStatement - PartiQL UPDATE RETURNING MODIFIED OLD * appending at exactly the list length returns an empty Items list
- ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED OLD * on a list index returns the removed element at its old value
- ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED NEW * on a list index returns the shifted element, not an empty list
- ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED NEW * on the last list index returns an empty Items list
- ExecuteStatement - PartiQL UPDATE REMOVE RETURNING MODIFIED OLD * on the last list index returns the removed element
- ExecuteStatement - PartiQL UPDATE SET on a list index of an absent attribute is rejected, not auto-created
- ExecuteStatement - PartiQL UPDATE on a non-existent key fails ConditionalCheckFailed (PartiQL UPDATE is not an upsert)
- ExecuteStatement - PartiQL rejects a statement with syntax error
- ExecuteStatement - PartiQL rejects a reference to a non-existent table
- ExecuteTransaction - PartiQL transactional INSERT and UPDATE both succeed atomically
- ExecuteTransaction - PartiQL transaction rolls back on duplicate key INSERT
- ExecuteTransaction - PartiQL multiple INSERTs in one transaction
- ExecuteTransaction - PartiQL idempotent replay under the same ClientRequestToken does not double-apply
- ExecuteTransaction - PartiQL rejects a RETURNING clause inside a transaction statement
- ExecuteTransaction - PartiQL rejects empty TransactStatements
-
vectorSearch Tier 2 none of it 26 not attempted
View these tests in the suite →- SearchVectors - ConsumedCapacity shape reports VectorSearchRequestBytes with no classic capacity fields
- SearchVectors - ConsumedCapacity shape reports the same shape under INDEXES as under TOTAL
- SearchVectors - ConsumedCapacity shape reports nothing under NONE
- PutItem - vector write capacity shape INDEXES adds a per-index VectorWriteRequestBytes map beside the classic fields
- PutItem - vector write capacity shape TOTAL folds nothing in: no vector fields at all
- PutItem - vector write capacity shape a write not touching the vector attribute reports no vector write
- PutItem - vector write capacity shape an identical overwrite reports no vector write: replication is delta-based
- CreateTable - vector index lifecycle walks CREATING to ACTIVE and describes the index faithfully
- CreateTable - vector index lifecycle round-trips a SearchSchema of HASH and INLINE_FILTER elements
- CreateTable - vector index lifecycle accepts the 4096-dimension boundary
- PartiQL - vector indexes are out of reach rejects a PartiQL read of a vector index
- PartiQL - vector indexes are out of reach reads the vector attribute off the base table like any other item
- SearchVectors - deterministic search behaviour COSINE: identical vector scores 0, opposite scores 2, lower is closer
- SearchVectors - deterministic search behaviour EUCLIDEAN: identical vector scores 0, straight-line distance otherwise
- SearchVectors - deterministic search behaviour DOT_PRODUCT: higher is closer and scores can be negative
- SearchVectors - deterministic search behaviour returns the identical vector as the top match
- SearchVectors - deterministic search behaviour omits the vector attribute by default and returns it when projected
- SearchVectors - deterministic search behaviour caps results at the item count with no pagination surface
- SearchVectors - deterministic search behaviour stores full precision on the base table and f32 in the index
- UpdateTable - vector index lifecycle adds an index that backfills like a GSI, enforces one online action, then deletes it
- CreateTable - vector index request validation rejects Dimensions of 0 at the request model layer
- CreateTable - vector index request validation rejects an index name shorter than 3 characters
- SearchVectors - request validation rejects TopK of 0 at the request model layer
- SearchVectors - request validation rejects a search against an index the table does not have
- PutItem - vector index write validation accepts a write missing the HASH attribute but leaves it unreachable through that index
- PutItem - vector index write validation removes an item from the index when its vector attribute is removed
-
contributorInsights Tier 2 none of it 2 not attempted
View these tests in the suite →- Contributor insights - enable/describe/list reports DISABLED by default
- Contributor insights - enable/describe/list enabling transitions the status and lists the table
-
export Tier 2 none of it 2 not attempted
View these tests in the suite →- Export and import - S3 ExportTableToPointInTime initiates an export and reports it
- Export and import - S3 ImportTable ingests S3 data into a new table
-
resourcePolicy Tier 2 none of it 2 not attempted
View these tests in the suite →- Resource policies - Put/Get/Delete GetResourcePolicy on a table with no policy throws PolicyNotFoundException
- Resource policies - Put/Get/Delete Put then Get round-trips the policy, and Delete removes it
-
kinesis Tier 2 none of it 1 not attempted
View these tests in the suite →- Kinesis streaming destination enables a streaming destination and reports it via Describe
Run history
Every percentage here is divergence, per tier and over the whole suite, so lower is better in each column. Coverage is the exception it is named as.