Skip to content

Dynalite

4.0.0 29 recorded runs

76.2%
correct, across 886 of 954 operations · 68 unsupported
Tier 1 91.4%
Tier 2 13.2%
Tier 3 70.9%

Per-region conformance

The headline is 27 regions. Real DynamoDB doesn't behave identically in every region, so a target can match some regions more closely than others. eu-west-2 is marked as the historical baseline, and a region that couldn't be resolved this sweep is flagged rather than counted as a disagreement.

76.2% 27 regions
af-south-1ap-east-1ap-northeast-1ap-northeast-3ap-south-1ap-south-2ap-southeast-2ap-southeast-3ap-southeast-4ap-southeast-5ap-southeast-6ap-southeast-7ca-central-1ca-west-1eu-central-2eu-south-1eu-south-2eu-west-1eu-west-3il-central-1me-central-1mx-central-1sa-east-1us-east-1us-east-2us-west-1us-west-2
76% 6 regions
ap-east-2ap-northeast-2ap-southeast-1eu-central-1eu-north-1eu-west-2 · baseline

Fully supports

Operation areas this target implements completely - every test passes, nothing skipped. Areas it only partly implements, or gets wrong, are in “Where it falls short” below.

  • batchGetItem tier 1
  • deleteItem tier 1
  • deleteTable tier 1
  • describeTable tier 1
  • getItem tier 1
  • listTables tier 1

Conformance over time

Dynalite conformance history, 2026-03-23 to 2026-07-18 70% 85% 100% 23 Mar 25 Apr 23 May 25 May 29 May 16 Jun 19 Jun 22 Jun 24 Jun 29 Jun 1 Jul 6 Jul 14 Jul 16 Jul 18 Jul 76.2%
Now 76.2% on 18 Jul, lowest 75.2% on 1 Jul.

By operation

Every operation this target implements, with the share of its tests that pass. full, partial, failing, unsupported. The matrix compares operations across targets.

Tier 1 - Core

  • batchGetItem 15/15 100.0% batchGetItem: supported (15 pass)
  • batchWriteItem 17/18 94.4% batchWriteItem: partially supported (17 pass, 1 fail)
  • createTable 23/30 76.7% createTable: partially supported (23 pass, 7 fail)
  • deleteItem 13/13 100.0% deleteItem: supported (13 pass)
  • deleteTable 3/3 100.0% deleteTable: supported (3 pass)
  • describeTable 4/4 100.0% describeTable: supported (4 pass)
  • getItem 40/40 100.0% getItem: supported (40 pass)
  • listTables 5/5 100.0% listTables: supported (5 pass)
  • putItem 101/116 87.1% putItem: partially supported (101 pass, 15 fail)
  • query 85/90 94.4% query: partially supported (85 pass, 5 fail)
  • scan 50/56 89.3% scan: partially supported (50 pass, 6 fail)
  • updateItem 68/70 97.1% updateItem: partially supported (68 pass, 2 fail)
  • updateTable 10/15 66.7% updateTable: partially supported (10 pass, 5 fail)

Tier 2 - Complete

  • account 0/0 · 2 skip n/a account: unsupported (2 skip)
  • backups 0/0 · 5 skip n/a backups: unsupported (5 skip)
  • contributorInsights 0/0 · 2 skip n/a contributorInsights: unsupported (2 skip)
  • export 0/0 · 2 skip n/a export: unsupported (2 skip)
  • kinesis 0/0 · 1 skip n/a kinesis: unsupported (1 skip)
  • partiql 0/0 · 36 skip n/a partiql: unsupported (36 skip)
  • resourcePolicy 0/0 · 2 skip n/a resourcePolicy: unsupported (2 skip)
  • streams 0/0 · 18 skip n/a streams: unsupported (18 skip)
  • tags 6/8 75.0% tags: partially supported (6 pass, 2 fail)
  • transactions 0/62 0.0% transactions: failing (62 fail)
  • ttl 2/7 28.6% ttl: partially supported (2 pass, 5 fail)
  • updateTable 4/14 28.6% updateTable: partially supported (4 pass, 10 fail)

Tier 3 - Strict

  • error-messages 88/154 57.1% error-messages: partially supported (88 pass, 66 fail)
  • legacy-api 41/42 97.6% legacy-api: partially supported (41 pass, 1 fail)
  • limits 77/93 82.8% limits: partially supported (77 pass, 16 fail)
  • validation-ordering 21/31 67.7% validation-ordering: partially supported (21 pass, 10 fail)

Where it falls short

Operation groups with gaps in the latest run, biggest first. Unsupported means the target doesn't implement that feature at all, often by design. Open one for the exact tests, or see the conformance suite.

  • error-messages Tier 3 66 failing
    View these tests in the suite →
    • BatchGetItem - exact error messages empty RequestItems: full required-parameter error
    • BatchGetItem - exact error messages > 100 keys across all tables: interpolated full error
    • BatchGetItem - exact error messages empty-binary lookup key: full empty-value message
    • BatchGetItem - exact error messages mixing ProjectionExpression on one table and AttributesToGet on another is rejected
    • BatchWriteItem - exact error messages empty RequestItems: full required-parameter error
    • BatchWriteItem - exact error messages > 25 requests: anchored regex on the constraint phrase
    • BatchWriteItem - exact error messages empty-binary delete-request key: full empty-value message
    • BatchWriteItem - exact error messages empty-string index key: full secondary-index-key message
    • BatchWriteItem - exact error messages empty-binary index key: full secondary-index-key message
    • Conditional check - exact error messages TransactionCanceledException message format
    • Conditional check - exact error messages ReturnValuesOnConditionCheckFailure: ALL_OLD returns item in error
    • CreateTable - exact error messages short table name (2 chars): minimum length 3 error
    • CreateTable - exact error messages more than 2 KeySchema elements
    • CreateTable - exact error messages GSI INCLUDE projection without NonKeyAttributes: full missing-attributes message
    • CreateTable - exact error messages LSI INCLUDE projection without NonKeyAttributes: full missing-attributes message
    • CreateTable - exact error messages StreamSpecification StreamEnabled:false with a StreamViewType: full conflict message
    • DeleteItem - exact error messages empty-binary key value: full ValidationException message
    • GetItem - exact error messages invalid ProjectionExpression syntax: full parser error
    • GetItem - exact error messages empty-binary key value: full ValidationException message
    • PutItem - exact error messages empty table name: minimum length 1 error
    • PutItem - exact error messages duplicate empty string members in SS: full duplicates error
    • PutItem - exact error messages duplicate zero-length members in BS: full duplicates error
    • PutItem - exact error messages empty-binary index key value: full secondary-index-key message
    • Query - exact error messages Limit of 0
    • Query - exact error messages Select SPECIFIC_ATTRIBUTES without ProjectionExpression: full required-projection message
    • Scan - exact error messages Segment without TotalSegments: full required-parameter error
    • Scan - exact error messages ConsistentRead on a GSI: full consistent-reads-unsupported message
    • Scan - exact error messages Select SPECIFIC_ATTRIBUTES without ProjectionExpression: full required-projection message
    • TransactGetItems - exact error messages empty TransactItems: full minimum-length error
    • TransactGetItems - exact error messages > 100 gets: anchored regex on the constraint phrase
    • TransactGetItems - exact error messages non-existent table: full ResourceNotFoundException message
    • TransactGetItems - exact error messages invalid ProjectionExpression syntax: full parser error
    • TransactGetItems - exact error messages missing key attribute: action-level ValidationError surfaces as TransactionCanceledException
    • TransactWriteItems - exact error messages empty TransactItems: full minimum-length error
    • TransactWriteItems - exact error messages > 100 actions: anchored regex on the constraint phrase
    • TransactWriteItems - exact error messages duplicate target keys in same transaction: full multi-op error
    • TransactWriteItems - exact error messages non-existent table: full ResourceNotFoundException message
    • TransactWriteItems - exact error messages two failing actions: multi-reason TransactionCanceledException
    • TransactWriteItems - exact error messages one passing, one failing: positional reason codes (None for the survivor)
    • TransactWriteItems - exact error messages Put wrong-typed table key: cancelled with full ValidationError reason
    • TransactWriteItems - exact error messages Put non-scalar table key: cancelled with full ValidationError reason
    • TransactWriteItems - exact error messages Put wrong-typed index key: cancelled with full ValidationError reason
    • TransactWriteItems - exact error messages Put non-scalar index key: cancelled with full ValidationError reason
    • TransactWriteItems - exact error messages Update wrong-typed index key: cancelled with full ValidationError reason
    • TransactWriteItems - exact error messages Update non-scalar index key: cancelled with full ValidationError reason
    • TransactWriteItems - exact error messages Put empty-string table key: top-level ValidationException
    • TransactWriteItems - exact error messages Put empty-string index key: top-level ValidationException
    • TransactWriteItems - exact error messages Update empty-string index key: top-level ValidationException
    • TransactWriteItems - exact error messages Update empty-string Key: top-level empty-value message
    • TransactWriteItems - exact error messages Delete empty-string Key: top-level empty-value message
    • TransactWriteItems - exact error messages ConditionCheck empty-string Key: top-level empty-value message
    • TransactWriteItems - exact error messages Update wrong-typed Key: cancelled with schema-mismatch reason
    • TransactWriteItems - exact error messages Update non-scalar Key: cancelled with schema-mismatch reason
    • TransactWriteItems - exact error messages Delete wrong-typed Key: cancelled with schema-mismatch reason
    • TransactWriteItems - exact error messages Delete non-scalar Key: cancelled with schema-mismatch reason
    • TransactWriteItems - exact error messages ConditionCheck wrong-typed Key: cancelled with schema-mismatch reason
    • TransactWriteItems - exact error messages ConditionCheck non-scalar Key: cancelled with schema-mismatch reason
    • TransactWriteItems - exact error messages Put empty-binary item key: top-level empty-value message
    • TransactWriteItems - exact error messages Update empty-binary Key: top-level empty-value message
    • TransactWriteItems - exact error messages Delete empty-binary Key: top-level empty-value message
    • TransactWriteItems - exact error messages ConditionCheck empty-binary Key: top-level empty-value message
    • TransactWriteItems - exact error messages Put empty-binary index key: top-level ValidationException
    • TransactWriteItems - exact error messages Update empty-binary index key: top-level ValidationException
    • UpdateItem - exact error messages invalid UpdateExpression syntax
    • UpdateItem - exact error messages empty-binary key value: full ValidationException message
    • UpdateItem - exact error messages empty-binary index key value: full secondary-index-key message
  • transactions Tier 2 62 failing
    View these tests in the suite →
    • TransactGetItems - basic gets multiple items atomically
    • TransactGetItems - basic gets items across tables (hashTableDef + compositeTableDef)
    • TransactGetItems - basic returns empty Item for non-existing items in the mix
    • TransactGetItems - basic gets a single item
    • TransactGetItems - basic returns all items when all exist
    • TransactGetItems - validation rejects empty TransactItems
    • TransactGetItems - validation rejects get on non-existent table
    • TransactGetItems - validation rejects duplicate keys in same transaction
    • TransactGetItems - ConsumedCapacity charges 2 read capacity units per item, including a missing item
    • TransactGetItems - ConsumedCapacity INDEXES breakdown includes the table read capacity units
    • TransactGetItems - projection matching nothing omits Item when the projection matches no attribute on a present item
    • TransactWriteItems - basic functionality executes Put + Update + Delete atomically
    • TransactWriteItems - basic functionality commits a Put whose string set carries an empty member
    • TransactWriteItems - basic functionality succeeds when ConditionCheck condition is met
    • TransactWriteItems - basic functionality rolls back entire transaction when ConditionCheck fails
    • TransactWriteItems - basic functionality applies ConditionExpression on Put action
    • TransactWriteItems - basic functionality applies ConditionExpression on Update action
    • TransactWriteItems - basic functionality applies ConditionExpression on Delete action
    • TransactWriteItems - basic functionality executes cross-table transaction (hashTableDef + compositeTableDef)
    • TransactWriteItems - basic functionality supports idempotency via ClientRequestToken
    • TransactWriteItems - basic functionality rejects same ClientRequestToken with different payload
    • TransactWriteItems - basic functionality includes CancellationReasons in error when condition fails
    • TransactWriteItems - basic functionality returns ALL_OLD item via ReturnValuesOnConditionCheckFailure
    • TransactWriteItems - multiple items puts multiple items in one transaction
    • TransactWriteItems - multiple items updates multiple items in one transaction
    • TransactWriteItems - multiple items deletes multiple items in one transaction
    • TransactWriteItems - validation rejects duplicate target items in same transaction
    • TransactWriteItems - validation rejects empty TransactItems
    • TransactWriteItems - validation rejects transaction on non-existent table
    • TransactWriteItems - validation Put with a wrong-typed table key cancels with a ValidationError reason
    • TransactWriteItems - validation Put with a non-scalar table key cancels with a ValidationError reason
    • TransactWriteItems - validation Put with a wrong-typed index key cancels with a ValidationError reason
    • TransactWriteItems - validation Put with a non-scalar index key cancels with a ValidationError reason
    • TransactWriteItems - validation Update setting a wrong-typed index key cancels with a ValidationError reason
    • TransactWriteItems - validation Update setting a non-scalar index key cancels with a ValidationError reason
    • TransactWriteItems - validation Put with an empty-string table key is a top-level ValidationException
    • TransactWriteItems - validation Put with an empty-string index key is a top-level ValidationException
    • TransactWriteItems - validation Update setting an empty-string index key is a top-level ValidationException
    • TransactWriteItems - validation Update with an empty-string Key is a top-level ValidationException
    • TransactWriteItems - validation Delete with an empty-string Key is a top-level ValidationException
    • TransactWriteItems - validation ConditionCheck with an empty-string Key is a top-level ValidationException
    • TransactWriteItems - validation Update with a wrong-typed Key cancels with a ValidationError reason
    • TransactWriteItems - validation Update with a non-scalar Key cancels with a ValidationError reason
    • TransactWriteItems - validation Delete with a wrong-typed Key cancels with a ValidationError reason
    • TransactWriteItems - validation Delete with a non-scalar Key cancels with a ValidationError reason
    • TransactWriteItems - validation ConditionCheck with a wrong-typed Key cancels with a ValidationError reason
    • TransactWriteItems - validation ConditionCheck with a non-scalar Key cancels with a ValidationError reason
    • TransactWriteItems - validation Update with attribute_exists rejects non-existent item
    • TransactWriteItems - validation Update with attribute_not_exists upserts on non-existent key
    • TransactWriteItems - validation Update with comparison condition cancels on non-existent key; no ghost item
    • TransactWriteItems - validation Update with combined attribute_exists + equality cancels on non-existent key; no ghost item
    • TransactWriteItems - validation mixed transaction: one passing, one failing on non-existent cancels everything
    • TransactWriteItems - ConditionExpression parens Put with per-condition parens succeeds on fresh key
    • TransactWriteItems - ConditionExpression parens Update with full-expression wrap succeeds when condition holds
    • TransactWriteItems - ConditionExpression parens Delete with non-redundant nested parens succeeds when condition holds
    • TransactWriteItems - ConditionExpression parens ConditionCheck with per-condition parens passes when condition holds
    • TransactWriteItems - ConditionExpression parens cancels transaction when any parenthesised condition fails
    • TransactWriteItems - ConsumedCapacity charges 2 write capacity units per item
    • TransactWriteItems - ConsumedCapacity: conditional, check, replay, cancel a passing conditional write costs the same 2 WCU/item as an unconditional one
    • TransactWriteItems - ConsumedCapacity: conditional, check, replay, cancel a standalone ConditionCheck action costs 2 write capacity units
    • TransactWriteItems - ConsumedCapacity: conditional, check, replay, cancel reports write capacity on the first call and read capacity on a same-token replay
    • TransactWriteItems - ConsumedCapacity: conditional, check, replay, cancel a cancelled conditional transaction reports no consumed capacity
  • partiql Tier 2 unsupported
    View these tests in the suite →
    • BatchExecuteStatement - PartiQL batch of multiple SELECT statements
    • BatchExecuteStatement - PartiQL batch of INSERT and UPDATE statements
    • BatchExecuteStatement - PartiQL partial failure - one valid and one invalid statement
    • BatchExecuteStatement - PartiQL rejects an empty Statements array
    • ExecuteStatement - PartiQL INSERTs a new item
    • ExecuteStatement - PartiQL SELECTs an item by primary key
    • ExecuteStatement - PartiQL SELECTs with WHERE clause using comparison
    • ExecuteStatement - PartiQL UPDATEs an existing item
    • ExecuteStatement - PartiQL DELETEs an item
    • ExecuteStatement - PartiQL rejects INSERT on an existing item (INSERT is not upsert)
    • ExecuteStatement - PartiQL INSERT succeeds after DELETE of same key
    • ExecuteStatement - PartiQL SELECT returns empty results for non-matching WHERE
    • ExecuteStatement - PartiQL parameterized INSERT with ? placeholders
    • ExecuteStatement - PartiQL parameterized SELECT with ? placeholder
    • ExecuteStatement - PartiQL SELECT with nested map path
    • ExecuteStatement - PartiQL SELECT with specific attributes
    • ExecuteStatement - PartiQL SELECT with begins_with in WHERE clause
    • ExecuteStatement - PartiQL PartiQL UPDATE with SET on attribute
    • ExecuteStatement - PartiQL PartiQL UPDATE with REMOVE
    • ExecuteStatement - PartiQL returns a populated ConsumedCapacity block when requested
    • ExecuteStatement - PartiQL evaluates negated predicates (NOT begins_with, IS NOT MISSING)
    • ExecuteStatement - PartiQL DELETE with a false non-key predicate fails ConditionalCheckFailed and leaves the item
    • ExecuteStatement - PartiQL DELETE with a true non-key predicate removes the item
    • ExecuteStatement - PartiQL UPDATE with a false non-key predicate fails ConditionalCheckFailed and leaves the item
    • ExecuteStatement - PartiQL UPDATE with a true non-key predicate mutates the item
    • ExecuteStatement - PartiQL DELETE with a false NOT begins_with predicate fails ConditionalCheckFailed
    • ExecuteStatement - PartiQL DELETE with a true NOT begins_with predicate removes the item
    • ExecuteStatement - PartiQL rejects a write WHERE clause that omits the primary key
    • ExecuteStatement - PartiQL DELETE on a missing key with a non-key predicate is a silent no-op
    • ExecuteStatement - PartiQL rejects a statement with syntax error
    • ExecuteStatement - PartiQL rejects a reference to a non-existent table
    • ExecuteTransaction - PartiQL transactional INSERT and UPDATE both succeed atomically
    • ExecuteTransaction - PartiQL transaction rolls back on duplicate key INSERT
    • ExecuteTransaction - PartiQL multiple INSERTs in one transaction
    • ExecuteTransaction - PartiQL idempotent replay under the same ClientRequestToken does not double-apply
    • ExecuteTransaction - PartiQL rejects empty TransactStatements
  • streams Tier 2 unsupported
    View these tests in the suite →
    • DynamoDB Streams - basic table with StreamSpecification has LatestStreamArn in DescribeTable
    • DynamoDB Streams - basic table StreamSpecification.StreamEnabled is true
    • DynamoDB Streams - basic table StreamSpecification.StreamViewType matches what was requested
    • DynamoDB Streams - basic ListStreams returns the stream for our table
    • DynamoDB Streams - basic ListStreams with TableName filter returns only our table's stream
    • DynamoDB Streams - basic DescribeStream returns stream status ENABLED or ENABLING
    • DynamoDB Streams - basic DescribeStream returns at least one shard
    • DynamoDB Streams - basic each shard has a ShardId
    • DynamoDB Streams - basic TRIM_HORIZON returns a valid iterator string
    • DynamoDB Streams - basic LATEST returns a valid iterator string
    • DynamoDB Streams - basic GetRecords after PutItem contains the new image (INSERT event)
    • DynamoDB Streams - basic INSERT record has eventName INSERT
    • DynamoDB Streams - basic INSERT record dynamodb.Keys contains the item key
    • DynamoDB Streams - basic INSERT record dynamodb.NewImage contains the full item
    • DynamoDB Streams - basic GetRecords after UpdateItem contains both old and new images (MODIFY event)
    • DynamoDB Streams - basic GetRecords after DeleteItem contains old image (REMOVE event)
    • DynamoDB Streams - basic NEW_IMAGE view type records have NewImage but no OldImage on MODIFY
    • DynamoDB Streams - basic KEYS_ONLY view type records have Keys but no NewImage or OldImage
  • limits Tier 3 16 failing
    View these tests in the suite →
    • Empty values - strings, binary, and sets duplicate empty string members in SS are rejected as duplicates
    • Empty values - strings, binary, and sets duplicate zero-length members in BS are rejected as duplicates
    • Expression size limit (4KB) - UpdateExpression rejects an UpdateExpression over the 4096-byte limit
    • Expression size limit (4KB) - ConditionExpression rejects a ConditionExpression over the 4096-byte limit
    • Expression size limit (4KB) - Query FilterExpression rejects a Query FilterExpression over the 4096-byte limit
    • Expression size limit (4KB) - Scan FilterExpression rejects a Scan FilterExpression over the 4096-byte limit
    • Expression size limit (4KB) - ProjectionExpression rejects a ProjectionExpression over the 4096-byte limit
    • Nesting depth - 32-level document limit rejects a stored attribute nested 32 levels (leaf at level 33)
    • Nesting depth - 32-level document limit rejects a 32-level ExpressionAttributeValue with ValidationException
    • Number precision - DynamoDB number limits and edge cases rejects arithmetic that overflows the supported number magnitude
    • TransactWriteItems limits TransactWriteItems with exactly 100 Put actions succeeds
    • TransactWriteItems limits TransactWriteItems with 101 actions fails with ValidationException
    • TransactWriteItems limits TransactWriteItems total item size approaching 4MB succeeds
    • TransactWriteItems limits TransactWriteItems total item size over 4MB fails with ValidationException
    • TransactGetItems limits TransactGetItems with exactly 100 items succeeds
    • TransactGetItems limits TransactGetItems with 101 items fails with ValidationException
  • putItem Tier 1 15 failing
    View these tests in the suite →
    • PutItem - ConditionExpression returns the conflicting item via ReturnValuesOnConditionCheckFailure on a failed Put
    • ReturnItemCollectionMetrics PutItem with SIZE returns ItemCollectionMetrics
    • ReturnItemCollectionMetrics DeleteItem with SIZE returns ItemCollectionMetrics
    • ReturnItemCollectionMetrics UpdateItem with SIZE returns ItemCollectionMetrics
    • PutItem - number format accepts "+5" and stores it as "5"
    • PutItem - number format accepts "+1.5" and stores it as "1.5"
    • PutItem - number format accepts "+0" and stores it as "0"
    • PutItem - number format accepts "+0.0" and stores it as "0"
    • PutItem - number format accepts "+1e2" and stores it as "100"
    • PutItem - number format accepts "+.5" and stores it as "0.5"
    • PutItem - number format accepts "+1e-2" and stores it as "0.01"
    • PutItem - number format accepts a leading + on a numeric sort key and normalises it
    • PutItem - validation rejects duplicate empty string members in a string set
    • PutItem - validation rejects duplicate zero-length members in a binary set
    • PutItem - validation rejects an empty-string index key attribute
  • updateTable Tier 2 10 failing
    View these tests in the suite →
    • UpdateTable - add GSI adds a hash-only GSI to an existing table
    • UpdateTable - add GSI adds a composite GSI (hash + range) to an existing table
    • UpdateTable - add GSI adds a GSI with KEYS_ONLY projection
    • UpdateTable - add GSI adds a GSI with INCLUDE projection and NonKeyAttributes
    • UpdateTable - add GSI can query a newly created GSI after putting items
    • UpdateTable - add GSI adds multiple GSIs sequentially
    • UpdateTable - add GSI accepts a conflicting redeclaration of an existing key attribute and keeps the stored type
    • UpdateTable - add GSI drops an unused AttributeDefinition supplied with a GSI add
    • UpdateTable - remove GSI removes a GSI from a table
    • UpdateTable - GSI validation rejects removing a non-existent GSI
  • validation-ordering Tier 3 10 failing
    View these tests in the suite →
    • CreateTable - validation ordering empty TableName reports only tableName constraint
    • CreateTable - validation ordering invalid table name pattern reports only tableName
    • DeleteItem - validation ordering empty TableName reports only tableName constraint
    • Non-existent index - error surface UpdateTable deleting a non-existent GSI reports ResourceNotFoundException
    • PutItem - validation ordering empty TableName reports tableName constraint (stops early)
    • PutItem - validation ordering empty TableName with invalid ReturnValues reports only tableName
    • Query - validation ordering reports invalid Limit value
    • Scan - validation ordering rejects Segment without TotalSegments
    • UpdateItem - validation ordering empty TableName reports only tableName constraint
    • UpdateItem - validation ordering rejects invalid ReturnValues (UpdateItem reports the first enum error)
  • createTable Tier 1 7 failing
    View these tests in the suite →
    • CreateTable - validation rejects a GSI INCLUDE projection without NonKeyAttributes
    • CreateTable - validation rejects StreamSpecification with StreamEnabled false plus a StreamViewType
    • CreateTable - configuration parameters DeletionProtectionEnabled round-trips
    • CreateTable - configuration parameters TableClass STANDARD_INFREQUENT_ACCESS round-trips
    • CreateTable - configuration parameters SSESpecification with the AWS-managed key round-trips
    • CreateTable - configuration parameters OnDemandThroughput round-trips on a PAY_PER_REQUEST table
    • CreateTable - LSI rejects an LSI INCLUDE projection without NonKeyAttributes
  • scan Tier 1 6 failing
    View these tests in the suite →
    • Scan - Select / ProjectionExpression rejections Select ALL_ATTRIBUTES with ProjectionExpression is rejected
    • Scan - Select / ProjectionExpression rejections Select ALL_PROJECTED_ATTRIBUTES without an IndexName is rejected
    • Scan - Select / ProjectionExpression rejections Select COUNT with ProjectionExpression is rejected
    • Scan - Select / ProjectionExpression rejections Select ALL_PROJECTED_ATTRIBUTES with ProjectionExpression and no IndexName is rejected
    • Scan - Select / ProjectionExpression rejections Select SPECIFIC_ATTRIBUTES without ProjectionExpression is rejected
    • Scan - validation rejects TotalSegments above the maximum
  • query Tier 1 5 failing
    View these tests in the suite →
    • Query - Select SPECIFIC_ATTRIBUTES Select SPECIFIC_ATTRIBUTES without ProjectionExpression is rejected
    • Query - Select / ProjectionExpression rejections Select ALL_ATTRIBUTES with ProjectionExpression is rejected
    • Query - Select / ProjectionExpression rejections Select ALL_PROJECTED_ATTRIBUTES without an IndexName is rejected
    • Query - Select / ProjectionExpression rejections Select COUNT with ProjectionExpression is rejected
    • Query - Select / ProjectionExpression rejections Select ALL_PROJECTED_ATTRIBUTES with ProjectionExpression and no IndexName is rejected
  • updateTable Tier 1 5 failing
    View these tests in the suite →
    • UpdateTable - configuration parameters UpdateTable enables DeletionProtectionEnabled
    • UpdateTable - configuration parameters UpdateTable changes TableClass
    • UpdateTable - configuration parameters UpdateTable changes OnDemandThroughput
    • DeleteTable - deletion protection enforcement rejects DeleteTable while deletion protection is enabled
    • DeleteTable - deletion protection enforcement allows DeleteTable after protection is disabled
  • backups Tier 2 unsupported
    View these tests in the suite →
    • Continuous backups - PITR reports PITR DISABLED by default
    • Continuous backups - PITR enabling PITR transitions PointInTimeRecoveryStatus to ENABLED
    • On-demand backups - lifecycle and restore CreateBackup → DescribeBackup → ListBackups → DeleteBackup
    • On-demand backups - lifecycle and restore RestoreTableFromBackup initiates a restore into a new table
    • On-demand backups - lifecycle and restore DescribeBackup on a deleted backup throws BackupNotFoundException
  • ttl Tier 2 5 failing
    View these tests in the suite →
    • TTL - basic enables TTL on a table
    • TTL - basic DescribeTimeToLive returns ENABLED status and correct attribute name after enabling
    • TTL - basic enables TTL with a different attribute name
    • TTL - validation rejects empty attribute name
    • TTL - validation rejects UpdateTimeToLive on non-existent table
  • updateItem Tier 1 2 failing
    View these tests in the suite →
    • UpdateItem - ReturnValues granularity REMOVE with UPDATED_NEW omits Attributes (nothing was set to a new value)
    • UpdateItem - ConditionExpression ReturnValuesOnConditionCheckFailure ALL_OLD returns existing item on failure
  • account Tier 2 unsupported
    View these tests in the suite →
    • Account reads - DescribeLimits, DescribeEndpoints DescribeLimits returns positive account and table capacity limits
    • Account reads - DescribeLimits, DescribeEndpoints DescribeEndpoints returns at least one endpoint with an address
  • contributorInsights Tier 2 unsupported
    View these tests in the suite →
    • Contributor insights - enable/describe/list reports DISABLED by default
    • Contributor insights - enable/describe/list enabling transitions the status and lists the table
  • export Tier 2 unsupported
    View these tests in the suite →
    • Export and import - S3 ExportTableToPointInTime initiates an export and reports it
    • Export and import - S3 ImportTable ingests S3 data into a new table
  • resourcePolicy Tier 2 unsupported
    View these tests in the suite →
    • Resource policies - Put/Get/Delete GetResourcePolicy on a table with no policy throws PolicyNotFoundException
    • Resource policies - Put/Get/Delete Put then Get round-trips the policy, and Delete removes it
  • tags Tier 2 2 failing
    View these tests in the suite →
    • Tags - validation rejects TagResource with an invalid ARN format
    • Tags - validation rejects ListTagsOfResource with a non-existent ARN
  • batchWriteItem Tier 1 1 failing
    View these tests in the suite →
    • BatchWriteItem - validation rejects an empty-string index key value
  • kinesis Tier 2 unsupported
    View these tests in the suite →
    • Kinesis streaming destination enables a streaming destination and reports it via Describe
  • legacy-api Tier 3 1 failing
    View these tests in the suite →
    • Legacy API - KeyConditions and QueryFilter QueryFilter with ComparisonOperator EQ on a BOOL attribute filters correctly

Run history

Run Total Movement
76.2% 0.0pp unchanged

Suite on

Per-region scoring lands complete. 2.0.0-pre put the scoring logic in place, comparing each target against every region's recorded answer, but the evidence half was never wired: no test recorded what a target actually answered and the classifier never read one, so a fail could not be credited to a region the target matched and the score could only ever subtract. 2.0.0 closes that loop, and the seed split runs its whole lifecycle in the same release.

What changed:

  • The split tests now record what the target actually answered (src/observation-sink.ts), in the same shape the registry stores each region's answer, and the classifier carries it onto the verdict. An engine that matches a rejecting region on a split is now scored as passing in that region. Evidence only ever redeems a committed fail: a pass keeps the committed assertion's deliberate wording tolerance and is never held to the byte-exact recorded string. Committed results predate the capture, so published scores move on each target's next run, not in this change.
  • Headline ties now prefer a region the registry characterises. A region absent from every split row ties the top score by having nothing recorded about it, and the Region column must not answer "conformant to what?" with a region the suite knows nothing about. Only the label is affected; a strictly higher score still wins whatever its source.
  • The seeded { NULL: false } split closed its own loop. eu-west-2 and eu-central-1 were the last regions accepting it; by the 2026-07-17 sweep both reject it again, so every region agrees, the split is retired, and its test asserts the shared rejection. The detect, admit and reconcile path the pre-release introduced, exercised end to end within days.
  • A tooling test now asserts every tracked file is text, after a raw NUL byte used as a delimiter in one source file made grep classify the file as binary and silently skip it.
 		76.2% +0.4pp rose 0.4 percentage points
75.8% 0.0pp unchanged
75.8% 0.0pp unchanged
75.8% +0.6pp rose 0.6 percentage points

Suite on

The scores barely move in this release, but what they mean has changed.

Until now the suite pinned one region, eu-west-2, as ground truth. That was quietly unfair: real DynamoDB disagrees with itself in a handful of places, and a one-region baseline takes a side without saying so. The clearest case is the { NULL: false } attribute value - accepted and normalised to { NULL: true } in eu-west-2 and eu-central-1, rejected with a ValidationException in us-east-1 and ap-southeast-2. An engine matching us-east-1 on that behaviour was marked non-conformant for doing exactly what real DynamoDB does in Virginia. From 2.0.0, ground truth is per region.

What changed:

  • A weekly sweep runs the full suite against real DynamoDB in every commercial region and publishes per-region ground truth (.github/workflows/sweep.yml, ground-truth/). It gates nothing: PR scoring stays offline and deterministic.
  • Confirmed regional splits live in a checked-in registry (registry/splits.json), each row carrying what every named region actually returned, when, and who admitted it. Detection is automatic; admission is not. The sweep files an issue with the evidence, and only a human commits a row. The registry ships seeded with the { NULL: false } split.
  • Each target is scored against every observed region's expectations, and its published number is its best-matching region - named in the results table's new Region column, with the full per-region view in results/summary.json (a versioned, additive artefact; the per-target results/<slug>.json files are unchanged in shape).
  • A third result state, indeterminate, for a failed observation: a timeout, an exhausted throttle, a transport fault. It is excluded from both sides of the score and cannot become a split, a registry row, or a fail. An absent answer is not a different answer.
  • Region health is tracked in registry/regions.json. A region that cannot complete a sweep is published as unresolved rather than silently omitted; two consecutive misses drop it from the scored set and page a maintainer in the same act.

One deliberate departure from the RFC that proposed this (#75): the RFC suggested a behaviour conforms if it matches any real region. 2.0.0 scores each target against one region at a time and headlines the best match, so a target only passes a behaviour when at least one real region does what it did, and its headline reflects one coherent region rather than a mix. Match-any scoring would have accepted an engine that combines eu-west-2's answer on one behaviour with us-east-1's on another - a deployment that exists nowhere. That is stricter than the RFC asked for, and it is deliberate.

No score moves at release: the one admitted split pins eu-west-2, which is the only region in the health record until the first sweep runs. Per-target deltas will be published once the sweep admits more regions; the expected movement is roughly a tenth of a percent for the six engines that match us-east-1 on the { NULL: false } split.

The suite also grew to 954 tests, up 81, all characterised against real DynamoDB - the control-plane pins in eu-west-2, everything else across four regions (eu-west-2, eu-central-1, us-east-1, ap-southeast-2):

  • UpdateTable AttributeDefinitions reconciliation (#77, #78, #79): delta-fed GSI adds merge into the stored union rather than replacing it, a conflicting redeclaration of an existing key keeps the stored type, and deleting a GSI prunes only its orphaned key attributes. An unused definition on an add is silently dropped where CreateTable rejects the same shape.
  • ProjectionExpression validation (#81): duplicate paths, alias collisions and parent/child overlaps rejected identically on GetItem, Query, Scan and BatchGetItem, pinned with exact messages; legal shared-prefix projections guarded as accepted; GetItem's misfiled validation tests rehomed into Tier 3.
  • Expression-size limit (#80): every expression parameter caps at 4096 bytes, measured on the raw string before ExpressionAttributeNames substitution - 4096 accepted, 4097 rejected, on all five expression surfaces. No tracked target enforces this limit today, so the Tier 3 movement it causes is new coverage, not a regression.
  • Empty set members (#82): empty strings in an SS and zero-length members in a BS are accepted and round-trip intact through every write path, and contains() can find them; an empty NS member and duplicate empty members are rejected, with messages pinned. One existing assertion got stricter: the empty-binary round-trip now asserts byte length zero.
 		75.2% 0.0pp unchanged
75.2% 0.0pp unchanged
75.2% 0.0pp unchanged

Suite on

Grew to 873 tests, up 49, all characterised against real DynamoDB in eu-west-2. New coverage in three areas:

  • ConsumedCapacity: the transactional read/write split on a same-token replay and on ExecuteTransaction, and a correction that single-item operations report the aggregate CapacityUnits and omit the split, which is transactional-only.
  • Empty-binary key values: rejected as a top-level ValidationException on every path, including secondary-index keys and inside transactions, mirroring the empty-string rejection.
  • Expression, limit and response-shape parity: KeyConditionExpression operand and nested-path rules, ExpressionAttributeNames/Values hygiene, projection validation and list-index fidelity, reversed-bounds BETWEEN, read-path key-length and segment caps, whitespace numbers, batch unprocessed fields and cross-table projection mixing, the bare no-op upsert, multi-subpath UPDATED_NEW, filter operand ordering, hash-only-GSI pagination, and CreateTable spec validation.
 		75.2% -0.5pp fell 0.5 percentage points

Suite on

Grew to 824 tests, up 7, in two parts: two sibling-parity gaps where one half of a rule was pinned and the other was not, and the capacity accounting of conditional and idempotent transactional writes. All characterised against real DynamoDB in eu-west-2.

The first covers the LSI side of the INCLUDE-projection-without-NonKeyAttributes rejection; the GSI side already had it. An LSI declared with ProjectionType INCLUDE and no NonKeyAttributes is rejected as a ValidationException in tier1 and pinned to the exact message in tier3, the same wording the GSI case returns.

The second pins the Query message for Select SPECIFIC_ATTRIBUTES with no ProjectionExpression, which Scan already had. Query and Scan enforce the same rule but word it differently: Query wraps the phrase in the "1 validation error detected:" envelope, Scan returns it bare.

The transaction cases settle what a conditional TransactWriteItems actually costs. A passing condition adds no read capacity: a conditional write bills the same 2 WCU per sub-1KB item as an unconditional one, and a standalone ConditionCheck costs 2 WCU, billed as write not read. Idempotent replay splits the accounting - the first call reports 2 write capacity units, a same-token replay within the window reports 2 read capacity units for re-reading the stored result. A failing condition cancels the transaction, and the response carries no ConsumedCapacity at all. Answers #27.

 		75.7% -0.7pp fell 0.7 percentage points
76.4% 0.0pp unchanged
76.4% -0.6pp fell 0.6 percentage points

Suite on

Grew to 762 tests, up 18, covering a malformed value in the lookup Key of a TransactWriteItems Update, Delete, or ConditionCheck - the path a Put item key does not take. Captured across four regions (eu-west-2, us-east-1, ap-southeast-2, eu-central-1), where every string was identical, so they pin exactly. An empty-string Key surfaces as a top-level ValidationException with the same message a Put item key gives; a wrong-typed or non-scalar Key cancels with a ValidationError reason carrying "The provided key element does not match the schema" - the key-only form, not the "Type mismatch for key" message the item-key path returns. The same run confirmed the BatchWriteItem table-key schema-mismatch message is region-invariant.

 		77.0% -2.0pp fell 2.0 percentage points

Suite on

Grew to 744 tests, up 8, pinning the Select / ProjectionExpression rules on Query and Scan. A ProjectionExpression is only valid with Select SPECIFIC_ATTRIBUTES, and ALL_PROJECTED_ATTRIBUTES is only valid with an IndexName; real DynamoDB rejects both with a ValidationException before reading anything. The cases span ALL_ATTRIBUTES, COUNT, and ALL_PROJECTED_ATTRIBUTES, including the request that breaks both rules at once, where AWS reports the ProjectionExpression one. They assert the contractual phrase, so they hold whether or not the engine carries the wrapper AWS adds on Query but not Scan.

 		79.0% -0.9pp fell 0.9 percentage points

Suite on

Grew to 736 tests, up 30, covering what TransactWriteItems and BatchWriteItem do with an item whose key value is malformed - the wrong type, non-scalar, or an empty string - across both table and index keys. PutItem already covered this; the transactional and batch paths covered none of it.

Characterising it against real AWS turned up a split worth pinning. An empty-string key value is rejected by up-front input validation, so even inside a transaction it surfaces as a top-level ValidationException. A wrong-typed or non-scalar key value is caught while the transaction runs, so it comes back as a TransactionCanceledException carrying a ValidationError reason rather than a top-level error. BatchWriteItem has no cancellation path, so every variant there is a plain ValidationException. The tests pin both halves, which catches two opposite mistakes: an engine that wraps the empty-string case as a cancellation, and one that surfaces the type-mismatch case as a top-level error.

 		80.0% -2.2pp fell 2.2 percentage points

Suite on

Grew to 706 tests, up 7, tightening secondary-index behaviour in Tier 1. Query and Scan on a GSI or LSI now assert sparse membership: an item that omits the index key stays off the index but remains on the base table. And PutItem now rejects an item whose GSI or LSI key value is the wrong type, non-scalar, or an empty string while the base-table keys are valid, holding an index key to its declared scalar type the same way a table key is held.

 		82.2% 0.0pp unchanged
82.1% 0.0pp unchanged
82.1% 0.0pp unchanged
82.1% 0.0pp unchanged

Suite on

Real DynamoDB in eu-west-2 reworded a chunk of its validation errors, and the Tier 3 error-message tests moved with it. They now assert the contract the error carries - its type, the field it objects to, and the constraint - rather than the exact prose, because AWS varies the wrapper, the echoed input value and the field casing from one region to the next. A four-region capture in June found eu-west-2 and eu-central-1 on the new wording and us-east-1 and ap-southeast-2 still on the old, so the line between contract and cosmetic is drawn from what is invariant across all four.

This moves some Tier 3 numbers. Targets that were only ever marked down for wording DynamoDB itself renders inconsistently now pass those checks, so their Tier 3 scores rise: the suite has stopped counting a cosmetic difference as a behavioural one. Genuine behavioural divergences are still pinned exactly, for example PutItem with a { NULL: false } attribute, which DynamoDB now accepts in eu-west-2 and normalises to { NULL: true }.

 		82.1% -0.9pp fell 0.9 percentage points
83.1% +0.2pp rose 0.2 percentage points

Suite on

Grew to 684 tests with a control-plane and table-configuration sweep: the CreateTable/UpdateTable config parameters and the secondary control-plane operations (limits, backups and PITR, exports and imports, Kinesis, resource policies, contributor insights), each characterised against real AWS and probe-skipped where a target doesn't implement it.

The published percentage changed with it. It now measures correctness over implemented operations, Pass / (Pass + Fail), so skips no longer count against the score. A skip is honest scope; a fail is a bug.

 		82.9% -1.2pp fell 1.2 percentage points
84.1% -0.6pp fell 0.6 percentage points

Suite on

Grew to 625 tests, up 24 on the previous run: eleven more in Tier 1 and thirteen more in Tier 3, tightening coverage of core operations and the strict edge cases.

 		84.7% +0.7pp rose 0.7 percentage points

Showing the 24 most recent runs. The chart above covers the full history, and every run is browsable from Runs.