Run
9 emulators 1056 tests
Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so an upward arrow can be the new tests biting rather than a target getting worse.
Suite grew from 1054 to 1056 tests this run.
That's 2 new tests measured against every target. Movement below compares to the previous run, so a rise here is as likely to be the stricter suite as a real regression.
What changed in the suite this run
Suite on
AWS corrected the vector index readiness documentation, prompted by a write-up of the earlier guidance that drew on the suite's measurements. The ACTIVE-plus-backfilling state the old advice was built around, and which no index ever occupies, is gone from the three pages that described it. The wait now reads "Backfilling is not true" rather than "is false", so a check written literally from it fires on both creation paths instead of neither. The tutorial no longer says a search during backfill can return incomplete results. Two things the suite had measured but nobody had written down are documented as well: that DescribeTable reporting ACTIVE leads the dedicated search endpoint, and that the readiness check depending on neither status field is a real search in a retry loop.
That contract is now pinned rather than described. The UpdateTable walk asserts that Backfilling true is only ever reported alongside CREATING, that an ACTIVE index reports no Backfilling field at all, and that the base table goes ACTIVE while the index is still building, which is what makes a table waiter the wrong gate for a search. The first search that succeeds has to carry every seeded item, since the backfill window answers with an error rather than a partial view. On the CreateTable path a new test runs the documented check the way an application would, and every rejection before the first served response has to be the retryable ValidationException rather than a not-found.
The suite's own search wait now absorbs those two rejections and rethrows every other answer, so a fixture waiting on an index that is ACTIVE but not yet served no longer fails on the lag it was waiting out. Two files asserting exact rejection messages wait for a served search rather than for ACTIVE: "does not have the specified index" is also what a freshly ACTIVE index says, and it would otherwise stand in for whichever message the case asked for.
The tutorial's other new claim, that a table cannot be deleted while a vector index is being created, has a test of its own. Only the UpdateTable path can ask it. Across three runs an index created with its table reached ACTIVE in the same 250ms poll as the table, so the table is never ACTIVE with the index still building there, and a DeleteTable during creation is refused for the table's own status rather than with the documented index wording. Adding an index to a live table opens that window about thirty seconds in. The test cancels the index afterwards instead of waiting out the backfill, which a still-creating vector index turns out to accept the way a backfilling GSI does, so it costs a minute rather than seventeen.
A release now dispatches its measurement as the results bot rather than with
the workflow's own token. GitHub raises no workflow_run event when a run
started by GITHUB_TOKEN finishes, so 3.2.0 measured green for three hours and
its board only landed once the results table was dispatched by hand.
-
Grade baselineground truth
no divergence in all 33 regions · covers 100.0% of the suite
Tier breakdown
Tier 1 · Core 0.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges100.0% coveredTier 3 · Strict 0.0% diverges100.0% covered -
Runs via: npx, Docker, Homebrew, binary, npm, cargo, embedded, GitHub Action, source
Disclosure: maintained by this board's author
Grade Aunchangedlow divergence (0.9%) in 2 regions · up to 1.3% in the other 31 · covers 94.5% of the suite
Tier breakdown
Tier 1 · Core 2.0% diverges100.0% coveredTier 2 · Complete 0.0% diverges81.5% covered 42 unsupportedTier 3 · Strict 0.0% diverges95.3% covered 16 unsupportedAlso built for WebAssembly / OPFS
-
Grade B
low divergence (0.9%) in 2 regions · up to 1.3% in the other 31 · covers 83.2% · coverage lowers this row to B
-
-
Grade Bdiverged 0.2 percentage points less
low divergence (1.7%) in 21 regions · up to 1.8% in the other 12 · covers 87.6% of the suite · coverage lowers this row to B
Tier breakdown
Tier 1 · Core 0.8% diverges100.0% coveredTier 2 · Complete 0.9% diverges51.1% covered 111 unsupportedTier 3 · Strict 3.5% diverges94.1% covered 20 unsupportedAlso built for SQLite
-
Grade B
low divergence (1.6%) in 21 regions · up to 1.7% in the other 12 · covers 87.6% · coverage lowers this row to B
-
-
Grade Bunchanged
moderate divergence (11.8%) in 21 regions · up to 11.9% in the other 12 · covers 95.8% of the suite
Tier breakdown
Tier 1 · Core 5.7% diverges100.0% coveredTier 2 · Complete 15.0% diverges87.7% covered 28 unsupportedTier 3 · Strict 18.5% diverges95.3% covered 16 unsupported -
Grade Cunchanged
moderate divergence (12.8%) in 25 regions · up to 13.0% in the other 8 · covers 76.9% of the suite · coverage lowers this row to C
Tier breakdown
Tier 1 · Core 10.8% diverges100.0% coveredTier 2 · Complete 12.3% diverges17.6% covered 187 unsupportedTier 3 · Strict 15.9% diverges83.2% covered 57 unsupported -
Grade Cunchanged
moderate divergence (14.8%) in 25 regions · up to 14.9% in the other 8 · covers 95.1% of the suite · coverage lowers this row to C
Tier breakdown
Tier 1 · Core 6.3% diverges100.0% coveredTier 2 · Complete 15.9% diverges84.1% covered 36 unsupportedTier 3 · Strict 26.2% diverges95.3% covered 16 unsupported -
Grade Cunchanged
high divergence (15.1%) in 25 regions · up to 15.2% in the other 8 · covers 93.8% of the suite
Tier breakdown
Tier 1 · Core 7.6% diverges100.0% coveredTier 2 · Complete 14.1% diverges78.4% covered 49 unsupportedTier 3 · Strict 26.5% diverges95.3% covered 16 unsupported -
bec9b9f74932 53 unsupported ; covers 95.0% of the suite
Runs via: Docker, Homebrew, install script, Scoop, binary, JAR
Grade Cunchangedhigh divergence (20.9%) in 21 regions · up to 21.0% in the other 12 · covers 95.0% of the suite
Tier breakdown
Tier 1 · Core 10.8% diverges100.0% coveredTier 2 · Complete 32.2% diverges83.7% covered 37 unsupportedTier 3 · Strict 27.9% diverges95.3% covered 16 unsupported