# Parity Suite

> How faithfully does each DynamoDB emulator match the real thing? An independent conformance suite, scored against live AWS DynamoDB and tracked run over run.

This file concatenates the About and Methodology pages, the agent guide, and the latest results as text, so the whole picture can be read in one fetch. It's regenerated at build time from the same results as the rest of the site, so it can't drift from the live figures. Data endpoints and the licence are listed at the end.

---

# Why measure conformance?

There's no official conformance suite for DynamoDB. AWS doesn't publish one, so every emulator author ends up guessing at how the real thing behaves and testing against their own assumptions. The closest the community had was Dynalite's test suite, and by the start of 2026 over half of its tests had drifted out of step with current DynamoDB. DynamoDB Local ships with no test suite at all.

That's the gap the [conformance suite](https://github.com/paritysuite/dynamodb-conformance) fills. It runs every test against real DynamoDB on AWS first, records what passes, and treats that as the baseline. An emulator only passes a test if it gives the same answer DynamoDB does. Real DynamoDB is the ground truth, which is why it sits at the top of every table diverging from itself nowhere - not because it scored well, but because it's the thing everything else is measured against. DynamoDB doesn't always behave the same way in every region, so the suite records the answer in each region and scores a target against all of them; the [methodology](https://paritysuite.org/methodology) has the detail.

## "Works with the SDK" isn't the same as "behaves like DynamoDB"

Plenty of emulators will happily take your requests and hand something back. The question that matters is whether they hand back the *same* something. A subtly different error, a validation check that fires in a different order, a filter that returns one extra item - none of it breaks the happy path, all of it bites you in production when your code was quietly relying on the real behaviour.

So the suite tests observable behaviour and nothing else. It drives the standard AWS SDK against an HTTP endpoint and asserts on what comes back: the response shape, the error it returns (its type, the field it objects to, and the constraint it failed), the order things are validated in. No internal hooks, no privileged access. If your application talks to the target through the SDK, that's precisely what gets checked.

## Three tiers, because one number hides too much


A target that gets 8% of the suite wrong tells you almost nothing on its own. Eight percent of *what*? Get 8% of the core operations wrong and you'll feel it constantly; get 8% of the strictest edge cases wrong and it's likely still fine for local development. Those are very different situations behind the same number, so the suite splits its tests into three tiers and reports divergence within each.

**Tier 1 - Core.** The operations roughly 90% of DynamoDB users rely on: CRUD, queries, scans, batch operations, GSIs, UpdateTable. The more an emulator diverges here, the more often everyday code will hit a difference.

**Tier 2 - Complete.** Documented but less common features: transactions, PartiQL, LSIs, TTL, streams, tags, vector search. A gap here only matters if you actually use that feature.

**Tier 3 - Strict.** Validation ordering, error behaviour at a range of strictness - exact where DynamoDB's wording is stable, structural (the type, field and constraint) where its rendering is non-deterministic - limits, and legacy API shapes. Missing some of this is usually fine when you're developing locally, where the exact error string rarely matters. It matters far more in CI: if your own test suite runs against an emulator and asserts on error messages or validation behaviour, a Tier 3 gap is exactly the kind of thing that lets a bug through a green build and only shows up against real DynamoDB in production.


So a target diverging 8% over the whole suite might be right about every core operation and wrong about a fifth of Tier 3, or the other way round. The tier columns say which, and only one of those two is a problem for most people.

## Skips are scope, fails are bugs

Skips here are deliberate. Each test file probes for feature support up front and skips itself if the target doesn't implement that operation at all. A skip says "I don't do this"; a fail says "I do this, but I get it wrong". Those are different problems for whoever is relying on the thing: an operation a target declines is something you find in minutes and plan around, and one it quietly gets wrong is something you find in production.

So the board reports them apart. **Divergence** counts the fails against the whole suite. **Coverage** says how much of the suite the target implements. A target with a narrow surface that gets it right shows a low divergence and a low coverage, and both are true at once. Because the denominator is the whole suite either way, a fail that turns into a skip leaves both figures together: divergence and coverage fall by exactly the same amount, so a target that stops attempting something it got wrong cannot improve one without paying for it in the other. The letter grade on each row is a reading of the pair: divergence sets the letter and coverage can only lower it, never raise it, under [criteria the methodology publishes in full](https://paritysuite.org/methodology#grading). The letter is weaker than the figures under a withdrawal, and the methodology says by how much. The [methodology](https://paritysuite.org/methodology) works the rest through.

## Why a whole site for it

The suite already publishes its latest numbers as a table in its README. That's fine for a single snapshot, but a markdown table can't show whether an emulator is improving or regressing, it can't carry a per-tier breakdown without overflowing the page, and it throws away every run before the current one.

The data to tell the fuller story was there all along. Every run stamps its results into the repo, and the git history holds every past snapshot. So this site reads that history and rebuilds the whole timeline: the [latest results](https://paritysuite.org/) with run-over-run movement, a page for [each target](https://paritysuite.org/targets), and [every recorded run](https://paritysuite.org/runs) you can step back through.

Every figure here is derived from the suite's own results at build time. None of it is typed in by hand, which is the point - the moment you copy a number into a second place, the two start drifting apart, and conformance figures that drift are worse than none at all.

## On independence

The suite and this site are built and maintained by [Martin Hicks](https://martinhicks.dev), who also maintains Dynoxide - one of the targets scored here. That's why the no-figure-by-hand rule matters for more than drift: it's what keeps the scoring honest. Every number is derived from the suite's own results at build time, and the [scoring logic is shared with the suite](https://paritysuite.org/methodology) rather than restated here. A target's score can't be tuned without changing the suite's published results first, in the open, and the code that turns those results into the numbers on this page is in the same public repository as the tests that produced them. Real DynamoDB is the baseline, every figure carries the region and date it was measured, and [anyone can suggest a target](https://github.com/paritysuite/dynamodb-conformance/issues). There's more for programmatic consumers, and the raw data, on the [agent guide](https://paritysuite.org/for-agents).

What is reproducible is the scoring, not the site. Clone the repository and you can rerun the scorer over the committed results and get the figures published here, test by test. A local build of the site itself renders a smaller thing: the pages assemble a timeline by fetching the history of `results/` from the API, and without a token that fetch falls back to a committed snapshot. The deploy sets `FAIL_ON_FALLBACK` precisely so a scheduled build refuses the snapshot rather than quietly publishing a thinner board.

The letter grades need a second answer, because that argument only covers the figures. Where the bands sit was a choice, and moving one regrades targets whose results never changed - no run required, nothing to notice. So the [criteria are versioned and dated](https://paritysuite.org/methodology#grading): they are grading criteria version 1, and any change to a band, the coverage weight or the A+ gate bumps the version and is dated on that page. Both figures print beside the letter on every surface the board controls, so you can recompute a grade yourself and check it, and a retune leaves a record rather than a quietly different board.

The bands are not the only hand-picked input, and the other one deserves naming. The [split registry](https://paritysuite.org/ground-truth) is written by hand by design: it records the behaviours where real DynamoDB's own regions disagree, with the evidence each region returned, and a target matching any recorded answer is scored as conformant rather than wrong. Admitting a row there can turn a fail into a pass with no re-run and no results file changing, which is the one thing "a score can't be tuned without changing the published results first" does not cover. So the registry is public, every row carries the captured evidence and the date it was refreshed, and a behaviour enters only once confirmed across regions. It currently holds 20 rows, and they are the entire set of behaviours on which matching one region rather than another can turn a fail into a pass.

---

# How the numbers work

The [About page](https://paritysuite.org/about) covers why this exists. This one is the how: where each figure comes from, how the history is rebuilt, and - just as important - what the suite doesn't tell you.

## How a score is worked out

Every test runs against live AWS DynamoDB first. Whatever real DynamoDB does is recorded as the expected answer, and an emulator passes a test only if it gives that same answer. Real DynamoDB doesn't behave identically in every region, though, so the suite records the answer in every region it can reach and scores each target against all of them, taking its best-matching region as the headline. That's why DynamoDB sits at the top of every table diverging nowhere: each region agrees with itself, so measured against its own answers it is right everywhere. The [regional ground truth](https://paritysuite.org/ground-truth) page has the why and the evidence.

The tests only ever look at observable behaviour. They drive the standard AWS SDK against the target's HTTP endpoint and assert on the response: its shape, the error returned (its type, the field and constraint it objects to, matched exactly where the wording is stable and structurally where AWS varies it), the order validation fires in. Nothing reaches inside the implementation. If your application would see it through the SDK, the suite checks it; if it wouldn't, the suite doesn't care about it.


Results are split into three tiers - Core, Complete, and Strict - so a single percentage can't hide a fatal gap behind a pile of passing edge cases. Each tier carries the same two figures as the headline, over that tier alone.

Every target carries two figures, and they are never added together. **Divergence** is the share of the whole suite a target answers differently from real DynamoDB. **Coverage** is the share of the suite's tests it implements at all. It is weighted by test count rather than by a count of features, so a target that declines one heavily-tested operation loses more coverage than one declining several lightly-tested ones - `putItem` alone carries 129 of the suite's tests, while `account`, `resourcePolicy` and `contributorInsights` carry two each. Read as a pair the two figures say something a single number can't: a target that diverges nowhere conforms on everything it implements, and its coverage figure is what says how much that is.

They stay apart because a skip and a fail are not the same kind of problem. A [skipped test](https://paritysuite.org/about) is the target's feature-probe declining to run, because the operation isn't implemented; you find that out in minutes and plan around it. A fail means the operation is there and behaves differently from DynamoDB, which you tend to find out in production. Adding the two would price them the same.

Divergence is measured over the whole suite rather than over what a target attempts, so the denominator never moves. That gives the two figures a fixed relationship. When a target stops attempting an operation it used to get wrong, that test leaves the divergence numerator and the coverage numerator together, over the same unchanged denominator, so **both figures fall by exactly the same amount**. Withdrawal doesn't go unrecorded: it costs precisely as much coverage as it gains divergence.


Dynalite is the live case. On 24 July 2026, 88 of its failing tests became skips. Nothing was fixed, nothing else changed, and the suite stayed at 998 tests. Its divergence fell 8.8 points and its coverage fell 8.8 points, both of them exactly 88/998. Under the correctness figure this replaced, the same act *raised* the score 8.4 points, because those 88 tests left the denominator as well as the numerator, and there was no second figure left to record what had happened.

Implementing a sliver and getting it right still reads as 0.0%, because nothing was got wrong. The coverage figure beside it is what says how thin that surface is.

The standings are ordered by the grade below, so what a target declines counts against its place as well as its letter. That isn't a verdict on which emulator you should pick, because that depends on which operations you need.

<a id="grading"></a>

## How the grade is read

The letter is a reading of the two figures, and both stay printed beside it. Coverage can only lower a letter, never raise it: a third of whatever a target leaves unimplemented is added to its divergence before the bands are read. A target implementing the whole suite is graded on divergence alone.


Divergence sets the letter: **A** under 5%, **B** under 15%, **C** under 25%, **D** under 35%, **F** beyond that. **A+** is exactly zero divergence at full coverage: zero failing tests against the target's best-matching region and nothing declined, both read as counts rather than as the published percentages - one fail in a large enough suite displays as 0.0% without being zero, and a target one test short of the suite does not print 100.0% coverage. Nothing on the board holds A+ today.

Both figures print beside the letter on the standings, each target's page, the results table and the data endpoints, so you can always recompute it from the criteria below. The [badge](https://paritysuite.org/for-agents) is the exception: a shields endpoint has room for a label and a message and nothing else, so it carries the letter alone.

A row says where coverage is holding its letter down, so a capped row is not mistaken for one with room above it. Picked by rule rather than by hand, the two capped rows with the lowest coverage: Dynalite diverges 10.7%, the B band on its own, but implements 68.6%, which reads it up to an effective 21.2 and grades it **C**. ExtendDB diverges 1.8%, the A band on its own, but implements 78.9%, which reads it up to an effective 8.8 and grades it **B**.

Real DynamoDB carries no letter at all. A grade reads how far a target sits from real DynamoDB, so grading the yardstick against itself seats it in a band an engine had to earn its way into. It reads `baseline` on the results table, on its badge and in the data endpoints, and it sits above the board rather than in it. Its two figures still publish, because they are the definition every other row is read against.

### The letter is weaker than the figures

Withdrawing a failing test lowers divergence and coverage by the same amount, which is what makes the pair hard to game. The letter inherits only two thirds of that, so a target that declines what it fails still gains a little. Closing the gap entirely would mean counting a declined test as heavily as a failed one, and the difference between those two is what the board is for.

**Rank on divergence and coverage, not the letter.** Whichever row currently sits closest to a band it has not earned is the one this costs least: on the current board that is DynamoDB Local, at 31 withdrawn tests.


The divergence boundaries at 5% and 25% are the numbers this board has published as its colour bands since it began. The numbers carry over; the denominator does not. Those bands sat on correctness - 95% and 75%, over the operations a target implements - and these sit on divergence, over the whole suite. The same digits therefore cut a different line everywhere but full coverage, and the coverage weight is what answers it: a target diverging 4.9% over 92% coverage is 94.7% correct, which the old sub-95% amber band would have caught, and reads an effective 7.6 here, which grades **B**. The splits at 15% and 35% are new lines, and so is the weight.


The grade reads the same figures as the row it sits on: the target's headline region, its best-matching one. Every target is measured against every observed region, and regions play no part in the letter. Regional disagreement in real DynamoDB currently moves a figure by at most 1.6 points, across a suite of 1,266 tests, so there is no region-based cap. Each row states its regional distribution with the figures attached - "no divergence in 6 regions · up to 0.3% in the other 26", or "low divergence (2.0%) in all 32 regions" - never as a bare count. A bare count read inverted: a target failing identically everywhere showed "32 of 32" while one perfect in six regions - and differing elsewhere only where DynamoDB differs from itself - showed "6 of 32", so the bigger number read as the better target. Paired with its figure, the count says what it means, and a zero can never read as more than the measurement: "no divergence in 6 regions" claims exactly what was observed. It is "up to" in the remainder because the remainder need not be uniform. Each target's page has the full per-region drilldown.

Where the letter would not hold in every region, the row says so. Scored against a region that answers some tests differently, a target can earn a lower letter than the one its row publishes, and near a band boundary a handful of split tests is enough. The row then names those regions and the letter they would give, for example "B against eu-south-2 and me-central-1, where real DynamoDB answers some tests differently", on the board, the target's page and the results table. The letter itself stays put. Grading from the worst region would make letters follow AWS's rollout schedule: a target that changed nothing would drop a band when one region diverged and go back up when it converged. So the note moves instead, and it can appear or disappear between runs with no change in the target at all. A note disappearing means a region changed its answers, not that the target improved. The badge carries the letter alone, so it carries no note either.

An A+ earned in a subset of regions cannot conceal ordinary bugs behind a friendly cohort, and that is a property of the scoring model rather than a promise. Per-region scoring only changes a verdict on a confirmed split - a behaviour where real DynamoDB's regions disagree with each other, held in the [registry](https://paritysuite.org/ground-truth) with per-region evidence - so a test with no split is expected to behave identically everywhere, and a target failing it fails it in every region, the headline included. Zero divergence in the headline region therefore means the target's remaining regional differences are confined to behaviours where DynamoDB disagrees with itself. The scale is small enough to check by eye, on any row you like: no target's worst region fails more than 20 tests beyond its headline, against a registry of 20 confirmed splits. A matching cohort can't be manufactured by a bad sweep either: an unresolved region is flagged and carried rather than scored, and a split enters the registry only once confirmed. As a backstop, a build-time guard asserts it: every fail a zero-divergence target records in any region must be one of the registry's split tests - checked by test identity, not by count, so unrelated fails could not hide behind a registry that happens to be the same size. The check gates the deploy rather than running beside it, and the suite publishes the failing test identities into its results artefact so the check can reach that verdict from the registry itself rather than taking a count on trust. It runs against the committed results, not the copy a given build fetches, so it can block a publish but cannot catch a fetch that returns something the repository does not hold. If that ever breaks, the build fails and the criteria are revisited in the open. A second check holds every row, not only zero-divergence ones, to the regional note above, and reads it in the built pages rather than in the data behind them. It replaced a rule that the letter must survive the target's worst region; the first time that rule failed, the revisit ([#271](https://github.com/paritysuite/dynamodb-conformance/issues/271)) kept the criteria and added the note. The grading version marks a change to the criteria themselves, the bands and the coverage weight, so a revisit that leaves those alone leaves the version where it is.


Everything is rounded to the one decimal place the board publishes before the bands are read, so recomputing a grade from the figures on a row lands where the grader did. The A+ gate alone reads the raw values: a divergence that merely rounds to 0.0% earns an A, and a target one test short of the suite does not print 100.0% coverage.

Letters compress, so two things are worth knowing at the boundaries. A target sitting near a band edge can change letter on a movement of a fraction of a point; the percentage-point figures are the finer instrument, and when a letter does change between runs the row says so beside the movement (for example "C → B"). And the standings sort by grade, then within a grade by the effective figure the letter was read from, then by divergence, coverage and name for anything still tied, so the order is deterministic and two targets sharing a letter still sort by their figures. The "Biggest moves" chips list the three largest movements of at least 0.1 percentage points among re-tested targets.

Every letter on the board is produced by one shared function from the two published figures and nothing else - no target is graded by different rules, the [board author's own engines](https://paritysuite.org/targets/dynoxide) included - and the suite's tests assert exactly that: the published grade must equal the one recomputed from the row's own divergence and coverage. A row with no cap clause is a letter set by its divergence band alone.

These are **grading criteria version 1**, in effect from 12 August 2026. Retuned thresholds move published grades on targets that changed nothing - the documented failure mode of every graded system - so any change to the bands, the coverage weight or the A+ gate bumps the criteria version and is dated here. The criteria also travel machine-readably in every [data endpoint](https://paritysuite.org/for-agents)'s `metrics.grade`.

A grade is an observation, not a certificate. It grades observed behaviour against this suite's published tests on a named date, in the regions named beside it. It is not an audit of production readiness, not a compatibility certification, and not an endorsement of any target.

The two figures are not properties of the same thing. Coverage is a property of the target: which of the suite's tests it implements at all doesn't depend on which region it is compared against, because scoring a target against a region only ever turns one of its passes into a fail or back again - a skip stays a skip everywhere. Divergence is a property of the target *measured against a region*, so it is the one figure a target's headline region decides. Every figure on a row comes from that same region, and the row names it, so a target's fails divided by the suite size always reproduces the divergence printed beside them.


One consequence worth spelling out: [the suite grows](https://paritysuite.org/changelog). It had 526 tests in March 2026 and over 600 by May. Raw counts from different runs aren't comparable, so every chart and every movement figure on this site is a **percentage**, never a count.

## When a project has more than one build

Some projects ship the same engine in more than one shape: a storage backend swapped underneath it, or the query layer compiled for somewhere else to run. Those builds are nested under the project rather than seated beside it, because a reader chooses between projects first and the build follows from where their code runs. Seating them as rivals would let one engine take several places near the top.

Every build is measured in full, and every build has its own row, its own figures and its own page. The nesting decides only how much of that you see at once: a project's other builds sit behind a disclosure on its row, which starts closed only when every build under it reads the same grade, divergence and coverage as the row above. A build repeating all three is a click away rather than three more lines of the same numbers, and one that differs is already showing - along with its siblings, since the disclosure opens as a whole.

Three things have to hold for a build to start closed, not one. The figures have to match. Both builds have to have been measured in the run being shown, on either side: a row carried from an earlier run has its figures frozen at the run that measured it, so a carried build, or a carried project row above it, opens the disclosure whatever the percentages say. And neither can be a row the suite declined to score, because a run that recorded a failed observation publishes no figures at all and two blanks are not agreement.

The second clause has a visible consequence worth naming: on a run where a project itself was not re-tested, every one of its builds shows open, even builds whose figures are identical to the row above them. That reads like a fault and is the opposite - it is the board declining to say two things agree when it has not measured them together.

What is compared is what the row prints: the grade, the divergence and the coverage. Not the version, the region cohort or the tier breakdown. So a closed disclosure says those three read the same, which is what its summary says, and no more than that.

The comparison is made from each run rather than from a setting anyone maintains, so it goes both ways: a build that converges with the one above starts closed on the next run, and one that later diverges starts open again. It picks a starting state and nothing else, so no figure is withheld by it either way.

The README table in the suite repository has no disclosure to offer, so it lists every build outright.

## How runs and movement are reconstructed

The suite publishes each run's results as JSON in its repository, and it has done since the first run. That means the full history is sitting in the git log, and this site rebuilds the timeline from it: it reads every version of those result files, scores each one with the suite's own logic, and assembles the runs you browse here.

A "run" is defined by the timestamp stamped into each result file, grouped by date - not by commit. That distinction matters more than it sounds. A single commit often refreshes only some targets, and one commit can carry results that were actually produced in different runs, so grouping by commit would invent runs that never happened and stitch unrelated results together. Grouping by date is robust to both, even when one run's targets finish over an hour apart.

When a target isn't re-tested in a run, its last measured result is carried forward and labelled as such, rather than dropped or silently restated as fresh. **Movement** compares a target against the previous run it was actually tested in, so the arrow always means "since last measured", never "since some run where nothing changed".

This site and the suite are one repository, and the scoring is shared code rather than a copy of it. The target list, the display names, the project links and the pass-rate arithmetic are imported from the suite's own modules, so adding a target or correcting a name happens once and lands in both places. What the site still renders on its own - assembling a scored run into the rows you see - is held to the suite's published per-region summary by a test.

That arrangement exists because the rule behind the whole site is that a target's figures are derived, never typed. The moment the same number lives in two places it starts to drift, and a number that has quietly drifted is worse than no number at all. When the two lived in separate repositories a new target was added to the suite a day before the site learned its name, and for that day the comparison was wrong.

What is not derived is disclosed where it applies. The [grade bands and the split registry](https://paritysuite.org/about#on-independence) are hand-picked inputs and always will be. The baseline row is a conditional case rather than a standing exception: real AWS is measured in three passes, and while any of them is outstanding the row is pinned to its last clean measurement instead of being derived from the current one. The [ground truth](https://paritysuite.org/ground-truth) page names the passes that have reported and what they were measured on, so it says which of those the row is standing on today rather than leaving you to assume.

What you can reproduce is the scoring. Clone the repository, run the scorer over the committed results, and you get the figures published here, test by test. A local build of the site renders a smaller thing: it assembles the timeline by fetching the history of `results/` from the API, and without a token that fetch falls back to a committed snapshot. The deploy sets `FAIL_ON_FALLBACK` so a scheduled build refuses the snapshot rather than quietly publishing a thinner board.

## Limitations

A score here is a useful signal, not a certificate. Worth keeping in mind:

- **It only tests what it tests.** A behaviour with no test is a blind spot, not a pass. The suite is broad and growing, but "0.0% Tier 1" means "nothing wrong among the Tier 1 tests that exist", not "every Core behaviour DynamoDB has".
- **The headline is tier-level; the per-operation detail is a click away.** The top-line number rolls up to per-tier percentages, so at a glance you see a target is weak on Tier 2, not which operation. Each target page then breaks its score down by operation, on the same divergence and coverage axes, the [emulator directory](https://paritysuite.org/targets) lists feature and operation support for each project, and the failing tests are listed by name. The [suite's repo](https://github.com/paritysuite/dynamodb-conformance) has the raw per-test results behind all of it.
- **Every result is a point in time, and a place.** A score is tied to the version of the target tested on that date, against DynamoDB's behaviour on that date - and DynamoDB's behaviour is neither identical across regions nor fixed over time. In June 2026, for instance, a `PutItem` with a `{ NULL: false }` attribute was accepted in eu-west-2 and eu-central-1 but rejected in us-east-1 and others; by mid-July the regions had converged again. The suite scores against every region it can reach and headlines the best match, so a score means conformance to real DynamoDB as it behaved across the regions on a named date. Both sides move.
- **Behaviour only, nothing else.** The suite says nothing about performance, scalability, durability, cost, or operational fit. An emulator can match DynamoDB's behaviour perfectly and still be the wrong tool for your job, or the right one despite a worse figure here.
- **A skip is recognised from the target's own answer, not declared.** Each test file probes for support before running, and the probe reads the target's response: an `UnknownOperationException`, a message matching `unknown operation`, `not implemented`, `unsupported operation` or `is not supported`, or an HTTP 501. Anything else arrives as an ordinary error and the test is scored as a failure. That matters because the same classification decides both published figures and the order of the board: a target that declines an operation in wording the probe doesn't recognise has its divergence inflated and its coverage overstated, and one that returns a 501 for an operation it half-implements has both understated. If a target's figures look wrong to its maintainer, this is the first thing to check.
- **Configuration matters.** Targets are tested in a representative setup. A differently configured deployment may behave differently.

## Trademarks and attribution

Amazon DynamoDB, DynamoDB, and AWS are trademarks of Amazon.com, Inc. or its affiliates. This is an independent project and is not affiliated with, endorsed by, or sponsored by Amazon, and nothing here grants any right to use those names or marks. DynamoDB Local, Dynalite, LocalStack, Ministack, Floci, ExtendDB, Kumo, and every other target named on this site are the trademarks or property of their respective owners.

The conformance suite is the work of [Martin Hicks](https://martinhicks.dev) and its contributors, released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0); see the [NOTICE](https://github.com/paritysuite/dynamodb-conformance/blob/main/NOTICE) for the full attribution. This site is built from the same repository, under the same licence, and is maintained by [Martin Hicks](https://martinhicks.dev). The fonts it uses, Inter and JetBrains Mono, are licensed separately under the SIL Open Font License 1.1.

---

# Reading these scores

This page is for anyone consuming the suite programmatically - an agent, a dashboard, a script - and for anyone who wants to read a number here and know exactly what it means. A single percentage is easy to misread as a verdict, so here's how the figures are built and where to get them as data.

## Get the data, don't scrape the page

Every figure on the site is published as JSON, regenerated at build time from the same results the pages render from. Read that instead of parsing HTML:

- [/data/latest.json](https://paritysuite.org/data/latest.json) - the latest run in full: every target's divergence and coverage, overall and per tier, its per-capability and per-operation-area state, and the full per-region breakdown, alongside the run's region health.
- [/data/runs.json](https://paritysuite.org/data/runs.json) - the whole history, newest first: per-target divergence and coverage, overall and per tier, plus run-over-run movement and headline region for every recorded run. Each run carries `gradedUnderCriteria` and `suite`, the measurement identity for that run - so a denominator that moved between two runs can be attributed to the release that moved it, not just observed. `suite` is null for runs measured before the identity was published. Every target in it carries a letter regardless, computed under the criteria in force now; the feed instead withholds the letter on those runs, because a feed entry is archived by other people and keeps its original timestamp. The flag says which you are reading, and `metrics.grade.effective` is the date it turns on.
- [/data/index.json](https://paritysuite.org/data/index.json) - a discovery manifest: the tier, capability and region vocabularies, where each endpoint lives, and the licence.
- [/feed.xml](https://paritysuite.org/feed.xml) - an Atom feed, one entry per run. Entries for runs measured before criteria version 1 took effect carry no letter, because none was published at the time; they say so in the summary and carry `<category term="ungraded"/>`, so a single entry read on its own is not ambiguous. Their `<updated>` is unchanged, so if you archived those entries when they were first published, the letters you may have seen in an earlier copy of this feed were applied retroactively and have been withdrawn.
- [registry/splits.json](https://raw.githubusercontent.com/paritysuite/dynamodb-conformance/main/registry/splits.json) - the behaviours where real AWS regions genuinely disagree, each with every region's recorded answer and the pinned one. Served from the suite repo rather than this site, because it is the suite's artefact. You need it to check the A+ premise for yourself: a zero-divergence target may fail a test in a non-headline region only where that test is recorded here.


The baseline's `observation` block, in every endpoint's envelope, says how much of the suite that row is currently standing on. Real AWS is measured in three passes for runtime reasons, and until all three have reported the row is pinned to its last clean measurement rather than derived: `lanes` is what has reported and when, `missingLanes` names what has not, and `unobserved` counts the tests carried rather than re-observed.

Every target carries the identical schema, live AWS DynamoDB included. The data is published under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/): use it freely, just credit paritysuite.org. The schema is versioned with a `schemaVersion` field, and what it entitles you to is this: a field you already read will not change type or meaning while the number stays put. New fields can appear at any version, so treat one you do not recognise as new rather than as an error - removals, renames and changes of meaning are what bump it. The grading criteria are a separate axis: they carry their own version, and a change to the bands changes what a letter means without `schemaVersion` moving, so anything storing letters over time should watch both. The `schema` block in [/data/index.json](https://paritysuite.org/data/index.json) says the same thing where a machine reader will find it.

Every board says what produced it. The `suite` block, on the board and in [/data/index.json](https://paritysuite.org/data/index.json), names the ref measured, its commit, the suite version at that ref, the region measured against, and when. Branch on `kind`: `tag` means a released suite with a dated changelog entry behind it, and anything else means the board was measured from an unreleased commit - treat its figures as provisional and do not compare them against a released board's. A board written before this field existed carries no `suite` block at all, which means not stated rather than not released. A board is graded against the suite manifest and split registry as they stood at that ref, so the denominator moves when a release moves it and not when `main` does. Region health is the exception and is read live, so a board's cohorts can be recomputed after its measurement without the measurement changing - which is why the timestamp describes the measurement rather than the figures. An identical `suite.commit` is not a promise of identical figures: the oracle is live AWS rather than a fixture, and an old tag can be deliberately re-measured, so two boards naming the same commit are independent observations rather than reproductions of each other. `region` and `measuredAt` are what tell them apart.

A project can ship several builds of one engine, and each is its own target with its own figures: `project` groups them, `configuration` names what distinguishes each one, and `isVariant` says a target is a build rather than its project's reference build, and `standsForProject` says which row the board treats as the project's own. Prefer `standsForProject` when you want the project's headline: on a run where the reference build recorded nothing, a build is promoted to stand for the project and every row of it reads `isVariant: true`. Every build has its own row on the board too, but a project's other builds sit behind a disclosure there, and `collapsedIntoProject` is true when that disclosure starts closed. That takes three things: the build reads the same grade, divergence and coverage as the row `standsForProject` names, which is normally the reference build but is a promoted build on a run the reference build did not record; both were measured in that run, on either side, since a carried row's figures are frozen at the run that measured it; and neither is a row the suite declined to score. It is one answer for the project rather than per build, because the disclosure opens as a whole, so a project whose builds disagree reads false on all of them. Do not read it as a data-quality signal: it is re-derived every run and says only what the board did with the row.

Every endpoint also carries a `metrics` block naming each published figure, its formula and its `direction` (`lower_is_better` for divergence, `higher_is_better` for coverage and correctness). Read the direction from there rather than assuming it. Schema 3 reversed which way is good, and a consumer that re-derived its own ranking on the old assumption would have inverted with nothing in the shape of the data to catch it.

## Badges

Each target also has a shields.io endpoint badge, served from the suite repo so a target's own README can show its live grade without copying a figure that goes stale:

```
https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/paritysuite/dynamodb-conformance/main/results/<slug>.badge.json
```

The slug is the target's slug on this site (`dynalite`, `localstack`, `dynoxide`, and so on), and the baseline's badge reads `baseline` rather than a letter. The path is the contract: it stays put, and what changes is the `message` inside it. It carries the letter grade under a `parity` label, and until 12 August 2026 it carried the retired correctness percentage under a `conformance` label - so if your alt text still says "conformance", it is describing a badge that no longer exists. The `schemaVersion` in the file is shields.io's own and is always `1`. It says nothing about this project's schema, and a badge has no other version channel, which is why the path is the one that has to hold.

## What a score actually is

Every target carries two figures over the whole suite. **Divergence** is failed divided by total: how much of DynamoDB's behaviour the target answers differently, so lower is better. **Coverage** is implemented divided by total: how much of the suite's tests it implements at all, so higher is better. Read both. A target with a thin surface that gets it right shows a low divergence and a low coverage, and folding them into one number would lose that.

The denominator is the same for both and never moves, which fixes their relationship: a test that goes from failing to skipped leaves both numerators at once, so divergence and coverage fall by exactly the same amount. If you are tracking a target over time, a divergence fall matched by an equal coverage fall is a withdrawal, not a fix.

The JSON keeps the raw counts (`passed`, `failed`, `skipped`, `implemented`, `total`) alongside both, so you can derive whatever figure you need rather than depending on the one the board leads with. The pass rate over implemented operations is still published, as `correctness`, for consumers that already read it. It used to be `total`, which is also the name of the raw test count in `counts`, so the same word meant a count in one place and a percentage in another.


From schema 4 each target also carries a `grade`: a letter (`A+` to `F`), a plain-language `qualifier`, a `band` (the colour tier: `pass`, `partial`, `fail` or `none`), a `capped` flag and a `capAt` letter. Two rows carry `letter: null` and must be handled before you recompute anything, because a literal reimplementation of the bands mis-grades a null (`null < 5` is `true` in JavaScript): a target with nothing scored has `qualifier: "not scored"`, and the live-AWS baseline has `qualifier: "baseline"`. The baseline is not graded at all. A letter reads how far a target sits from real DynamoDB, so there is nothing for the yardstick to measure against itself; its two figures still publish, and they are the definition every other row is read against.


`capped` says whether coverage lowered the letter, and `capAt` is that lowered letter - so a target diverging 12.3% (the B band on divergence alone) over 80.0% coverage reads `capped: true, capAt: "C"`, and cannot read better than C until its coverage rises. The two carry the same fact: `capAt` is the letter whenever `capped` is true and null otherwise, so test `capped` and read `capAt`. Where coverage binds nothing they are `false` and `null`. Two rows sharing a letter are not comparable when one of them is at a ceiling.

One property the letter does not inherit from the figures. Divergence cannot be lowered by declining operations without coverage falling by exactly as much, which is what makes the pair hard to game. The letter is weaker: a third of whatever a target leaves unimplemented is added to its divergence before the bands are read, so withdrawing a failing test still moves the effective figure down by two thirds of what left. Withdrawal costs more than it used to and is not free of gain. **If you are ranking targets programmatically, rank on `divergence` and `coverage`.** The letter is for reading, and the two figures are what carry the guarantee.

A letter change between tested runs travels as `movement.gradeChange` (`{from, to, label}`), null when the letter held. Two more fields keep the data at parity with the cards: `runDate` is the run that actually measured the row (it lags the run's own date when a target was carried forward untested), and `region.worst` is the worst observed region's divergence - the figure behind a row's "up to X% in the other N" clause - null when the target has no regional spread.

The grade is a reading of the two figures, never a blend of them - divergence sets the letter, with A+ meaning exactly zero failing tests against the target's headline region and nothing declined, both read as counts rather than as the published percentages, and low coverage can only cap it - and the full criteria (bands, the A+ gate, the coverage divisor) travel in `metrics.grade`, versioned as `gradingVersion` separately from the schema, because a criteria change regrades targets whose figures didn't move. Recompute the letter from the two values if you need to check it; the [methodology](https://paritysuite.org/methodology#grading) has the same criteria in prose. A grade is an observation against this suite's tests on a named date, not a certification, and not an endorsement.

[Skips are scope, not failure.](https://paritysuite.org/about) A skipped test is the target's own feature-probe declining to run because it doesn't implement that operation at all. That's kept out of the score and reported separately. A fail means the operation is there and behaves differently from real DynamoDB, and that counts. They mean opposite things, so don't fold skips into a pass rate.


There are [three tiers](https://paritysuite.org/about) - Core, Complete and Strict - and one figure over the whole suite hides too much. A target diverging 8% overall might be right about every Core operation and wrong about a fifth of Strict, or the reverse, and those are different problems. Each tier carries divergence and coverage on the same terms as the headline: lower is better for every divergence figure, and higher is better for every coverage one. If a user only needs everyday CRUD, the Core figure is the one that matters; if they assert on error behaviour in CI, Strict is where a gap bites. Read the tier that maps to what they actually do.


DynamoDB sits at the top of every table at 0.0% divergence over full coverage, and wears no letter. That's the baseline, not a competitor that happened to win: it's the thing everything else is measured against, so it agrees with itself by definition, and grading it would seat the yardstick in a band an engine had to earn its way into.

## What the numbers don't tell you

A score is tied to a target version, tested on a date, against DynamoDB's behaviour on that date. DynamoDB is neither identical across regions nor fixed over time, so the suite scores each target against every region it can reach and headlines its best-matching one; a figure here means conformance to real DynamoDB as it behaved across the regions on a named date, nothing wider. Both sides move. The [regional ground truth](https://paritysuite.org/ground-truth) page has the detail.

And it's behaviour only. The suite says nothing about performance, scalability, durability, cost, or operational fit. A target can match DynamoDB's behaviour perfectly and still be the wrong tool for a job, or the right one despite a worse figure here. The [methodology](https://paritysuite.org/methodology) has the full limitations.

## Comparing on a capability

If a decision hangs on a specific feature - PartiQL, transactions, GSIs, LSIs, streams, TTL - don't read off the headline. The [emulator directory](https://paritysuite.org/targets#capabilities) carries each project's feature support beside what it takes to run, and the same data is in the `capabilities` array for each target in [/data/latest.json](https://paritysuite.org/data/latest.json). The directory reads a project at a time; to line every target up against one feature, take that key from each target's `capabilities` array in the endpoint. The suite scores each target against real DynamoDB, never against each other, so the comparison is like-for-like.

The site won't tell you which target to pick. It gives you the evidence per target, on equal terms.

## Who maintains this

The suite and this site are built and maintained by [Martin Hicks](https://martinhicks.dev), who also maintains Dynoxide, one of the targets scored here. That relationship is why nothing on the site is hand-authored: every figure is derived from the suite's own published results at build time, and the [scoring logic is shared with the suite](https://paritysuite.org/methodology) rather than restated here. A target's score can't be tuned without changing the suite's published results first, in the open, and the tests, the results and the code that scores them are all in [one public repository](https://github.com/paritysuite/dynamodb-conformance) you can clone and run. Real DynamoDB is the baseline, every figure carries the region and date it was measured, and [suggesting a target](https://github.com/paritysuite/dynamodb-conformance/issues) is an open GitHub issue away.

---

# Latest results

Run 2026-10-11 (2026-10-11), 1266 tests. Divergence is failed / total and coverage is implemented / total, over the whole suite and again within each tier; lower divergence is better and the two are never added together. The grade is a reading of the pair: divergence sets the letter and coverage can only lower it, never raise it, by adding a third of whatever is unimplemented to the divergence before the bands are read. Rank on the two figures rather than the letter: withdrawing a failing test still moves the effective figure down by two thirds of what left. DynamoDB is the baseline, diverging nowhere by definition.

- DynamoDB (baseline) - grade baseline; diverges 0.0% of the suite; covers 100.0%; diverges per tier Tier 1 0.0%, Tier 2 0.0%, Tier 3 0.0%; version live (AWS)
- Ministack - grade A; diverges 0.1% of the suite; covers 100.0%; diverges per tier Tier 1 0.0%, Tier 2 0.3%, Tier 3 0.0%; version f90981c85c9b
- Floci - grade A; diverges 0.1% of the suite; covers 99.4%; diverges per tier Tier 1 0.0%, Tier 2 0.0%, Tier 3 0.2%; version e97cd0c1dc2a
- Dynoxide - grade A (coverage lowers this row to A); diverges 0.0% of the suite; covers 98.9%; diverges per tier Tier 1 0.0%, Tier 2 0.0%, Tier 3 0.0%; version 1.3.0
- Dynoxide (wasm), a build of Dynoxide - grade A (coverage lowers this row to A) (B against eu-south-2 and me-central-1, where real DynamoDB answers some tests differently); diverges 0.0% of the suite; covers 88.0%; diverges per tier Tier 1 0.0%, Tier 2 0.0%, Tier 3 0.0%; version 1.3.0
- LocalStack - grade A (B against eu-south-2 and me-central-1, where real DynamoDB answers some tests differently); diverges 2.4% of the suite; covers 95.7%; diverges per tier Tier 1 0.0%, Tier 2 6.1%, Tier 3 2.4%; version 2026.9.1
- ExtendDB - grade B (coverage lowers this row to B); diverges 1.8% of the suite; covers 78.9%; diverges per tier Tier 1 0.0%, Tier 2 0.0%, Tier 3 5.4%; version v0.1.13
- ExtendDB (SQLite), a build of ExtendDB - grade B (coverage lowers this row to B); diverges 2.1% of the suite; covers 83.0%; diverges per tier Tier 1 0.0%, Tier 2 1.2%, Tier 3 5.2%; version v0.1.13
- DynamoDB Local - grade C (coverage lowers this row to C); diverges 14.7% of the suite; covers 94.2%; diverges per tier Tier 1 7.5%, Tier 2 15.4%, Tier 3 22.6%; version ff89bd48ff32
- Dynalite - grade C (coverage lowers this row to C); diverges 10.7% of the suite; covers 68.6%; diverges per tier Tier 1 10.7%, Tier 2 8.1%, Tier 3 12.9%; version 4.0.0
- Kumo - grade F; diverges 44.7% of the suite; covers 78.6%; diverges per tier Tier 1 46.8%, Tier 2 13.9%, Tier 3 67.3%; version 76e97ef55735

---

# Data

- Latest run (JSON): https://paritysuite.org/data/latest.json
- All runs (JSON): https://paritysuite.org/data/runs.json
- Data index (JSON): https://paritysuite.org/data/index.json
- Runs feed (Atom): https://paritysuite.org/feed.xml

Published under CC BY 4.0; credit paritysuite.org.