Skip to content

Run

7 emulators 998 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so a downward arrow can be the new tests biting rather than a target regressing.

Suite grew from 982 to 998 tests this run.

That's 16 new tests measured against every target. Movement below compares to the previous run, so a fall here is as likely to be the stricter suite as a real regression.

Scored against 33 real regions. 1 dropped, out of scoring until it returns: me-south-1
  1. live (AWS) full coverage
    100% ground truth
    region all regions
    Tier 1 100%
    Tier 2 100%
    Tier 3 100%
  2. 94.7% -1.6pp fell 1.6 percentage points
    region eu-west-2 + 5 regions
    Tier 1 99.8%
    Tier 2 77.8%
    Tier 3 96.9%
  3. 84.2% -0.9pp fell 0.9 percentage points
    region 27 regions scores higher here than in eu-west-2
    Tier 1 93.5%
    Tier 2 81.2%
    Tier 3 72.2%
  4. region all regions
    Tier 1 85.5%
    Tier 2 86.2%
    Tier 3 78.8%
  5. 83.0% -0.9pp fell 0.9 percentage points
    region 27 regions scores higher here than in eu-west-2
    Tier 1 92.2%
    Tier 2 78.5%
    Tier 3 71.9%
  6. 7485cab02cd8 full coverage
    82.0% -1.3pp fell 1.3 percentage points
    region all regions
    Tier 1 89.1%
    Tier 2 70.4%
    Tier 3 78.7%
  7. 78.9% -1.3pp fell 1.3 percentage points
    region all regions
    Tier 1 91.4%
    Tier 2 61.6%
    Tier 3 70.7%
  8. region 27 regions scores higher here than in eu-west-2
    Tier 1 91.4%
    Tier 2 13.2%
    Tier 3 70.9%