Skip to content

Run

7 emulators 954 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in. The suite grew this run, so a downward arrow can be the new tests biting rather than a target regressing.

Suite grew from 873 to 954 tests this run.

That's 81 new tests measured against every target. Movement below compares to the previous run, so a fall here is as likely to be the stricter suite as a real regression.

  1. live (AWS) full coverage
    100% ground truth
    Tier 1 100%
    Tier 2 100%
    Tier 3 100%
  2. 99.4% -0.6pp fell 0.6 percentage points
    Tier 1 100.0%
    Tier 2 99.3%
    Tier 3 98.4%
  3. 0b5dba5153fc full coverage
    85.5% -1.1pp fell 1.1 percentage points
    Tier 1 89.1%
    Tier 2 87.4%
    Tier 3 79.4%
  4. 84.9% +0.8pp rose 0.8 percentage points
    Tier 1 93.5%
    Tier 2 86.1%
    Tier 3 71.6%
  5. 83.6% +1.0pp rose 1.0 percentage points
    Tier 1 92.2%
    Tier 2 82.9%
    Tier 3 71.3%
  6. 83.1% -2.9pp fell 2.9 percentage points
    Tier 1 85.5%
    Tier 2 86.2%
    Tier 3 78.4%
  7. 82.1% -2.6pp fell 2.6 percentage points
    Tier 1 91.4%
    Tier 2 76.0%
    Tier 3 71.3%
  8. 75.8% +0.7pp rose 0.7 percentage points
    Tier 1 91.4%
    Tier 2 13.2%
    Tier 3 70.6%