Martin Hicks · Parity Suite - DynamoDB conformance suite

Results Runs

8 emulators 998 tests

Results as of this run. The arrow shows each target's movement since the previous run it was tested in.

When this run was published the board led with a single correctness percentage. The figures here are that run's own results restated as divergence and coverage, so they read differently from what this page showed at the time. The changelog has the change.

  1. DynamoDB

    live (AWS)

    Runs via: AWS service

    Grade baseline

    no divergence in all 33 regions · covers 100.0% of the suite

    ground truth

    Tier breakdown

    Tier 1 · Core 0.0% diverges

    100.0% covered

    Tier 2 · Complete 0.0% diverges

    100.0% covered

    Tier 3 · Strict 0.0% diverges

    100.0% covered

  2. Grade A

    no divergence in 6 regions · up to 0.3% in the other 27 · covers 98.6% of the suite · coverage lowers this row to A

    diverged 5.2 percentage points less

    Tier breakdown

    Tier 1 · Core 0.0% diverges

    100.0% covered

    Tier 2 · Complete 0.0% diverges

    93.0% covered 14 unsupported

    Tier 3 · Strict 0.0% diverges

    100.0% covered

    Also built for WebAssembly / OPFS
    • Grade B

      no divergence in 6 regions · up to 0.3% in the other 26 · covers 78.7% · coverage lowers this row to B

  3. Grade A

    low divergence (1.6%) in all 33 regions · covers 91.3% of the suite

    diverged 13.7 percentage points less

    Tier breakdown

    Tier 1 · Core 0.0% diverges

    100.0% covered

    Tier 2 · Complete 3.5% diverges

    58.3% covered 83 unsupported

    Tier 3 · Strict 2.8% diverges

    98.8% covered 4 unsupported

  4. Grade C

    moderate divergence (12.3%) in 27 regions · up to 12.5% in the other 6 · covers 80.0% of the suite · coverage lowers this row to C

    diverged 9.0 percentage points less

    Tier breakdown

    Tier 1 · Core 8.6% diverges

    100.0% covered

    Tier 2 · Complete 14.1% diverges

    20.1% covered 159 unsupported

    Tier 3 · Strict 16.7% diverges

    87.3% covered 41 unsupported

  5. LocalStack

    Runs via: Docker, pip, Homebrew, binary

    Grade C

    high divergence (15.6%) in 27 regions · up to 15.7% in the other 6 · covers 99.2% of the suite

    diverged 0.1 percentage points less

    Tier breakdown

    Tier 1 · Core 6.5% diverges

    100.0% covered

    Tier 2 · Complete 18.1% diverges

    96.0% covered 8 unsupported

    Tier 3 · Strict 27.5% diverges

    100.0% covered

  6. Grade C

    high divergence (15.9%) in 27 regions · up to 16.0% in the other 6 · covers 97.9% of the suite

    diverged 0.9 percentage points less

    Tier breakdown

    Tier 1 · Core 7.8% diverges

    100.0% covered

    Tier 2 · Complete 16.1% diverges

    89.4% covered 21 unsupported

    Tier 3 · Strict 27.8% diverges

    100.0% covered

  7. Ministack

    c3c2e86f19ff

    Runs via: pip, Docker, source

    Grade C

    high divergence (18.0%) in all 33 regions · covers 100.0% of the suite

    unchanged

    Tier breakdown

    Tier 1 · Core 10.9% diverges

    100.0% covered

    Tier 2 · Complete 29.6% diverges

    100.0% covered

    Tier 3 · Strict 21.3% diverges

    100.0% covered

  8. Floci

    Runs via: Docker, Homebrew, install script, Scoop, binary, JAR

    Grade C

    high divergence (20.9%) in all 33 regions · covers 99.1% of the suite

    unchanged

    Tier breakdown

    Tier 1 · Core 8.6% diverges

    100.0% covered

    Tier 2 · Complete 36.7% diverges

    95.5% covered 9 unsupported

    Tier 3 · Strict 29.3% diverges

    100.0% covered

Older run Newer run

Read the original on paritysuite.org ↗