The Most Annoying AI Benchmark

WHAT THIS IS A failure-trap coding benchmark. Fixed coding traps, recorded answers, dual checks, and receipt roots. It counts verified mistakes, not general model quality.

WHAT / WHY / HOW

  1. 01

    What

    A specialized coding benchmark built from fixed, annoying failure traps. It tracks catches, clean results, and unresolved evidence separately.

  2. 02

    Why

    Broad evaluations can miss small, plausible coding mistakes. This suite makes those mistakes inspectable instead of compressing them into a single score.

  3. 03

    How

    Each model gets the same sealed pack. We score only the recorded answer, then require independent checks before a mistake counts.

  4. 04

    Limits

    This is not a universal model ranking. Read it beside HumanEval-style tests, agent benchmarks, and preference arenas.

The claim is narrow on purpose: a transparent, dual-checked count of verified mistakes on a fixed pack of coding traps. Complete rows can be ranked; incomplete rows stay visible but withheld.

Fixed pack
The same trap set is used for comparable ranked runs
Recorded answer
The submitted model reply is the scored artifact
Dual check
A catch needs reproduced agreement before it counts
Receipts
Hashes bind public evidence to the released archive

Certified rankings

Complete GavelTM-scored runs only

Annoying Mistakes Clean Results Unresolved Missing
Least annoyingMost annoying

Only complete GenXis GavelTM-certified models appear in the ranked chart. Incomplete or withheld evidence stays visible below without being counted as a failure.

LATEST RUN

Model results

0 models
Loading verified results

Nothing is ranked until its evidence is complete.

Ranked complete results

Shown by selected sort. Benchmark rank #1 means fewest verified mistakes; ties share rank.

    AUDITABILITY

    Unresolved runs.

    Incomplete runs

    Unresolved evidence. Not certified. Rows follow the selected sort; P-ranks still use resolved-only provisional quality.

      Not Yet Tested

      Tracked models with no benchmark attempt for this exact ID.

        Limits

        Only complete runs get a benchmark rank. Ties share it. OpenRouter's weekly order is a separate list. Missing or incomplete is not a zero. Scores are the latest 50-trap pack until enough 100-task cycles exist for a rolling average.

        AUDIT FILES

        Download the proof JSON.

        One week after each test completes, we publish one JSON file with the challenge questions, exact prompts, recorded model responses, judge reasons, and receipts. Late-added models may be tested afterward to keep the benchmark cadence comparable; those after-the-fact rows are shown on the live page but are not part of an earlier date-sealed JSON bundle.

        Loading evidence archive.

          SHARE

          Use the result.