Find the closest public benchmarks for a task — see which agent does best, or compare two models head-to-head →
Repositories Cost vs. quality Cost Speed
-150 -100 -50 0 50 $0.00 $109.04 $218.09 cost · $/run code quality (BT) gpt-5.5 · xhigh effort
rating 24 · $181.74 gpt-5.5 · high effort
rating 16 · $112.79 fable-5 · xhigh effort
rating 9 · $155.03 gpt-5.5 · medium effort
rating 8 · $79.05 fable-5 · medium effort
rating 1 · $135.62 opus-4.8 · low effort
rating -5 · $108.43 fable-5 · high effort
rating -9 · $158.95 opus-4.8 · xhigh effort
rating -11 · $112.15 fable-5 · low effort
rating -13 · $142.78 gpt-5.5 · low effort
rating -14 · $35.49 opus-4.8 · high effort
rating -18 · $94.52 sonnet-4.6 · high effort
rating -23 · $68.05 sonnet-4.6 · xhigh effort
rating -43 · $66.61 opus-4.8 · medium effort
rating -44 · $97.25 sonnet-4.6 · low effort
rating -49 · $60.60 sonnet-4.6 · medium effort
rating -54 · $52.07 glm-5.2
rating -84 · $45.75 X H X M M L H X L L H H X M L M gpt-5.5 fable-5 opus-4.8 sonnet-4.6 glm-5.2 Each color is a model family; connected dots are its effort variants, tagged L /M /H /X = low / medium / high / xhigh (hover a dot for exact numbers). Filled = on the value frontier · hollow = dominated (another agent matches or beats its quality for less money). Up-and-left is better value.
Agent standings — code quality, across all benchmarked repos Ranked by code quality (Bradley-Terry, timeouts excluded). completes % is how often the agent produces a diff at all; speed is the chart above.
01 codex/gpt-5.5 (xhigh)
completes 91% · 422s avg · $181.741/run · 4 runs
24elo
02 codex/gpt-5.5 (high)
completes 91% · 294s avg · $112.787/run · 4 runs
16elo
03 claude-code/fable-5 (xhigh)
completes 51% · 1539s avg · $155.034/run · 4 runs
9elo
04 codex/gpt-5.5 (medium)
completes 91% · 211s avg · $79.045/run · 4 runs
8elo
05 claude-code/fable-5 (medium)
completes 72% · 950s avg · $135.623/run · 4 runs
1elo
06 claude-code/opus-4.8 (low)
completes 82% · 908s avg · $108.432/run · 4 runs
-5elo
07 claude-code/fable-5 (high)
completes 67% · 1159s avg · $158.955/run · 4 runs
-9elo
08 claude-code/opus-4.8 (xhigh)
completes 88% · 859s avg · $112.149/run · 4 runs
-11elo
09 claude-code/fable-5 (low)
completes 87% · 566s avg · $142.778/run · 4 runs
-13elo
10 codex/gpt-5.5 (low)
completes 91% · 112s avg · $35.489/run · 4 runs
-14elo
11 claude-code/opus-4.8 (high)
completes 77% · 929s avg · $94.521/run · 4 runs
-18elo
12 claude-code/sonnet-4.6 (high)
completes 93% · 485s avg · $68.050/run · 4 runs
-23elo
13 claude-code/sonnet-4.6 (xhigh)
completes 90% · 494s avg · $66.614/run · 4 runs
-43elo
14 claude-code/opus-4.8 (medium)
completes 82% · 843s avg · $97.249/run · 4 runs
-44elo
15 claude-code/sonnet-4.6 (low)
completes 87% · 493s avg · $60.599/run · 4 runs
-49elo
16 claude-code/sonnet-4.6 (medium)
completes 86% · 441s avg · $52.071/run · 4 runs
-54elo
17 opencode/glm-5.2
completes 89% · 778s avg · $45.754/run · 4 runs
-84elo