SWE-BenchBrowse 284 families
SWE-Bench
Can the model complete substantial programming work under this benchmark's agent setup?
- Domain
- Coding
- Ranked models
- 10
- Reported values
- 10
- Reports
- 1 · maintainer
- Direction
- Higher is better
- Status
- Latest stable
Rankings
Best reported score per model across effort settings
| # | Model | Best setting | Score |
|---|---|---|---|
| 1 | Claude Opus 5Anthropic | xhighcost n/a1 value · Sep 22 · source | 99.40% |
| 2 | Claude Fable 5.1Anthropic | highcost n/a1 value · Sep 22 · source | 99.10% |
| 3 | Kimi K3Moonshot AIOW | maxcost n/a1 value · Sep 22 · source | 97.70% |
| 4 | GPT-6 AstraOpenAI | highcost n/a1 value · Sep 22 · source | 96.90% |
| 5 | GLM-5.3Z.aiOW | maxcost n/a1 value · Sep 22 · source | 95.60% |
| 6 | GPT-5.6 SolOpenAI | xhighcost n/a1 value · Sep 22 · source | 95.50% |
| 7 | Gemini 3.8 FlashGoogle | highcost n/a1 value · Sep 22 · source | 94.86% |
| 8 | Claude Sonnet 5Anthropic | xhighcost n/a1 value · Sep 22 · source | 93.15% |
| 9 | GPT-5.6 TerraOpenAI | xhighcost n/a1 value · Sep 22 · source | 92.37% |
| 10 | InklingThinking Machines LabOW | xhighcost n/a1 value · Sep 22 · source | 89.88% |
Effort curve
Score vs. reported cost. Lines connect complete sweeps only.
Definition and comparison boundarypublic methodology
Can the model complete substantial programming work under this benchmark's agent setup?
Software implementation, debugging, or repository work under the published evaluation protocol. Higher is better. Resolve Rate on the official 642-task Public Full split across 11 repositories.
Scale's V2 release changes task composition and protocol relative to the 731-task V1 Public set. Compare only rows in this explicit V2 Full view; exact harness, effort, source label, rank, and displayed uncertainty are retained per model.
Exact Scale AI label Opus 5 (Claude Code) xhigh; harness Claude Code; effort xhigh; rank 1; displayed score 99.40%; displayed uncertainty ±0.40; swe-bench-pro-v2-public-642. Exact Scale AI label Fable 5.1 (Claude Code) high; harness Claude Code; effort high; rank 2; displayed score 99.10%; displayed uncertainty ±0.50; swe-bench-pro-v2-public-642. Exact Scale AI label Kimi-K3 (mini-swe-agent) max; harness mini-swe-agent; effort max; rank 3; displayed score 97.70%; displayed uncertainty ±0.90; swe-bench-pro-v2-public-642. Exact Scale AI label GPT-6-Astra (Codex) high; harness Codex; effort high; rank 4; displayed score 96.90%; displayed uncertainty ±1.10; swe-bench-pro-v2-public-642. Exact Scale AI label GLM-5.3 (mini-swe-agent) max; harness mini-swe-agent; effort max; rank 5; displayed score 95.60%; displayed uncertainty ±1.30; swe-bench-pro-v2-public-642. Exact Scale AI label GPT-5.6-sol (Codex) xhigh; harness Codex; effort xhigh; rank 6; displayed score 95.50%; displayed uncertainty ±1.40; swe-bench-pro-v2-public-642. Exact Scale AI label Gemini 3.8 Flash (mini-swe-agent) high; harness mini-swe-agent; effort high; rank 7; displayed score 94.86%; displayed uncertainty ±1.46; swe-bench-pro-v2-public-642. Exact Scale AI label Sonnet 5 (Claude Code) xhigh; harness Claude Code; effort xhigh; rank 8; displayed score 93.15%; displayed uncertainty ±1.71; swe-bench-pro-v2-public-642. Exact Scale AI label GPT-5.6-Terra (Codex) xhigh; harness Codex; effort xhigh; rank 9; displayed score 92.37%; displayed uncertainty ±1.81; swe-bench-pro-v2-public-642. Exact Scale AI label Inkling (mini-swe-agent) xhigh; harness mini-swe-agent; effort xhigh; rank 10; displayed score 89.88%; displayed uncertainty ±2.10; swe-bench-pro-v2-public-642.
Method / source ↗