AstaBenchBrowse 284 families
AstaBench
Can the model reason accurately about difficult scientific material?
Version / measure
- Domain
- Science
- Ranked models
- 7
- Reported values
- 7
- Reports
- 1 · lab
- Direction
- Higher is better
- Status
- Version not specified
Rankings
Best reported score per model across effort settings
| # | Model | Best setting | Score |
|---|---|---|---|
| 1 | Claude Opus 4.7Anthropic | max$3.54 / task1 value · Apr 30 · source | 58.0% |
| 2 | Claude Opus 4.6Anthropic | maxcost n/a1 value · Apr 30 · source | 55.3% |
| 3 | Claude Sonnet 4.6Anthropic | maxcost n/a1 value · Apr 30 · source | 54.5% |
| 4 | Asta v0Allen AIOW | reportedcost n/a1 value · Apr 30 · source | 53.0% |
| 5 | GPT-5.5OpenAI | xhigh$1.61 / task1 value · Apr 30 · source | 52.9% |
| 6 | Gemini 3.1 Pro PreviewGoogle | highcost n/a1 value · Apr 30 · source | 49.6% |
| 7 | GPT-5.4OpenAI | xhighcost n/a1 value · Apr 30 · source | 46.5% |
Effort curve
Score vs. reported cost. Lines connect complete sweeps only.
Top swept score
Claude Opus 4.7 · max reaches 58.0% at $3.54/task.
No cost-linked effort sweep for this version.
Definition and comparison boundarypublic methodology
Can the model reason accurately about difficult scientific material?
Scientific knowledge and reasoning under the benchmark's published question set. Higher is better. The value is the percentage reported in this lab's table.
Harness, tools, prompts, attempt count, and benchmark version can materially change the result. Compare within one reporting context.
ReAct agent framework. The source directly reports observed per-problem cost only for Claude Opus 4.7 ($3.54) and GPT-5.5 ($1.61).
Method / source ↗