Sandbox BenchBrowse 284 families
Sandbox Bench
Can the model diagnose or complete difficult cybersecurity tasks?
- Domain
- Cybersecurity
- Ranked models
- 2
- Reported values
- 2
- Reports
- 1 · lab
- Direction
- Higher is better
- Status
- Latest stable
Rankings
Best reported score per model across effort settings
| # | Model | Best setting | Score |
|---|---|---|---|
| 1 | GPT-6 AstraOpenAI | reportedcost n/a1 value · Sep 3 · source | 45.5% |
| 2 | GPT-5.6 SolOpenAI | reportedcost n/a1 value · Sep 3 · source | 4.5% |
Effort curve
Score vs. reported cost. Lines connect complete sweeps only.
Definition and comparison boundaryinternal methodology
Can the model diagnose or complete difficult cybersecurity tasks?
Security analysis or exploitation under a controlled evaluation environment. Higher is better. Percentage of private targets on which the model recovered the protected flag.
This is a publisher-defined internal evaluation. The task set or grading details are not fully public, so treat it as directional evidence.
OpenAI reports Astra on 10 of 22 targets (45.5%) and GPT-5.6 Sol on 1 of 22 (4.5%); Astra's successes were four of five runtime targets, five of fourteen parser targets, and one of three egress proxies. The affected software details are withheld during coordinated disclosure.
Method / source ↗