Sandbox Bench

Can the model diagnose or complete difficult cybersecurity tasks?

Security analysis or exploitation under a controlled evaluation environment. Higher is better. Percentage of private targets on which the model recovered the protected flag.

This is a publisher-defined internal evaluation. The task set or grading details are not fully public, so treat it as directional evidence.

OpenAI reports Astra on 10 of 22 targets (45.5%) and GPT-5.6 Sol on 1 of 22 (4.5%); Astra's successes were four of five runtime targets, five of fourteen parser targets, and one of three egress proxies. The affected software details are withheld during coordinated disclosure.