MMMUBrowse 284 families
MMMU
Can the model combine visual evidence with college-level knowledge and deliberate reasoning?
- Domain
- Vision
- Ranked models
- 23
- Reported values
- 33
- Reports
- 5 · lab + maintainer
- Direction
- Higher is better
- Status
- Latest stable
Rankings
Best reported score per model across effort settings
| # | Model | Best setting | Score |
|---|---|---|---|
| 1 | Gemini 3.5 FlashGoogle | reportedcost n/a1 value · May 19 · source | 83.6% |
| 2 | GPT-5.6 SolOpenAI | maxcost n/a2 values · Jul 17 · source | 83.0% |
| 3 | Kimi K3Moonshot AIOW | maxcost n/a1 value · Jul 17 · source | 81.6% |
| 4 | Claude Fable 5Anthropic | maxcost n/a1 value · Jul 17 · source | 81.2% |
| 5 | GPT-5.5OpenAI | xhighcost n/a3 values · Jul 17 · source | 81.2% |
| 6 | Gemini 3 FlashGoogle | reportedcost n/a1 value · May 19 · source | 81.2% |
| 7 | GPT-5.4OpenAI | reportedcost n/a1 value · Mar 5 · source | 81.2% |
| 8 | Gemini 3 ProGoogle | highcost n/a1 value · Feb 19 · source | 81.0% |
| 9 | GPT-5.6 TerraOpenAI | reportedcost n/a1 value · Jul 9 · source | 80.7% |
| 10 | Gemini 3.1 Pro PreviewGoogle | reportedcost n/a2 values · Jul 9 · source | 80.5% |
Effort curve
Score vs. reported cost. Lines connect complete sweeps only.
Definition and comparison boundarypublic methodology
Can the model combine visual evidence with college-level knowledge and deliberate reasoning?
Multidiscipline multimodal questions filtered for genuine visual dependence under a no-tools protocol. Higher is better. MMMU Team's pro.overall percentage under the official zero-shot evaluation.
Tool access, exact source setup label, model publication date, model size, author-provided marker, and the maintainer's selected column are material. Do not mix this row with validation, test, or provider-reported values.
MMMU Team exact source label "Claude Opus 4.6 w/o tools"; pro.overall printed value 73.9; source marker author; publication date 2026-02-05; model type proprietary; size -; publisher provenance https://www.anthropic.com/news/claude-opus-4-6. MMMU Team exact source label "Claude Sonnet 4.6 w/o tools"; pro.overall printed value 74.5; source marker author; publication date 2026-02-17; model type proprietary; size -; publisher provenance https://www.anthropic.com/news/claude-sonnet-4-6. MMMU Team exact source label "Gemini 3.1 Pro Thinking (High)"; pro.overall printed value 80.5; source marker author; publication date 2026-02-19; model type proprietary; size -; publisher provenance https://deepmind.google/models/gemini/pro/. MMMU Team exact source label "Gemma 4 26B A4B"; pro.overall printed value 73.8; source marker author; publication date 2026-04-02; model type open_source; size 26B; publisher provenance https://deepmind.google/models/gemma/gemma-4/. MMMU Team exact source label "Gemma 4 31B"; pro.overall printed value 76.9; source marker author; publication date 2026-04-02; model type open_source; size 31B; publisher provenance https://deepmind.google/models/gemma/gemma-4/. MMMU Team exact source label "Gemma 4 E2B"; pro.overall printed value 44.2; source marker author; publication date 2026-04-02; model type open_source; size 2B; publisher provenance https://deepmind.google/models/gemma/gemma-4/. MMMU Team exact source label "Gemma 4 E4B"; pro.overall printed value 52.6; source marker author; publication date 2026-04-02; model type open_source; size 4B; publisher provenance https://deepmind.google/models/gemma/gemma-4/. MMMU Team exact source label "GPT-5.1 Thinking"; pro.overall printed value 79.0; source marker author; publication date 2025-11-13; model type proprietary; size -; publisher provenance https://openai.com/index/gpt-5-1-for-developers/. MMMU Team exact source label "GPT-5.2 Thinking w/o Python"; pro.overall printed value 80.4; source marker author; publication date 2025-12-11; model type proprietary; size -; publisher provenance https://openai.com/index/introducing-gpt-5-2/. MMMU Team exact source label "GPT-5.2 Thinking w/o tools"; pro.overall printed value 79.5; source marker author; publication date 2025-12-11; model type proprietary; size -; publisher provenance https://openai.com/index/introducing-gpt-5-2/. MMMU Team exact source label "GPT-5.4 Thinking w/o tools"; pro.overall printed value 81.2; source marker author; publication date 2026-03-05; model type proprietary; size -; publisher provenance https://openai.com/index/introducing-gpt-5-4/. MMMU Team exact source label "Muse Spark Thinking"; pro.overall printed value 80.4; source marker author; publication date 2026-04-08; model type proprietary; size -; publisher provenance https://ai.meta.com/blog/introducing-muse-spark-msl/. No tools.
Method / source ↗