AI model comparison

Chat, image and video AI models
in one place.

Language models from Anthropic, OpenAI, Google, Meta, xAI, DeepSeek, Alibaba, Moonshot, Mistral and NVIDIA: release dates, pricing, context window and published benchmark results.

Data as of 17 September 2026 · AI tools guide →

109models
10companies
96.0%GPQA Diamond · GPT-6 Astra
24open-weight models

Scores over time

Each dot is a model. Click a company to show or hide it. Models without a published score are not plotted.

PhD-level science questions

Release timeline by year

20235 models
  1. GPT-3.5 Turbo
  2. GPT-4
  3. Claude
  4. Claude 2
  5. Claude 2.1
202424 models
  1. Gemini 1.0 Pro
  2. Grok-1.5
  3. Claude 3 Opus
  4. Claude 3 Sonnet
  5. Claude 3 Haiku
  6. GPT-4 Turbo
  7. GPT-4o
  8. Gemini 1.5 Pro
  9. Gemini 1.5 Flash
  10. DeepSeek-V2.5
  11. Codestral 22B
  12. Claude 3.5 Sonnet
  13. GPT-4o mini
  14. Llama 3.1 405B
  15. Mistral Large 2
  16. Grok-2
  17. Qwen2.5 72B
  18. Llama 3.1 Nemotron 70B
  19. Claude 3.5 Sonnet (new)
  20. Claude 3.5 Haiku
  21. o1
  22. Gemini 2.0 Flash
  23. Llama 3.3 70B
  24. DeepSeek-V3
202537 models
  1. o3-mini
  2. DeepSeek-R1
  3. Mistral Small 3
  4. GPT-4.5
  5. Grok-3
  6. Claude 3.7 Sonnet
  7. Gemma 3 27B
  8. GPT-4.1
  9. o3
  10. o4-mini
  11. Llama 4 Maverick
  12. Qwen3 32B
  13. Llama 3.1 Nemotron Ultra 253B
  14. Gemini 2.5 Pro
  15. Gemini 2.5 Flash
  16. Claude Opus 4
  17. Claude Sonnet 4
  18. Grok-4
  19. Grok-4 Heavy
  20. Qwen3-Coder 480B
  21. Kimi K2
  22. GPT OSS 120B
  23. GPT-5
  24. Nemotron Nano 9B v2
  25. Claude Opus 4.1
  26. Claude Sonnet 4.5
  27. Claude Haiku 4.5
  28. GPT-5.1
  29. Gemini 3 Pro
  30. Grok-4.1 Fast
  31. Claude Opus 4.5
  32. GPT-5.2
  33. Gemini 3 Flash
  34. DeepSeek-V3.2
  35. Qwen3 Max
  36. Mistral Large 3
  37. Nemotron 3 Nano (30B)
202643 models
  1. Kimi K2.5
  2. GPT-5.3 Codex
  3. Gemini 3.1 Pro
  4. Qwen3.5-397B-A17B
  5. Claude Opus 4.6
  6. Claude Sonnet 4.6
  7. GPT-5.4
  8. GPT-5.4 mini
  9. Qwen3.6 Plus
  10. Mistral Small 4
  11. Nemotron 3 Super (120B)
  12. GPT-5.5
  13. Gemma 4 31B
  14. Muse Spark
  15. DeepSeek-V4-Pro-Max
  16. DeepSeek-V4-Flash-Max
  17. Kimi K2.6
  18. Mistral Medium 3.5
  19. Claude Mythos Preview
  20. Claude Opus 4.7
  21. Gemini 3.5 Flash
  22. Grok 4.3
  23. Qwen3.7 Max
  24. Claude Opus 4.8
  25. Nemotron 3 Ultra (550B)
  26. Claude Fable 5
  27. Claude Sonnet 5
  28. GPT-5.6 Sol
  29. GPT-5.6 Terra
  30. GPT-5.6 Luna
  31. Gemini 3.6 Flash
  32. Muse Spark 1.1
  33. Grok 4.5
  34. Kimi K3
  35. Claude Opus 5
  36. Grok 4.6
  37. Qwen3.8 Max
  38. Nemotron 3.5 Lightning (30B)
  39. GPT-6 Astra
  40. Gemini 3.8 Flash
  41. Muse Spark 1.3
  42. DeepSeek-V4.1-Flash
  43. Claude Fable 5.1

Capabilities: who can do what, and how well?

Some models cannot read images, use tools or operate a computer. Among those that can, scores vary widely.

👁

Vision

Can read and understand photos, screenshots, documents and charts.

✓ 64✕ 25? 20

MMMU-Pro · Vision: multi-discipline questions with charts, diagrams and images (hard version) · models since Sep 2025

Best

  1. Gemini 3.5 Flash83.6
  2. GPT-5.583.2
  3. GPT-5.6 Sol83.0

Weakest

  1. DeepSeek-V4.1-Flash56.5
  2. Claude Sonnet 4.675.6
  3. GPT-5.4 mini76.6

Without this capability: Nemotron 3 Nano (30B), DeepSeek-V3.2, Nemotron Nano 9B v2, GPT OSS 120B, Kimi K2, Qwen3-Coder 480B, Llama 3.1 Nemotron Ultra 253B, Qwen3 32B +17

🛠

Agent / tool use

Can run tasks on its own by using tools such as web search, code execution, APIs and apps.

✓ 93✕ 4? 12

Terminal-Bench 2.x · Agentic tasks in the terminal · models since Sep 2025

Best

  1. Gemini 3.8 Flash90.8
  2. Claude Opus 589.1
  3. GPT-5.6 Sol88.8

Weakest

  1. Claude Haiku 4.541.8
  2. DeepSeek-V3.246.4
  3. Gemini 3 Flash47.6

Without this capability: DeepSeek-R1, Grok-1.5, Claude 2, Claude

🖥

Computer use

Can see the screen and operate a computer or browser with mouse and keyboard like a person.

✓ 33✕ 36? 40

OSWorld-Verified · Using a computer (screen, mouse, keyboard) · models since Sep 2025

Best

  1. Qwen3.8 Max86.1
  2. Claude Fable 585.0
  3. Kimi K384.8

Weakest

  1. Qwen3.6 Plus62.5
  2. GPT-5.3 Codex64.7
  3. GPT-5.4 mini72.1

Without this capability: Nemotron 3 Nano (30B), DeepSeek-V3.2, Nemotron Nano 9B v2, GPT OSS 120B, Kimi K2, Qwen3-Coder 480B, Llama 3.1 Nemotron Ultra 253B, Qwen3 32B +28

All models

By capability:

Click a column header to sort · 109 models shown

Model Company Released Capabilities Humanity's Last ExamGPQA DiamondSWE-bench Verified🛠 Terminal-Bench 2.x🛠 τ²-Bench (Telecom)🖥 OSWorld-Verified👁 MMMU-Pro👁 MMMU Price $ / MTok (in / out) Context Open weights
Claude Fable 5.1 Anthropic
👁✓ —
🛠✓ 55.8TB 4.0
🖥✓ —
65.0S — — — — — — — $10 / $50 1M —
DeepSeek-V4.1-Flash DeepSeek
👁✓ 56.5MMMU-Pro
🛠✓ —
🖥? —
36.8S 90.9S — — — — 56.5D — $0.22 / $0.66 1M —
Muse Spark 1.3 Meta
👁? —
🛠✓ —
🖥? —
— — — — — — — — $0.1 / $0.2 1M —
Gemini 3.8 Flash Google
👁✓ —
🛠✓ 90.8TB 2.x
🖥✓ 59.0OSWorld 2.0
— — — 90.8D — — — — $0.75 / $3.75 1M —
GPT-6 Astra OpenAI
👁✓ 92.7ScreenSpot-Pro
🛠✓ 57.7TB 4.0
🖥✓ 72.6OSWorld 2.0
— 96.0S — — — — — — $10 / $50 1.1M —
Nemotron 3.5 Lightning (30B) NVIDIA
👁? —
🛠✓ —
🖥? —
— — — — — — — — $0.08 / $0.2 262K Yes
Qwen3.8 Max Alibaba (Qwen)
👁✓ 82.3MMMU-Pro
🛠✓ —
🖥✓ 86.1OSWorld-V
43.6S 92.6S — — — 86.1S 82.3S — $1.65 / $4.95 1M —
Grok 4.6 xAI
👁? —
🛠✓ —
🖥? —
— — — — — — — — $2 / $6 500K —
Claude Opus 5 Anthropic
👁✓ —
🛠✓ 89.1TB 2.x
🖥✓ —
64.7S — — 89.1D — — — — $5 / $25 1M —
Kimi K3 Moonshot (Kimi)
👁✓ 81.6MMMU-Pro
🛠✓ 88.3TB 2.x
🖥✓ 84.8OSWorld-V
56.0S 93.5S — 88.3O — 84.8O 81.6S — $2.85 / $14.3 1M —
Grok 4.5 xAI
👁? —
🛠✓ 83.3TB 2.x
🖥? —
— 93.0S — 83.3D — — — — $2 / $6 500K —
Muse Spark 1.1 Meta
👁✓ —
🛠✓ —
🖥✓ 80.8OSWorld-V
62.1S — — — — 80.8S — — $1.25 / $4.25 1M —
Gemini 3.6 Flash Google
👁✓ —
🛠✓ —
🖥✓ 83.0OSWorld-V
— — — — — 83.0S — — $1.5 / $7.5 1M —
GPT-5.6 Luna OpenAI
👁✓ 78.4MMMU-Pro
🛠✓ 84.7TB 2.x
🖥✓ 45.6OSWorld 2.0
— 92.3S — 84.7D — — 78.4S — $0.2 / $1.2 1.1M —
GPT-5.6 Terra OpenAI
👁✓ 80.7MMMU-Pro
🛠✓ 87.4TB 2.x
🖥✓ 50.2OSWorld 2.0
— 92.9S — 87.4D — — 80.7S — $2 / $12 1.1M —
GPT-5.6 Sol OpenAI
👁✓ 83.0MMMU-Pro
🛠✓ 88.8TB 2.x
🖥✓ 65.7OSWorld 2.0
— 94.6S — 88.8D — — 83.0S — $5 / $30 1.1M —
Claude Sonnet 5 Anthropic
👁✓ —
🛠✓ 80.4TB 2.x
🖥✓ 81.2OSWorld-V
57.4S — 85.2S 80.4S — 81.2S — — $2 / $10 1M —
Claude Fable 5 Anthropic
👁✓ 87.3ScreenSpot-Pro
🛠✓ —
🖥✓ 85.0OSWorld-V
64.5S — 95.0S — — 85.0S — — $10 / $50 1M —
Nemotron 3 Ultra (550B) NVIDIA
👁? —
🛠✓ —
🖥? —
37.4S — — — — — — — $0.5 / $2.2 262K Yes
Claude Opus 4.8 Anthropic
👁✓ —
🛠✓ 74.6TB 2.x
🖥✓ 83.4OSWorld-V
57.9S 93.6S 88.6S 74.6S — 83.4S — — $5 / $25 1M —
Qwen3.7 Max Alibaba (Qwen)
👁? —
🛠✓ 69.7TB 2.x
🖥? —
41.4S 92.4S 80.4S 69.7S — — — — $1.25 / $3.75 1M —
Grok 4.3 xAI
👁? —
🛠✓ —
🖥? —
— — — — — — — — $1.25 / $2.5 1M —
Gemini 3.5 Flash Google
👁✓ 83.6MMMU-Pro
🛠✓ 76.2TB 2.x
🖥✓ 78.4OSWorld-V
40.2S — — 76.2S — 78.4S 83.6S — $1.5 / $9 1M —
Claude Opus 4.7 Anthropic
👁✓ —
🛠✓ 69.4TB 2.x
🖥✓ 78.0OSWorld-V
54.7S 94.2S 87.6S 69.4S — 78.0S — — $5 / $25 1M —
Claude Mythos Preview Anthropic
👁✓ —
🛠✓ 82.0TB 2.x
🖥✓ 79.6OSWorld-V
64.7S 94.6S 93.9S 82.0S — 79.6S — — — — —
Mistral Medium 3.5 Mistral AI
👁? —
🛠✓ —
🖥? —
— — 77.6S — — — — — $1.5 / $7.5 256K —
Kimi K2.6 Moonshot (Kimi)
👁✓ 80.1MMMU-Pro
🛠✓ 66.7TB 2.x
🖥✓ 73.1OSWorld-V
36.4S 90.5S 80.2S 66.7S — 73.1S 80.1S — $0.75 / $3.5 262K —
DeepSeek-V4-Flash-Max DeepSeek
👁? —
🛠✓ 56.9TB 2.x
🖥? —
45.1S 88.1S 79.0S 56.9S — — — — $0.09 / $0.18 1M —
DeepSeek-V4-Pro-Max DeepSeek
👁? —
🛠✓ 67.9TB 2.x
🖥? —
48.2S 90.1S 80.6S 67.9S — — — — $1.3 / $2.6 1M —
Muse Spark Meta
👁✓ 80.4MMMU-Pro
🛠✓ 59.0TB 2.x
🖥? —
58.4S 89.5S 77.4S 59.0S 91.5S — 80.4S — — — —
Gemma 4 31B Google
👁✓ 76.9MMMU-Pro
🛠✓ —
🖥? —
— 84.3O — — — — 76.9S — $0.09 / $0.34 262K Yes
GPT-5.5 OpenAI
👁✓ 83.2MMMU-Pro
🛠✓ 82.7TB 2.x
🖥✓ 78.7OSWorld-V
52.2S 93.6S — 82.7S 98.0S 78.7S 83.2S — $5 / $30 1.1M —
Nemotron 3 Super (120B) NVIDIA
👁? —
🛠✓ 64.4τ²-Bench
🖥? —
— 79.2O — — 64.4S — — — $0.09 / $0.4 262K Yes
Mistral Small 4 Mistral AI
👁? —
🛠✓ —
🖥? —
— — — — — — — — $0.15 / $0.6 256K —
Qwen3.6 Plus Alibaba (Qwen)
👁✓ 78.8MMMU-Pro
🛠✓ 61.6TB 2.x
🖥✓ 62.5OSWorld-V
— 90.4S 78.8S 61.6S — 62.5S 78.8S 86.0S $0.5 / $3 1M —
GPT-5.4 mini OpenAI
👁✓ 76.6MMMU-Pro
🛠✓ 60.0TB 2.x
🖥✓ 72.1OSWorld-V
— 88.0S — 60.0S 93.4S 72.1S 76.6S — $0.75 / $4.5 400K —
GPT-5.4 OpenAI
👁✓ 81.2MMMU-Pro
🛠✓ 75.1TB 2.x
🖥✓ 75.0OSWorld-V
39.8S 92.8S — 75.1S 98.9S 75.0S 81.2S — $2.5 / $15 1M —
Claude Sonnet 4.6 Anthropic
👁✓ 75.6MMMU-Pro
🛠✓ 59.1TB 2.x
🖥✓ 78.5OSWorld-V
49.0S 89.9S 79.6S 59.1S 97.9S 78.5D 75.6S — $3 / $15 1M —
Claude Opus 4.6 Anthropic
👁✓ 77.3MMMU-Pro
🛠✓ 65.4TB 2.x
🖥✓ —
53.1S 91.3S 80.8S 65.4S 99.3S — 77.3S — $5 / $25 1M —
Qwen3.5-397B-A17B Alibaba (Qwen)
👁? —
🛠✓ 52.5TB 2.x
🖥? —
— 88.4S 76.4S 52.5S — — — — $0.45 / $3 262K Yes
Gemini 3.1 Pro Google
👁✓ 80.5MMMU-Pro
🛠✓ 68.5TB 2.x
🖥? —
51.4S 94.3S 80.6S 68.5S — — 80.5S — $2 / $12 1M —
GPT-5.3 Codex OpenAI
👁✓ —
🛠✓ 77.3TB 2.x
🖥✓ 64.7OSWorld-V
— — — 77.3S — 64.7S — — $1.75 / $14 400K —
Kimi K2.5 Moonshot (Kimi)
👁✓ 78.5MMMU-Pro
🛠✓ 50.8TB 2.x
🖥? —
50.2S 87.6S 76.8S 50.8S — — 78.5S — — — —
Nemotron 3 Nano (30B) NVIDIA
👁✕ —
🛠✓ 42.2τ²-Bench
🖥✕ —
— — — — 42.2S — — — $0.05 / $0.2 262K Yes
Mistral Large 3 Mistral AI
👁? —
🛠✓ —
🖥? —
— — — — — — — — $0.5 / $1.5 262K Yes
Qwen3 Max Alibaba (Qwen)
👁? —
🛠✓ —
🖥? —
— — — — — — — — $1.2 / $6 256K —
DeepSeek-V3.2 DeepSeek
👁✕ —
🛠✓ 46.4TB 2.x
🖥✕ —
40.8S 82.4O — 46.4S — — — — $0.26 / $0.38 164K Yes
Gemini 3 Flash Google
👁✓ 81.2MMMU-Pro
🛠✓ 47.6TB 2.x
🖥? —
43.5S 90.4S 78.0S 47.6S — — 81.2S — $0.5 / $3 1M —
GPT-5.2 OpenAI
👁✓ 79.5MMMU-Pro
🛠✓ 98.7τ²-Bench
🖥? —
34.5S 92.4S 80.0S — 98.7S — 79.5S — $1.75 / $14 400K —
Claude Opus 4.5 Anthropic
👁✓ —
🛠✓ 59.3TB 2.x
🖥✓ —
— 87.0O 80.9S 59.3S 98.2S — — — $5 / $25 200K —
Grok-4.1 Fast xAI
👁? —
🛠✓ —
🖥? —
— — — — — — — — $0.2 / $0.5 2M —
Gemini 3 Pro Google
👁✓ 81.0MMMU-Pro
🛠✓ 54.2TB 2.x
🖥? —
45.8S 91.9S 76.2S 54.2S — — 81.0S — — — —
GPT-5.1 OpenAI
👁✓ 85.4MMMU
🛠✓ 95.6τ²-Bench
🖥? —
— 88.1S 76.3S — 95.6S — — 85.4S $1.25 / $10 400K —
Claude Haiku 4.5 Anthropic
👁✓ —
🛠✓ 41.8TB 2.x
🖥✓ —
— 73.0O 73.3S 41.8O 83.0S — — — $1 / $5 200K —
Claude Sonnet 4.5 Anthropic
👁✓ —
🛠✓ —
🖥✓ 61.4OSWorld*
— 83.4O 77.2O — — 61.4*O — — $3 / $15 200K —
Claude Opus 4.1 Anthropic
👁✓ —
🛠✓ —
🖥✓ —
— 80.9O 74.5S — — — — — $15 / $75 200K —
Nemotron Nano 9B v2 NVIDIA
👁✕ —
🛠✓ —
🖥✕ —
— — — — — — — — — — Yes
GPT-5 OpenAI
👁✓ 78.4MMMU-Pro
🛠✓ 96.7τ²-Bench
🖥? —
— — 74.9S — 96.7S — 78.4S 84.2S — — —
GPT OSS 120B OpenAI
👁✕ —
🛠✓ —
🖥✕ —
— 80.1O 62.4O — — — — — $0.04 / $0.17 131K Yes
Kimi K2 Moonshot (Kimi)
👁✕ —
🛠✓ 65.8τ²-Bench
🖥✕ —
— 75.1O 65.8O — 65.8S — — — — — Yes
Qwen3-Coder 480B Alibaba (Qwen)
👁✕ —
🛠✓ 37.5TB 2.x
🖥✕ —
— — 67.0O 37.5S — — — — $0.3 / $1 262K Yes
Grok-4 Heavy xAI
👁? —
🛠✓ —
🖥? —
50.7S 88.4S — — — — — — — — —
Grok-4 xAI
👁? —
🛠✓ —
🖥? —
40.0S 87.5S — — — — — — — — —
Claude Sonnet 4 Anthropic
👁✓ 74.4MMMU
🛠✓ —
🖥✓ 42.2OSWorld*
— — 72.7O — — 42.2*O — 74.4S $3 / $15 200K —
Claude Opus 4 Anthropic
👁✓ —
🛠✓ —
🖥✓ —
— — 72.5O — — — — — $15 / $75 200K —
Gemini 2.5 Flash Google
👁✓ 79.7MMMU
🛠✓ —
🖥? —
— 82.8O 60.4O — — — — 79.7S $0.3 / $2.5 1M —
Gemini 2.5 Pro Google
👁✓ 79.6MMMU
🛠✓ —
🖥? —
— 84.0O 63.8O — — — — 79.6S $1.25 / $10 1M —
Llama 3.1 Nemotron Ultra 253B NVIDIA
👁✕ —
🛠✓ —
🖥✕ —
— 72.8O — — — — — — — 128K Yes
Qwen3 32B Alibaba (Qwen)
👁✕ —
🛠✓ —
🖥✕ —
— — — — — — — — $0.08 / $0.28 128K Yes
Llama 4 Maverick Meta
👁✓ 73.4MMMU
🛠✓ —
🖥? —
— 69.8O — — — — — 73.4S $0.2 / $0.8 1M Yes
o4-mini OpenAI
👁✓ 81.6MMMU
🛠✓ —
🖥? —
— 81.4O 68.1O — — — — 81.6S $1.1 / $4.4 200K —
o3 OpenAI
👁✓ 76.4MMMU-Pro
🛠✓ 58.2τ²-Bench
🖥? —
— 83.3O 69.1O — 58.2S — 76.4S 82.9S — — —
GPT-4.1 OpenAI
👁✓ 74.8MMMU
🛠✓ —
🖥? —
— 66.3O 54.6O — — — — 74.8S $2 / $8 1M —
Gemma 3 27B Google
👁? —
🛠✓ —
🖥? —
— 42.4O — — — — — — $0.08 / $0.16 131K Yes
Claude 3.7 Sonnet Anthropic
👁✓ 75.0MMMU
🛠✓ —
🖥✓ —
— 84.8O 63.7O — — — — 75.0S $3 / $15 200K —
Grok-3 xAI
👁✓ 78.0MMMU
🛠✓ —
🖥? —
— 84.6O — — — — — 78.0S $3 / $15 128K —
GPT-4.5 OpenAI
👁✓ 75.2MMMU
🛠✓ —
🖥? —
— 71.4O 38.0O — — — — 75.2S — — —
Mistral Small 3 Mistral AI
👁✕ —
🛠✓ —
🖥✕ —
— — — — — — — — $0.05 / $0.08 33K Yes
DeepSeek-R1 DeepSeek
👁✕ —
🛠✕ —
🖥✕ —
— 71.5O 49.2O — — — — — — — Yes
o3-mini OpenAI
👁✕ —
🛠✓ —
🖥✕ —
— 79.7O 49.3O — — — — — $1.1 / $4.4 200K —
DeepSeek-V3 DeepSeek
👁✕ —
🛠✓ —
🖥✕ —
— 59.1O 42.0O — — — — — $0.32 / $0.89 164K Yes
Llama 3.3 70B Meta
👁✕ —
🛠✓ —
🖥✕ —
— 50.5O — — — — — — $0.1 / $0.32 131K Yes
Gemini 2.0 Flash Google
👁✓ 70.7MMMU
🛠✓ —
🖥? —
— 65.2O 21.4O — — — — 70.7S — — —
o1 OpenAI
👁✓ 77.6MMMU
🛠✓ —
🖥? —
— 78.0O 48.9O — — — — 77.6S — — —
Claude 3.5 Haiku Anthropic
👁✓ —
🛠✓ —
🖥? —
— — 40.6O — — — — — $0.8 / $4 200K —
Claude 3.5 Sonnet (new) Anthropic
👁✓ —
🛠✓ —
🖥✓ 14.9OSWorld*
— — 49.0O — — 14.9*O — — $3 / $15 200K —
Llama 3.1 Nemotron 70B NVIDIA
👁✕ —
🛠✓ —
🖥✕ —
— — — — — — — — — — Yes
Qwen2.5 72B Alibaba (Qwen)
👁✕ —
🛠✓ —
🖥✕ —
— 49.0O — — — — — — $0.36 / $0.4 33K Yes
Grok-2 xAI
👁✓ 66.1MMMU
🛠✓ —
🖥✕ —
— 56.0O — — — — — 66.1S — — —
Mistral Large 2 Mistral AI
👁✕ —
🛠✓ —
🖥✕ —
— — — — — — — — — — —
Llama 3.1 405B Meta
👁✕ —
🛠✓ —
🖥✕ —
— 50.7O — — — — — — — — Yes
GPT-4o mini OpenAI
👁✓ 59.4MMMU
🛠✓ —
🖥✕ —
— 40.2O — — — — — 59.4S $0.15 / $0.6 128K —
Claude 3.5 Sonnet Anthropic
👁✓ 68.3MMMU
🛠✓ —
🖥✕ —
— 59.4O 33.4O — — — — 68.3S $3 / $15 200K —
Codestral 22B Mistral AI
👁✕ —
🛠? —
🖥✕ —
— — — — — — — — — — —
DeepSeek-V2.5 DeepSeek
👁✕ —
🛠? —
🖥✕ —
— — — — — — — — — — Yes
Gemini 1.5 Flash Google
👁✓ 62.3MMMU
🛠? —
🖥✕ —
— — — — — — — 62.3S — — —
Gemini 1.5 Pro Google
👁✓ 65.9MMMU
🛠? —
🖥✕ —
— — — — — — — 65.9S — — —
GPT-4o OpenAI
👁✓ 72.2MMMU
🛠✓ 23.5τ²-Bench
🖥✕ —
— 53.6O 33.2O — 23.5S — — 72.2S $2.5 / $10 128K —
GPT-4 Turbo OpenAI
👁? —
🛠? —
🖥✕ —
— — — — — — — — $10 / $30 128K —
Claude 3 Haiku Anthropic
👁✓ —
🛠? —
🖥✕ —
— 33.3O — — — — — — $0.25 / $1.25 200K —
Claude 3 Sonnet Anthropic
👁✓ —
🛠? —
🖥✕ —
— 40.4O — — — — — — $3 / $15 200K —
Claude 3 Opus Anthropic
👁✓ —
🛠? —
🖥✕ —
— 50.4O — — — — — — $15 / $75 200K —
Grok-1.5 xAI
👁✕ —
🛠✕ —
🖥✕ —
— — — — — — — — — — —
Gemini 1.0 Pro Google
👁✓ —
🛠? —
🖥✕ —
— — — — — — — — — — —
Claude 2.1 Anthropic
👁✕ —
🛠? —
🖥✕ —
— — — — — — — — — 200K —
Claude 2 Anthropic
👁✕ —
🛠✕ —
🖥✕ —
— — — — — — — — — 100K —
Claude Anthropic
👁✕ —
🛠✕ —
🖥✕ —
— — — — — — — — — — —
GPT-4 OpenAI
👁✕ —
🛠? —
🖥✕ —
— — — — — — — — — 8K —
GPT-3.5 Turbo OpenAI
👁✕ —
🛠? —
🖥✕ —
— — — — — — — — $0.5 / $1.5 16K —

✓ yes · ✕ no · ? no clear info
* Original OSWorld (pre-Verified version) · O = official company announcement · S = llm-stats.com · D = independent compilation · — = No published score

What does each benchmark measure?

Humanity's Last Exam
The hardest expert-written academic questions
GPQA Diamond
PhD-level science questions
SWE-bench Verified
Fixing real GitHub issues (software)
Terminal-Bench 2.x
Agentic tasks in the terminal
τ²-Bench (Telecom)
Acting as an agent with tools to solve a customer issue end to end
OSWorld-Verified
Using a computer (screen, mouse, keyboard)
MMMU-Pro
Vision: multi-discipline questions with charts, diagrams and images (hard version)
MMMU
Vision: college-level questions with charts, diagrams and images

Method & sources

Dates, prices and context come from company docs and llm-stats.com organisation pages; for some models only the release month is known. Scores are either published in the company's official announcement (O), taken from llm-stats.com leaderboards (S), or compiled from independent write-ups of the company's launch tables (D). For the newest models companies report newer benchmark versions (e.g. OSWorld 2.0, Terminal-Bench 4.0); these appear in the capability column with their version name and should not be compared directly with older versions. Sources may use different settings, so treat scores as directional. No unpublished value was estimated. The table lists each company's notable models, not every sub-variant.