Noometry

Model comparison

Claude 3 Opus vs o3-mini

o3-mini is the stronger model overall, scoring 36.7 to 29.5 on the Noometry Index.

Last verified . 37 shared benchmarks.

Claude 3 Opus Anthropic

29.5

Rank #310 Confirmed

o3-mini OpenAI

36.7

Rank #212 Confirmed

Summary

  • They share 37 benchmarks with published results for both. Claude 3 Opus scores higher in 1 category and o3-mini in 8 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where o3-mini leads 38.3 to 24.5.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.7% for Claude 3 Opus and 76.9% for o3-mini.

Side by side

Claude 3 Opuso3-mini
ProviderAnthropicOpenAI
Noometry Index29.536.7
Released2024-02-292024-12-20
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$1.10
Output $ / M tokens—$4.40
Results tracked4651

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3-mini leads

Claude 3 Opus: 32.9 (#267), o3-mini: 40.8 (#132)

BenchmarkClaude 3 Opuso3-mini
WeirdML19.2%43.7%
LiveBench Coding38.6%82.7%
LMArena Coding12641378
Aider Polyglot—60.4%
SciCode—39.8%
GSO—1.3%
BigCodeBench Instruct45.5%—
BigCodeBench Complete57.4%—
CadEval—54%
HumanEval+77.4%—
MBPP+73.3%—

Agentic & Tool Use o3-mini leads

Claude 3 Opus: 24.6 (#116), o3-mini: 29.6 (#84)

BenchmarkClaude 3 Opuso3-mini
Cybench10%22.5%
METR Time Horizons29.5%—

Reasoning o3-mini leads

Claude 3 Opus: 14.6 (#324), o3-mini: 16.3 (#305)

BenchmarkClaude 3 Opuso3-mini
SimpleBench23.5%22.8%
Chess Puzzles5%17%
LiveBench Reasoning40.6%89.6%
LMArena Hard Prompts12451366
DTBench61.6%68.8%
LiveBench Data Analysis57.9%70.6%
LMCA17%19%
Epoch Capabilities Index126.91140.34
ForecastBench58.459.6
LiveBench49.2%75.9%
ARC-AGI-2—3%
ARC-AGI-1—34.5%
CritPt—0.3%
EnigmaEval0.8%—
Mystery Game Puzzles—7%
WinoGrande88.5%—

Math o3-mini leads

Claude 3 Opus: 14.8 (#299), o3-mini: 28.1 (#244)

BenchmarkClaude 3 Opuso3-mini
OTIS Mock AIME 2024-20254.7%76.9%
LiveBench Math43.6%77.3%
LMArena Math12731396
MATH Level 537.5%96.5%
FrontierMath (Tiers 1-3)—18.6%
FrontierMath Tier 4—0%
FrontierMath (Feb 2025 set)—12.4%
FrontierMath Tier 4 (v1)—4.2%

Knowledge o3-mini leads

Claude 3 Opus: 24.5 (#267), o3-mini: 38.3 (#146)

BenchmarkClaude 3 Opuso3-mini
GPQA Diamond47.2%77%
SimpleQA Verified12.6%15.3%
Confabulations22.7%17.9%
LMArena Expert12231364
MMLU84.6%—

Multimodal Not comparable

Claude 3 Opus: 27.1 (#116), o3-mini: —

BenchmarkClaude 3 Opuso3-mini
LMArena Vision1023—

Multilingual o3-mini leads

Claude 3 Opus: 41.4 (#207), o3-mini: 45.7 (#164)

BenchmarkClaude 3 Opuso3-mini
LMArena Non-English12581319
LMArena Chinese12481379
LMArena French12751334
LMArena German12581303
LMArena Japanese12041286
LMArena Korean11871314
LMArena Russian12801304
LMArena Spanish12461321

Instruction Following o3-mini leads

Claude 3 Opus: 64.1 (#228), o3-mini: 75.1 (#72)

BenchmarkClaude 3 Opuso3-mini
LiveBench Instruction Following63.9%84.4%
LMArena Instruction Following12481337

Long Context Claude 3 Opus leads

Claude 3 Opus: 38.2 (#202), o3-mini: 33.8 (#256)

BenchmarkClaude 3 Opuso3-mini
LMArena Longer Query12591343
Fiction.LiveBench—50%

Writing & Preference o3-mini leads

Claude 3 Opus: 47.2 (#213), o3-mini: 50.3 (#182)

BenchmarkClaude 3 Opuso3-mini
LMArena Text12621337
LMArena Creative Writing12351286
LMArena Multi-Turn12751320
LiveBench Language50.4%50.7%
Short-Story Creative Writing—61.7%

Frequently asked questions

Is Claude 3 Opus better than o3-mini?

o3-mini is the stronger model overall, scoring 36.7 to 29.5 on the Noometry Index.

Is Claude 3 Opus or o3-mini better for coding?

o3-mini scores higher on coding benchmarks: 40.8 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3 Opus and o3-mini share?

37 benchmarks have published results for both models. Claude 3 Opus has 46 scored results on Noometry and o3-mini has 51.

Related comparisons

Go deeper