Noometry

Model comparison

Claude Sonnet 4 vs GPT-4o mini

Claude Sonnet 4 is the stronger model overall, scoring 40.8 to 25.5 on the Noometry Index. GPT-4o mini costs 23× less per token, which makes it the better buy when Claude Sonnet 4's lead doesn't matter for your workload.

Last verified . 39 shared benchmarks.

Claude Sonnet 4 Anthropic

40.8

Rank #145 Confirmed

GPT-4o mini OpenAI

25.5

Rank #343 Confirmed

Summary

  • They share 39 benchmarks with published results for both. Claude Sonnet 4 scores higher in 9 categories and GPT-4o mini in 1 category; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude Sonnet 4 leads 43.3 to 10.4.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 71.1% for Claude Sonnet 4 and 6.9% for GPT-4o mini.
  • GPT-4o mini is cheaper at $0.15 / $0.60 per million input/output tokens, against $3 / $15 for Claude Sonnet 4.
  • Claude Sonnet 4 accepts more context: 200K tokens versus 128K.

Side by side

Claude Sonnet 4GPT-4o mini
ProviderAnthropicOpenAI
Noometry Index40.825.5
Released2025-05-222024-07-18
WeightsProprietaryProprietary
Context window200K128K
Max output64K16K
Input $ / M tokens$3$0.15
Output $ / M tokens$15$0.60
Results tracked5860

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Sonnet 4 leads

Claude Sonnet 4: 43.5 (#88), GPT-4o mini: 22.0 (#335)

BenchmarkClaude Sonnet 4GPT-4o mini
Aider Polyglot61.3%3.6%
WeirdML46.1%11.8%
LMArena Coding14141290
SWE-bench Verified (bash only)64.9%—
SciCode40%—
GSO4.9%—
BigCodeBench Instruct—46.1%
LiveBench Coding—43.1%
BigCodeBench Complete—57.4%
ALE-Bench655.35—
HumanEval+—83.5%
MBPP+—72.2%

Agentic & Tool Use Claude Sonnet 4 leads

Claude Sonnet 4: 38.5 (#31), GPT-4o mini: 27.5 (#101)

BenchmarkClaude Sonnet 4GPT-4o mini
TheAgentCompany33.1%—
Cybench35%—
DeepResearch Bench46.6%—
OSWorld43.9%—
BALROG—17.4%
METR Time Horizons62%—

Reasoning Claude Sonnet 4 leads

Claude Sonnet 4: 22.9 (#187), GPT-4o mini: 8.7 (#347)

BenchmarkClaude Sonnet 4GPT-4o mini
ARC-AGI-25.9%0%
SimpleBench45.5%10.7%
Kagi LLM Benchmark73%28.8%
LMArena Hard Prompts13721267
DTBench77.1%54.4%
LMCA29%10.4%
Epoch Capabilities Index141.69126.56
ARC-AGI-140%—
CritPt0.3%—
Chess Puzzles—0%
EnigmaEval3.1%—
LiveBench Reasoning—32.8%
Mystery Game Puzzles—12%
LiveBench Data Analysis—50%
ForecastBench60.2—
LiveBench—41.3%
PIQA—88.7%

Math Claude Sonnet 4 leads

Claude Sonnet 4: 43.3 (#80), GPT-4o mini: 10.4 (#314)

BenchmarkClaude Sonnet 4GPT-4o mini
OTIS Mock AIME 2024-202571.1%6.9%
Omni-MATH60.2%28%
LMArena Math13751267
MATH Level 584.4%52.6%
FrontierMath (Tiers 1-3)—0.7%
LiveBench Math—36.3%
FrontierMath (Feb 2025 set)4.1%—
FrontierMath Tier 4 (v1)0%—
GSM8K—91.3%

Knowledge Claude Sonnet 4 leads

Claude Sonnet 4: 41.8 (#108), GPT-4o mini: 17.7 (#284)

BenchmarkClaude Sonnet 4GPT-4o mini
GPQA Diamond79.2%37.7%
MMLU-Pro84.3%60.3%
Confabulations13.2%37.2%
GPQA (HELM)70.6%36.8%
LMArena Expert13721235
Humanity's Last Exam7.8%—
SimpleQA Verified—8.3%
Vectara Hallucination Rate10.3%—
BoolQ—88.7%
MMLU—81.8%

Multimodal Too close to call

Claude Sonnet 4: 26.2 (#121), GPT-4o mini: 25.9 (#122)

BenchmarkClaude Sonnet 4GPT-4o mini
LMArena Vision11911066
GeoBench37%64%
VPCT34%34%
Video-MME—64.8%
MindCube44.8%—

Multilingual Claude Sonnet 4 leads

Claude Sonnet 4: 46.7 (#156), GPT-4o mini: 42.0 (#199)

BenchmarkClaude Sonnet 4GPT-4o mini
LMArena Non-English13331266
LMArena Chinese13501265
LMArena French13631297
LMArena German13311272
LMArena Japanese13021216
LMArena Korean12911195
LMArena Russian13551275
LMArena Spanish13571276

Instruction Following Claude Sonnet 4 leads

Claude Sonnet 4: 71.7 (#145), GPT-4o mini: 61.9 (#239)

BenchmarkClaude Sonnet 4GPT-4o mini
IFEval84%78.2%
LMArena Instruction Following13761258
LiveBench Instruction Following—56.8%

Long Context GPT-4o mini leads

Claude Sonnet 4: 33.7 (#259), GPT-4o mini: 39.1 (#186)

BenchmarkClaude Sonnet 4GPT-4o mini
LMArena Longer Query13981289
Fiction.LiveBench46.9%—

Writing & Preference Claude Sonnet 4 leads

Claude Sonnet 4: 57.1 (#132), GPT-4o mini: 39.5 (#248)

BenchmarkClaude Sonnet 4GPT-4o mini
LMArena Text13511286
LMArena Creative Writing13451268
Short-Story Creative Writing81.4%67.2%
EQ-Bench Creative Writing1483873
WildBench83.8%79.1%
LMArena Multi-Turn13761285
LiveBench Language—28.6%

Frequently asked questions

Is Claude Sonnet 4 better than GPT-4o mini?

Claude Sonnet 4 is the stronger model overall, scoring 40.8 to 25.5 on the Noometry Index. GPT-4o mini costs 23× less per token, which makes it the better buy when Claude Sonnet 4's lead doesn't matter for your workload.

Which is cheaper, Claude Sonnet 4 or GPT-4o mini?

GPT-4o mini is cheaper. It lists at $0.15 per million input tokens and $0.60 per million output tokens; Claude Sonnet 4 lists at $3 and $15.

Is Claude Sonnet 4 or GPT-4o mini better for coding?

Claude Sonnet 4 scores higher on coding benchmarks: 43.5 versus 22.0 in the Noometry coding category.

Which has the bigger context window?

Claude Sonnet 4 does, with 200K tokens against 128K.

How many benchmarks do Claude Sonnet 4 and GPT-4o mini share?

39 benchmarks have published results for both models. Claude Sonnet 4 has 58 scored results on Noometry and GPT-4o mini has 60.

Related comparisons

Go deeper