Model comparison
Claude 3.5 Sonnet vs GPT-4.1 mini
Claude 3.5 Sonnet and GPT-4.1 mini score almost the same on the Noometry Index (34.6 vs 33.6), so choose on price, context window or the category you care about most.
Last verified . 34 shared benchmarks.
Summary
- They share 34 benchmarks with published results for both. Claude 3.5 Sonnet scores higher in 4 categories and GPT-4.1 mini in 6 categories; 9 gaps are clear of the uncertainty.
- The widest gap is in reasoning, where Claude 3.5 Sonnet leads 23.1 to 10.8.
- The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 8.5% for Claude 3.5 Sonnet and 44.7% for GPT-4.1 mini.
Side by side
| Claude 3.5 Sonnet | GPT-4.1 mini | |
|---|---|---|
| Provider | Anthropic | OpenAI |
| Noometry Index | 34.6 | 33.6 |
| Released | 2024-06-20 | 2025-04-14 |
| Weights | Proprietary | Proprietary |
| Context window | — | 1.05M |
| Max output | — | 33K |
| Input $ / M tokens | — | $0.40 |
| Output $ / M tokens | — | $1.60 |
| Results tracked | 60 | 47 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding Claude 3.5 Sonnet leads
Claude 3.5 Sonnet: 39.0 (#165), GPT-4.1 mini: 30.6 (#293)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| Aider Polyglot | 51.6% | 32.4% |
| WeirdML | 40% | 37.6% |
| BigCodeBench Instruct | 46.8% | 48.9% |
| LMArena Coding | 1342 | 1367 |
| CadEval | 48% | 16% |
| SWE-bench Verified (bash only) | — | 23.9% |
| SciCode | — | 40.4% |
| GSO | 4.6% | — |
| LiveBench Coding | 67.1% | — |
| BigCodeBench Complete | 58.6% | — |
| HumanEval+ | 81.7% | — |
| MBPP+ | 74.3% | — |
Agentic & Tool Use Too close to call
Claude 3.5 Sonnet: 32.3 (#67), GPT-4.1 mini: 33.3 (#55)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| Berkeley Function Calling Leaderboard | — | 50.5% |
| TheAgentCompany | 24% | — |
| Cybench | 17.5% | — |
| BALROG | 32.6% | — |
| METR Time Horizons | 45.2% | — |
Reasoning Claude 3.5 Sonnet leads
Claude 3.5 Sonnet: 23.1 (#183), GPT-4.1 mini: 10.8 (#340)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| LMArena Hard Prompts | 1305 | 1349 |
| DTBench | 67.8% | 68.8% |
| Epoch Capabilities Index | 133.55 | 135.01 |
| ARC-AGI-2 | — | 0% |
| SimpleBench | 41.4% | — |
| Kagi LLM Benchmark | — | 48.6% |
| ARC-AGI-1 | — | 3.5% |
| CritPt | — | 0% |
| Chess Puzzles | — | 7% |
| EnigmaEval | 0.9% | — |
| LiveBench Reasoning | 56.7% | — |
| Mystery Game Puzzles | — | 7% |
| LiveBench Data Analysis | 55% | — |
| LMCA | — | 21.1% |
| ForecastBench | 60.7 | — |
| LiveBench | 59% | — |
Math GPT-4.1 mini leads
Claude 3.5 Sonnet: 19.2 (#288), GPT-4.1 mini: 24.1 (#270)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| OTIS Mock AIME 2024-2025 | 8.5% | 44.7% |
| Omni-MATH | 27.6% | 49.1% |
| LMArena Math | 1307 | 1343 |
| MATH Level 5 | 56.9% | 87.3% |
| FrontierMath (Feb 2025 set) | 2.1% | 4.5% |
| FrontierMath (Tiers 1-3) | — | 6.7% |
| LiveBench Math | 52.3% | — |
| FrontierMath Tier 4 (v1) | 0% | — |
Knowledge GPT-4.1 mini leads
Claude 3.5 Sonnet: 28.6 (#245), GPT-4.1 mini: 34.7 (#194)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| GPQA Diamond | 55.3% | 65.8% |
| MMLU-Pro | 77.7% | 78.3% |
| GPQA (HELM) | 56.5% | 61.4% |
| LMArena Expert | 1265 | 1338 |
| Humanity's Last Exam | 4.1% | — |
| SimpleQA Verified | — | 12.7% |
| Confabulations | 19.9% | — |
| MMLU | 87.3% | — |
Multimodal GPT-4.1 mini leads
Claude 3.5 Sonnet: 26.5 (#120), GPT-4.1 mini: 35.8 (#82)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| LMArena Vision | 1125 | 1181 |
| Video-MME | 60% | — |
| GeoBench | 62% | — |
| VPCT | 33% | — |
Multilingual GPT-4.1 mini leads
Claude 3.5 Sonnet: 43.2 (#185), GPT-4.1 mini: 45.7 (#166)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| LMArena Non-English | 1283 | 1318 |
| LMArena Chinese | 1272 | 1329 |
| LMArena French | 1305 | 1358 |
| LMArena German | 1297 | 1351 |
| LMArena Japanese | 1234 | 1290 |
| LMArena Korean | 1200 | 1298 |
| LMArena Russian | 1306 | 1324 |
| LMArena Spanish | 1290 | 1319 |
Instruction Following GPT-4.1 mini leads
Claude 3.5 Sonnet: 68.8 (#182), GPT-4.1 mini: 73.7 (#118)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| IFEval | 85.5% | 90.4% |
| LMArena Instruction Following | 1297 | 1333 |
| LiveBench Instruction Following | 69.3% | — |
Long Context Claude 3.5 Sonnet leads
Claude 3.5 Sonnet: 39.9 (#167), GPT-4.1 mini: 31.8 (#275)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| LMArena Longer Query | 1311 | 1344 |
| Fiction.LiveBench | — | 44.4% |
Writing & Preference Claude 3.5 Sonnet leads
Claude 3.5 Sonnet: 52.9 (#164), GPT-4.1 mini: 48.6 (#199)
| Benchmark | Claude 3.5 Sonnet | GPT-4.1 mini |
|---|---|---|
| LMArena Text | 1298 | 1340 |
| LMArena Creative Writing | 1292 | 1300 |
| EQ-Bench Creative Writing | 1451 | 1147 |
| WildBench | 79.2% | 83.8% |
| LMArena Multi-Turn | 1326 | 1354 |
| Short-Story Creative Writing | 80.3% | — |
| LiveBench Language | 53.8% | — |
Frequently asked questions
Is Claude 3.5 Sonnet better than GPT-4.1 mini?
Claude 3.5 Sonnet and GPT-4.1 mini score almost the same on the Noometry Index (34.6 vs 33.6), so choose on price, context window or the category you care about most.
Is Claude 3.5 Sonnet or GPT-4.1 mini better for coding?
Claude 3.5 Sonnet scores higher on coding benchmarks: 39.0 versus 30.6 in the Noometry coding category.
How many benchmarks do Claude 3.5 Sonnet and GPT-4.1 mini share?
34 benchmarks have published results for both models. Claude 3.5 Sonnet has 60 scored results on Noometry and GPT-4.1 mini has 47.