Noometry

Model comparison

GPT-5 vs Mistral Large

GPT-5 is the stronger model overall, scoring 50.9 to 31.9 on the Noometry Index.

Last verified . 38 shared benchmarks.

GPT-5 OpenAI

50.9

Rank #45 Confirmed

Mistral Large Mistral AI

31.9

Rank #263 Confirmed

Summary

  • They share 38 benchmarks with published results for both. GPT-5 scores higher in 9 categories and Mistral Large in 0 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-5 leads 55.0 to 18.2.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 91.4% for GPT-5 and 8.5% for Mistral Large.
  • Mistral Large is cheaper at $2 / $6 per million input/output tokens, against $1.25 / $10 for GPT-5.
  • GPT-5 accepts more context: 400K tokens versus 131K.
  • Mistral Large has downloadable open weights; the other is API-only.

Side by side

GPT-5Mistral Large
ProviderOpenAIMistral AI
Noometry Index50.931.9
Released2025-08-072024-02-26
WeightsProprietaryOpen
Context window400K131K
Max output128K16K
Input $ / M tokens$1.25$2
Output $ / M tokens$10$6
Results tracked6951

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5 leads

GPT-5: 50.3 (#47), Mistral Large: 34.3 (#240)

BenchmarkGPT-5Mistral Large
SciCode42.9%36.2%
LMArena Coding14361277
ALE-Bench1,162264.7
SWE-bench Verified73.6%—
SWE-bench Verified (bash only)65%—
Aider Polyglot88%—
LMArena WebDev1418—
GSO6.9%—
WeirdML60.7%—
BigCodeBench Instruct—30%
LiveBench Coding—47.1%
BigCodeBench Complete—38.3%
AlgoTune1.67—
HumanEval+—62.2%
MBPP+—59.5%

Agentic & Tool Use GPT-5 leads

GPT-5: 33.1 (#56), Mistral Large: 28.6 (#89)

BenchmarkGPT-5Mistral Large
Terminal-Bench49.6%—
Berkeley Function Calling Leaderboard—38.4%
GDPval34.8%—
Remote Labor Index1.7%—
DeepResearch Bench49.6%—
BALROG32.8%—
LMArena Search1133—
METR Time Horizons69.6%—

Reasoning GPT-5 leads

GPT-5: 38.3 (#64), Mistral Large: 15.8 (#310)

BenchmarkGPT-5Mistral Large
SimpleBench56.7%22.5%
CritPt12.6%0%
LMArena Hard Prompts14161257
DTBench90.7%65.1%
LMCA40%16.7%
Epoch Capabilities Index150128.52
ForecastBench61.457.1
ARC-AGI-29.9%—
Kagi LLM Benchmark72.7%—
ARC-AGI-165.7%—
Chess Puzzles37%—
EnigmaEval10.5%—
EBR-Bench12.7%—
LiveBench Reasoning—43.5%
Mystery Game Puzzles23%—
LiveBench Data Analysis—50.1%
LiveBench—48.4%

Math GPT-5 leads

GPT-5: 55.0 (#44), Mistral Large: 18.2 (#291)

BenchmarkGPT-5Mistral Large
OTIS Mock AIME 2024-202591.4%8.5%
Omni-MATH64.7%28.1%
LMArena Math14071262
MATH Level 598.1%50.3%
FrontierMath (Feb 2025 set)32.4%0.3%
FrontierMath (Tiers 1-3)55.4%—
FrontierMath Tier 422%—
ProofBench18%—
LiveBench Math—42.5%
FrontierMath Tier 4 (v1)12.5%—

Knowledge GPT-5 leads

GPT-5: 56.6 (#43), Mistral Large: 30.1 (#230)

BenchmarkGPT-5Mistral Large
GPQA Diamond86.2%51.3%
MMLU-Pro86.3%59.9%
Confabulations10.3%21.4%
Vectara Hallucination Rate14.7%4.5%
GPQA (HELM)79.2%43.5%
LMArena Expert14191232
Humanity's Last Exam25.3%—
SimpleQA Verified50.1%—
MMLU—80%

Multimodal Not comparable

GPT-5: 46.8 (#13), Mistral Large: —

BenchmarkGPT-5Mistral Large
LMArena Vision1232—
GeoBench81%—
VPCT66%—

Multilingual GPT-5 leads

GPT-5: 51.4 (#110), Mistral Large: 40.0 (#219)

BenchmarkGPT-5Mistral Large
LMArena Non-English13971237
LMArena Chinese14221240
LMArena French14101325
LMArena German14161254
LMArena Japanese14091188
LMArena Korean13601202
LMArena Russian14061257
LMArena Spanish13991268

Instruction Following GPT-5 leads

GPT-5: 73.8 (#113), Mistral Large: 67.9 (#191)

BenchmarkGPT-5Mistral Large
IFEval87.5%87.7%
LMArena Instruction Following13881249
LiveBench Instruction Following—67.9%

Long Context GPT-5 leads

GPT-5: 69.5 (#2), Mistral Large: 38.3 (#199)

BenchmarkGPT-5Mistral Large
LMArena Longer Query13991261
Fiction.LiveBench97.2%—

Writing & Preference GPT-5 leads

GPT-5: 63.4 (#65), Mistral Large: 40.7 (#242)

BenchmarkGPT-5Mistral Large
LMArena Text14061266
LMArena Creative Writing13651243
Short-Story Creative Writing86%69%
EQ-Bench Creative Writing1627985
WildBench85.7%80.1%
LMArena Multi-Turn14261260
LiveBench Language—39.4%

Frequently asked questions

Is GPT-5 better than Mistral Large?

GPT-5 is the stronger model overall, scoring 50.9 to 31.9 on the Noometry Index.

Which is cheaper, GPT-5 or Mistral Large?

Mistral Large is cheaper. It lists at $2 per million input tokens and $6 per million output tokens; GPT-5 lists at $1.25 and $10.

Is GPT-5 or Mistral Large better for coding?

GPT-5 scores higher on coding benchmarks: 50.3 versus 34.3 in the Noometry coding category.

Which has the bigger context window?

GPT-5 does, with 400K tokens against 131K.

How many benchmarks do GPT-5 and Mistral Large share?

38 benchmarks have published results for both models. GPT-5 has 69 scored results on Noometry and Mistral Large has 51.

Related comparisons

Go deeper