Noometry

Model comparison

GPT-4 Turbo vs Mistral Medium 3.1

Mistral Medium 3.1 is the stronger model overall, scoring 31.9 to 30.5 on the Noometry Index.

Last verified . 0 shared benchmarks.

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Mistral Medium 3.1 Mistral AI

31.9

Rank #266 Reported

Summary

  • The widest gap is in writing & preference, where Mistral Medium 3.1 leads 55.5 to 47.7.
  • Mistral Medium 3.1 is cheaper at $0.40 / $2 per million input/output tokens, against $10 / $30 for GPT-4 Turbo.
  • Mistral Medium 3.1 accepts more context: 131K tokens versus 128K.

Side by side

GPT-4 TurboMistral Medium 3.1
ProviderOpenAIMistral AI
Noometry Index30.531.9
Released2023-11-06—
WeightsProprietaryProprietary
Context window128K131K
Max output4K105K
Input $ / M tokens$10$0.40
Output $ / M tokens$30$2
Results tracked363

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

GPT-4 Turbo: 33.8 (#249), Mistral Medium 3.1: —

BenchmarkGPT-4 TurboMistral Medium 3.1
WeirdML18%—
BigCodeBench Instruct48.2%—
LMArena Coding1268—
BigCodeBench Complete58.2%—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Mistral Medium 3.1: —

BenchmarkGPT-4 TurboMistral Medium 3.1
METR Time Horizons36.7%—

Reasoning GPT-4 Turbo leads

GPT-4 Turbo: 15.3 (#317), Mistral Medium 3.1: 10.6 (#341)

BenchmarkGPT-4 TurboMistral Medium 3.1
SimpleBench25.1%—
NYT Connections (extended)—6.5%
Chess Puzzles6%—
Thematic Generalization—20.3%
LMArena Hard Prompts1251—
DTBench61.6%—
LMCA9.8%—
Epoch Capabilities Index127.25—
ForecastBench59.4—

Math Not comparable

GPT-4 Turbo: 9.0 (#322), Mistral Medium 3.1: —

BenchmarkGPT-4 TurboMistral Medium 3.1
FrontierMath (Tiers 1-3)0.7%—
OTIS Mock AIME 2024-20256.7%—
LMArena Math1272—
MATH Level 546.7%—

Knowledge Not comparable

GPT-4 Turbo: 24.3 (#268), Mistral Medium 3.1: —

BenchmarkGPT-4 TurboMistral Medium 3.1
GPQA Diamond46.6%—
Confabulations28.4%—
LMArena Expert1223—
MMLU81.3%—

Multimodal Not comparable

GPT-4 Turbo: 30.6 (#110), Mistral Medium 3.1: —

BenchmarkGPT-4 TurboMistral Medium 3.1
LMArena Vision1090—

Multilingual Not comparable

GPT-4 Turbo: 40.5 (#216), Mistral Medium 3.1: —

BenchmarkGPT-4 TurboMistral Medium 3.1
LMArena Non-English1245—
LMArena Chinese1242—
LMArena French1276—
LMArena German1259—
LMArena Japanese1194—
LMArena Korean1187—
LMArena Russian1259—
LMArena Spanish1260—

Instruction Following Not comparable

GPT-4 Turbo: 65.8 (#216), Mistral Medium 3.1: —

BenchmarkGPT-4 TurboMistral Medium 3.1
LMArena Instruction Following1249—

Long Context Not comparable

GPT-4 Turbo: 38.0 (#206), Mistral Medium 3.1: —

BenchmarkGPT-4 TurboMistral Medium 3.1
LMArena Longer Query1254—

Writing & Preference Mistral Medium 3.1 leads

GPT-4 Turbo: 47.7 (#206), Mistral Medium 3.1: 55.5 (#145)

BenchmarkGPT-4 TurboMistral Medium 3.1
LMArena Text1272—
LMArena Creative Writing1269—
EQ-Bench Creative Writing—1476
LMArena Multi-Turn1267—

Frequently asked questions

Is GPT-4 Turbo better than Mistral Medium 3.1?

Mistral Medium 3.1 is the stronger model overall, scoring 31.9 to 30.5 on the Noometry Index.

Which is cheaper, GPT-4 Turbo or Mistral Medium 3.1?

Mistral Medium 3.1 is cheaper. It lists at $0.40 per million input tokens and $2 per million output tokens; GPT-4 Turbo lists at $10 and $30.

Which has the bigger context window?

Mistral Medium 3.1 does, with 131K tokens against 128K.

How many benchmarks do GPT-4 Turbo and Mistral Medium 3.1 share?

0 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Mistral Medium 3.1 has 3.

Related comparisons

Go deeper