Noometry

Model comparison

Claude Instant vs Mistral Medium

Mistral Medium has enough public results to be ranked (#218); Claude Instant does not yet, so treat this comparison as directional.

Last verified . 1 shared benchmarks.

Claude Instant Anthropic

29.5

Unranked Sparse

Mistral Medium Mistral AI

36.3

Rank #218 Confirmed

Summary

  • They share 1 benchmark with published results for both. Claude Instant scores higher in 0 categories and Mistral Medium in 1 category; one gap is clear of the uncertainty.
  • The widest gap is in reasoning, where Mistral Medium leads 24.0 to 19.6.
  • The biggest single-benchmark swing is DTBench: 45.8% for Claude Instant and 75.5% for Mistral Medium.
  • Mistral Medium has downloadable open weights; the other is API-only.

Side by side

Claude InstantMistral Medium
ProviderAnthropicMistral AI
Noometry Index29.536.3
Released2023-08-092023-12-11
WeightsProprietaryOpen
Context window—262K
Max output—262K
Input $ / M tokens—$1.50
Output $ / M tokens—$7.50
Results tracked736

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Claude Instant: —, Mistral Medium: 34.2 (#243)

BenchmarkClaude InstantMistral Medium
FrontierCode—8%
SciCode—40.2%
WeirdML—43.7%
LMArena Coding—1434
ALE-Bench—763.98
HumanEval+50.6%—

Agentic & Tool Use Not comparable

Claude Instant: —, Mistral Medium: 28.3 (#90)

BenchmarkClaude InstantMistral Medium
Berkeley Function Calling Leaderboard—37.7%

Reasoning Mistral Medium leads

Claude Instant: 19.6, Mistral Medium: 24.0 (#167)

BenchmarkClaude InstantMistral Medium
DTBench45.8%75.5%
Kagi LLM Benchmark—50%
CritPt—0%
LMArena Hard Prompts—1426
LMCA—26.1%
Surface Evolver Bench—26.9%
Epoch Capabilities Index120.28—

Math Not comparable

Claude Instant: —, Mistral Medium: 28.1 (#245)

BenchmarkClaude InstantMistral Medium
OTIS Mock AIME 2024-2025—32.2%
ProofBench—9%
LMArena Math—1408
MATH Level 5—81.6%
FrontierMath (Feb 2025 set)—0.3%
GSM8K86.7%—

Knowledge Not comparable

Claude Instant: —, Mistral Medium: 25.0 (#265)

BenchmarkClaude InstantMistral Medium
GPQA Diamond—59.5%
Humanity's Last Exam—4.5%
Vectara Hallucination Rate—22.7%
LMArena Expert—1408
ARC (AI2) Challenge86.3%—
MMLU73.4%—
TriviaQA78.9%—

Multimodal Not comparable

Claude Instant: —, Mistral Medium: 35.3 (#88)

BenchmarkClaude InstantMistral Medium
LMArena Vision—1172

Multilingual Not comparable

Claude Instant: —, Mistral Medium: 52.1 (#91)

BenchmarkClaude InstantMistral Medium
LMArena Non-English—1408
LMArena Chinese—1447
LMArena French—1459
LMArena German—1432
LMArena Japanese—1378
LMArena Korean—1380
LMArena Russian—1411
LMArena Spanish—1433

Instruction Following Not comparable

Claude Instant: —, Mistral Medium: 73.7 (#116)

BenchmarkClaude InstantMistral Medium
LMArena Instruction Following—1398

Long Context Not comparable

Claude Instant: —, Mistral Medium: 42.9 (#114)

BenchmarkClaude InstantMistral Medium
LMArena Longer Query—1406

Writing & Preference Not comparable

Claude Instant: —, Mistral Medium: 60.0 (#103)

BenchmarkClaude InstantMistral Medium
LMArena Text—1424
LMArena Creative Writing—1391
Short-Story Creative Writing—77.3%
LMArena Multi-Turn—1418

Frequently asked questions

Is Claude Instant better than Mistral Medium?

Mistral Medium has enough public results to be ranked (#218); Claude Instant does not yet, so treat this comparison as directional.

How many benchmarks do Claude Instant and Mistral Medium share?

1 benchmark has published results for both models. Claude Instant has 7 scored results on Noometry and Mistral Medium has 36.

Related comparisons

Go deeper