Noometry

Model comparison

Mistral Medium vs Tulu 3 (Tülu 3) 70B

Mistral Medium is the stronger model overall, scoring 36.0 to 32.6 on the Noometry Index.

Last verified . 14 shared benchmarks.

Mistral Medium Mistral AI

36.0

Rank #211 Confirmed

Tulu 3 (Tülu 3) 70B Allen Institute for AI (Ai2)

32.6

Rank #242 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Mistral Medium scores higher in 6 categories and Tulu 3 (Tülu 3) 70B in 2 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Medium leads 60.2 to 45.8.
  • The biggest single-benchmark swing is MATH Level 5: 81.6% for Mistral Medium and 42.7% for Tulu 3 (Tülu 3) 70B.

Side by side

Mistral MediumTulu 3 (Tülu 3) 70B
ProviderMistral AIAllen Institute for AI (Ai2)
Noometry Index36.032.6
Released2023-12-112024-11-21
WeightsOpenOpen
Context window262K—
Max output262K—
Input $ / M tokens$1.50—
Output $ / M tokens$7.50—
Results tracked3614

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Tulu 3 (Tülu 3) 70B leads

Mistral Medium: 33.8 (#241), Tulu 3 (Tülu 3) 70B: 35.4 (#211)

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
LMArena Coding14341236
FrontierCode8%—
SciCode40.2%—
WeirdML43.7%—
ALE-Bench763.98—

Agentic & Tool Use Not comparable

Mistral Medium: 28.1 (#84), Tulu 3 (Tülu 3) 70B: —

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
Berkeley Function Calling Leaderboard37.7%—

Reasoning Too close to call

Mistral Medium: 23.3 (#165), Tulu 3 (Tülu 3) 70B: 22.9 (#172)

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
LMArena Hard Prompts14261220
Kagi LLM Benchmark50%—
CritPt0%—
DTBench75.5%—
LMCA26.1%—
Surface Evolver Bench26.9%—

Math Mistral Medium leads

Mistral Medium: 27.2 (#238), Tulu 3 (Tülu 3) 70B: 13.3 (#297)

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
OTIS Mock AIME 2024-202532.2%4.4%
LMArena Math14091242
MATH Level 581.6%42.7%
ProofBench9%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge Too close to call

Mistral Medium: 24.9 (#260), Tulu 3 (Tülu 3) 70B: 25.1 (#257)

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
GPQA Diamond59.5%46.3%
Humanity's Last Exam4.5%—
Vectara Hallucination Rate22.7%—
LMArena Expert1408—

Multimodal Not comparable

Mistral Medium: 34.6 (#82), Tulu 3 (Tülu 3) 70B: —

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
LMArena Vision1172—

Multilingual Mistral Medium leads

Mistral Medium: 52.3 (#86), Tulu 3 (Tülu 3) 70B: 40.1 (#217)

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
LMArena Non-English14081236
LMArena Chinese14481249
LMArena Russian14111246
LMArena French1460—
LMArena German1433—
LMArena Japanese1378—
LMArena Korean1382—
LMArena Spanish1433—

Instruction Following Mistral Medium leads

Mistral Medium: 73.9 (#111), Tulu 3 (Tülu 3) 70B: 65.0 (#222)

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
LMArena Instruction Following13981233

Long Context Mistral Medium leads

Mistral Medium: 42.9 (#108), Tulu 3 (Tülu 3) 70B: 37.1 (#217)

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
LMArena Longer Query14061224

Writing & Preference Mistral Medium leads

Mistral Medium: 60.2 (#97), Tulu 3 (Tülu 3) 70B: 45.8 (#218)

BenchmarkMistral MediumTulu 3 (Tülu 3) 70B
LMArena Text14241256
LMArena Creative Writing13911231
LMArena Multi-Turn14181252
Short-Story Creative Writing77.3%—

Frequently asked questions

Is Mistral Medium better than Tulu 3 (Tülu 3) 70B?

Mistral Medium is the stronger model overall, scoring 36.0 to 32.6 on the Noometry Index.

Is Mistral Medium or Tulu 3 (Tülu 3) 70B better for coding?

Tulu 3 (Tülu 3) 70B scores higher on coding benchmarks: 35.4 versus 33.8 in the Noometry coding category.

How many benchmarks do Mistral Medium and Tulu 3 (Tülu 3) 70B share?

14 benchmarks have published results for both models. Mistral Medium has 36 scored results on Noometry and Tulu 3 (Tülu 3) 70B has 14.

Related comparisons

Go deeper