Noometry

Model comparison

Grok-2 (Dec 2024) vs Ministral 8B

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.7 to 28.2 on the Noometry Index.

Last verified . 15 shared benchmarks.

Grok-2 (Dec 2024) xAI

33.7

Rank #239 Confirmed

Ministral 8B Mistral AI

28.2

Rank #325 Confirmed

Summary

  • They share 15 benchmarks with published results for both. Grok-2 (Dec 2024) scores higher in 5 categories and Ministral 8B in 3 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok-2 (Dec 2024) leads 29.8 to 12.6.
  • The biggest single-benchmark swing is MATH Level 5: 63.5% for Grok-2 (Dec 2024) and 14.9% for Ministral 8B.
  • Ministral 8B has downloadable open weights; the other is API-only.

Side by side

Grok-2 (Dec 2024)Ministral 8B
ProviderxAIMistral AI
Noometry Index33.728.2
Released2024-08-132024-10-01
WeightsProprietaryOpen
Context window—262K
Max output—262K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.15
Results tracked3417

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Ministral 8B leads

Grok-2 (Dec 2024): 33.3 (#258), Ministral 8B: 35.0 (#230)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Coding12871202
WeirdML22.2%—
LiveBench Coding46.4%—

Agentic & Tool Use Not comparable

Grok-2 (Dec 2024): —, Ministral 8B: 16.4 (#148)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
Berkeley Function Calling Leaderboard—11.1%

Reasoning Ministral 8B leads

Grok-2 (Dec 2024): 16.9 (#299), Ministral 8B: 18.4 (#281)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Hard Prompts12721191
DTBench65.2%45.7%
SimpleBench22.7%—
LiveBench Reasoning54.8%—
LiveBench Data Analysis54.5%—
Epoch Capabilities Index130.48—
LiveBench54.3%—

Math Ministral 8B leads

Grok-2 (Dec 2024): 20.8 (#284), Ministral 8B: 25.7 (#267)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Math12831188
MATH Level 563.5%14.9%
OTIS Mock AIME 2024-202511.5%—
LiveBench Math54.9%—
FrontierMath (Feb 2025 set)0.7%—

Knowledge Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 29.8 (#233), Ministral 8B: 12.6 (#297)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
GPQA Diamond53.8%27.1%
LMArena Expert12541170
Confabulations20.1%—
Vectara Hallucination Rate—7.4%

Multilingual Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 43.1 (#188), Ministral 8B: 35.1 (#247)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Non-English12821165
LMArena Chinese12891193
LMArena Russian12861195
LMArena French1318—
LMArena German1287—
LMArena Japanese1244—
LMArena Korean1237—
LMArena Spanish1281—

Instruction Following Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 66.9 (#202), Ministral 8B: 60.5 (#250)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Instruction Following12701161
LiveBench Instruction Following69.6%—

Long Context Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 38.8 (#190), Ministral 8B: 36.7 (#227)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Longer Query12761212

Writing & Preference Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 48.6 (#198), Ministral 8B: 39.6 (#246)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Text13051191
LMArena Creative Writing12841175
LMArena Multi-Turn12901166
Short-Story Creative Writing63.6%—
LiveBench Language45.6%—

Frequently asked questions

Is Grok-2 (Dec 2024) better than Ministral 8B?

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.7 to 28.2 on the Noometry Index.

Is Grok-2 (Dec 2024) or Ministral 8B better for coding?

Ministral 8B scores higher on coding benchmarks: 35.0 versus 33.3 in the Noometry coding category.

How many benchmarks do Grok-2 (Dec 2024) and Ministral 8B share?

15 benchmarks have published results for both models. Grok-2 (Dec 2024) has 34 scored results on Noometry and Ministral 8B has 17.

Related comparisons

Go deeper