Noometry

Model comparison

Gemma 3 12B vs Ministral 8B

Gemma 3 12B is the stronger model overall, scoring 31.8 to 27.8 on the Noometry Index.

Last verified . 15 shared benchmarks.

Gemma 3 12B Google

31.8

Rank #255 Confirmed

Ministral 8B Mistral AI

27.8

Rank #321 Confirmed

Summary

  • They share 15 benchmarks with published results for both. Gemma 3 12B scores higher in 6 categories and Ministral 8B in 3 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Gemma 3 12B leads 26.5 to 12.5.
  • The biggest single-benchmark swing is Berkeley Function Calling Leaderboard: 30.4% for Gemma 3 12B and 11.1% for Ministral 8B.
  • Gemma 3 12B is cheaper at $0.05 / $0.15 per million input/output tokens, against $0.10 / $0.10 for Ministral 8B.
  • Gemma 3 12B accepts more context: 131K tokens versus 128K.

Side by side

Gemma 3 12BMinistral 8B
ProviderGoogleMistral AI
Noometry Index31.827.8
Released2025-03-122024-10-01
WeightsOpenOpen
Context window131K128K
Max output8K128K
Input $ / M tokens$0.05$0.10
Output $ / M tokens$0.15$0.10
Results tracked2417

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Ministral 8B leads

Gemma 3 12B: 31.2 (#276), Ministral 8B: 34.4 (#226)

BenchmarkGemma 3 12BMinistral 8B
LMArena Coding12811202
SciCode17.4%—

Agentic & Tool Use Gemma 3 12B leads

Gemma 3 12B: 25.3 (#101), Ministral 8B: 16.2 (#140)

BenchmarkGemma 3 12BMinistral 8B
Berkeley Function Calling Leaderboard30.4%11.1%

Reasoning Ministral 8B leads

Gemma 3 12B: 15.2 (#307), Ministral 8B: 17.7 (#279)

BenchmarkGemma 3 12BMinistral 8B
LMArena Hard Prompts13091191
DTBench48.8%45.7%
CritPt0%—
Chess Puzzles0%—
LMCA4.5%—
Epoch Capabilities Index123.49—

Math Ministral 8B leads

Gemma 3 12B: 21.0 (#273), Ministral 8B: 23.7 (#262)

BenchmarkGemma 3 12BMinistral 8B
LMArena Math13071188
OTIS Mock AIME 2024-202516.7%—
MATH Level 5—14.9%

Knowledge Gemma 3 12B leads

Gemma 3 12B: 26.5 (#250), Ministral 8B: 12.5 (#292)

BenchmarkGemma 3 12BMinistral 8B
GPQA Diamond39.5%27.1%
Vectara Hallucination Rate4.4%7.4%
LMArena Expert12481170

Multimodal Not comparable

Gemma 3 12B: —, Ministral 8B: —

BenchmarkGemma 3 12BMinistral 8B
MindCube46.7%—

Multilingual Gemma 3 12B leads

Gemma 3 12B: 45.9 (#160), Ministral 8B: 35.3 (#242)

BenchmarkGemma 3 12BMinistral 8B
LMArena Non-English13181165
LMArena Russian13351195
LMArena Chinese—1193
LMArena German1370—

Instruction Following Gemma 3 12B leads

Gemma 3 12B: 68.8 (#179), Ministral 8B: 60.7 (#245)

BenchmarkGemma 3 12BMinistral 8B
LMArena Instruction Following12991161

Long Context Gemma 3 12B leads

Gemma 3 12B: 40.0 (#157), Ministral 8B: 36.7 (#222)

BenchmarkGemma 3 12BMinistral 8B
LMArena Longer Query13171212

Writing & Preference Gemma 3 12B leads

Gemma 3 12B: 47.7 (#205), Ministral 8B: 39.8 (#241)

BenchmarkGemma 3 12BMinistral 8B
LMArena Text13341191
LMArena Creative Writing13311175
LMArena Multi-Turn13341166
EQ-Bench Creative Writing1126—

Frequently asked questions

Is Gemma 3 12B better than Ministral 8B?

Gemma 3 12B is the stronger model overall, scoring 31.8 to 27.8 on the Noometry Index.

Which is cheaper, Gemma 3 12B or Ministral 8B?

Gemma 3 12B is cheaper. It lists at $0.05 per million input tokens and $0.15 per million output tokens; Ministral 8B lists at $0.10 and $0.10.

Is Gemma 3 12B or Ministral 8B better for coding?

Ministral 8B scores higher on coding benchmarks: 34.4 versus 31.2 in the Noometry coding category.

Which has the bigger context window?

Gemma 3 12B does, with 131K tokens against 128K.

How many benchmarks do Gemma 3 12B and Ministral 8B share?

15 benchmarks have published results for both models. Gemma 3 12B has 24 scored results on Noometry and Ministral 8B has 17.

Related comparisons

Go deeper