Noometry

Model comparison

Gemma 2 27B vs Magistral Small

Gemma 2 27B and Magistral Small score almost the same on the Noometry Index (29.4 vs 30.2), so choose on price, context window or the category you care about most.

Last verified . 4 shared benchmarks.

Gemma 2 27B Google

29.4

Rank #312 Confirmed

Magistral Small Mistral AI

30.2

Rank #296 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Gemma 2 27B scores higher in 1 category and Magistral Small in 3 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in math, where Magistral Small leads 26.2 to 10.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 1.4% for Gemma 2 27B and 30% for Magistral Small.
  • Gemma 2 27B is cheaper at $0.65 / $0.65 per million input/output tokens, against $0.50 / $1.50 for Magistral Small.
  • Magistral Small accepts more context: 128K tokens versus 8K.

Side by side

Gemma 2 27BMagistral Small
ProviderGoogleMistral AI
Noometry Index29.430.2
Released2024-06-242025-06-10
WeightsOpenOpen
Context window8K128K
Max output2K40K
Input $ / M tokens$0.65$0.50
Output $ / M tokens$0.65$1.50
Results tracked3410

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Magistral Small leads

Gemma 2 27B: 34.1 (#246), Magistral Small: 38.4 (#176)

BenchmarkGemma 2 27BMagistral Small
SciCode—35.2%
BigCodeBench Instruct42.8%—
LiveBench Coding36%—
LMArena Coding1211—
BigCodeBench Complete52.5%—

Reasoning Gemma 2 27B leads

Gemma 2 27B: 15.3 (#315), Magistral Small: 6.8 (#350)

BenchmarkGemma 2 27BMagistral Small
DTBench48%61.3%
Epoch Capabilities Index122.08133.19
ARC-AGI-2—0%
Kagi LLM Benchmark—6.3%
ARC-AGI-1—5%
CritPt—0.3%
Chess Puzzles—3%
LiveBench Reasoning28.1%—
LMArena Hard Prompts1198—
LiveBench Data Analysis47.9%—
LMCA7.1%—
LiveBench38.2%—

Math Magistral Small leads

Gemma 2 27B: 10.7 (#311), Magistral Small: 26.2 (#261)

BenchmarkGemma 2 27BMagistral Small
OTIS Mock AIME 2024-20251.4%30%
LiveBench Math26.5%—
LMArena Math1212—
MATH Level 527.9%—

Knowledge Magistral Small leads

Gemma 2 27B: 19.0 (#280), Magistral Small: 30.9 (#223)

BenchmarkGemma 2 27BMagistral Small
GPQA Diamond36.5%56.1%
Confabulations27.1%—
LMArena Expert1172—
MMLU75.7%—

Multilingual Not comparable

Gemma 2 27B: 38.6 (#226), Magistral Small: —

BenchmarkGemma 2 27BMagistral Small
LMArena Non-English1217—
LMArena Chinese1221—
LMArena French1247—
LMArena German1209—
LMArena Japanese1175—
LMArena Korean1174—
LMArena Russian1234—
LMArena Spanish1228—

Instruction Following Not comparable

Gemma 2 27B: 60.5 (#249), Magistral Small: —

BenchmarkGemma 2 27BMagistral Small
LiveBench Instruction Following58.1%—
LMArena Instruction Following1206—

Long Context Not comparable

Gemma 2 27B: 37.3 (#218), Magistral Small: —

BenchmarkGemma 2 27BMagistral Small
LMArena Longer Query1231—

Writing & Preference Not comparable

Gemma 2 27B: 44.2 (#225), Magistral Small: —

BenchmarkGemma 2 27BMagistral Small
LMArena Text1231—
LMArena Creative Writing1241—
LMArena Multi-Turn1224—
LiveBench Language32.6%—

Frequently asked questions

Is Gemma 2 27B better than Magistral Small?

Gemma 2 27B and Magistral Small score almost the same on the Noometry Index (29.4 vs 30.2), so choose on price, context window or the category you care about most.

Which is cheaper, Gemma 2 27B or Magistral Small?

Gemma 2 27B is cheaper. It lists at $0.65 per million input tokens and $0.65 per million output tokens; Magistral Small lists at $0.50 and $1.50.

Is Gemma 2 27B or Magistral Small better for coding?

Magistral Small scores higher on coding benchmarks: 38.4 versus 34.1 in the Noometry coding category.

Which has the bigger context window?

Magistral Small does, with 128K tokens against 8K.

How many benchmarks do Gemma 2 27B and Magistral Small share?

4 benchmarks have published results for both models. Gemma 2 27B has 34 scored results on Noometry and Magistral Small has 10.

Related comparisons

Go deeper