Noometry

Model comparison

Gemma 3 4B vs Magistral Small

Magistral Small is the stronger model overall, scoring 29.9 to 27.9 on the Noometry Index. Gemma 3 4B costs 15× less per token, which makes it the better buy when Magistral Small's lead doesn't matter for your workload.

Last verified . 6 shared benchmarks.

Gemma 3 4B Google

27.9

Rank #320 Confirmed

Magistral Small Mistral AI

29.9

Rank #289 Confirmed

Summary

  • They share 6 benchmarks with published results for both. Gemma 3 4B scores higher in 1 category and Magistral Small in 3 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Magistral Small leads 30.9 to 11.8.
  • The biggest single-benchmark swing is GPQA Diamond: 23.2% for Gemma 3 4B and 56.1% for Magistral Small.
  • Gemma 3 4B is cheaper at $0.04 / $0.08 per million input/output tokens, against $0.50 / $1.50 for Magistral Small.
  • Gemma 3 4B accepts more context: 131K tokens versus 128K.

Side by side

Gemma 3 4BMagistral Small
ProviderGoogleMistral AI
Noometry Index27.929.9
Released2025-03-122025-06-10
WeightsOpenOpen
Context window131K128K
Max output4K40K
Input $ / M tokens$0.04$0.50
Output $ / M tokens$0.08$1.50
Results tracked2210

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Magistral Small leads

Gemma 3 4B: 35.3 (#214), Magistral Small: 37.8 (#172)

BenchmarkGemma 3 4BMagistral Small
SciCode—35.2%
LMArena Coding1230—

Agentic & Tool Use Not comparable

Gemma 3 4B: 20.7 (#134), Magistral Small: —

BenchmarkGemma 3 4BMagistral Small
Berkeley Function Calling Leaderboard19.6%—

Reasoning Gemma 3 4B leads

Gemma 3 4B: 12.7 (#330), Magistral Small: 6.9 (#345)

BenchmarkGemma 3 4BMagistral Small
Kagi LLM Benchmark25.2%6.3%
Chess Puzzles0%3%
DTBench50.9%61.3%
Epoch Capabilities Index116.02133.19
ARC-AGI-2—0%
ARC-AGI-1—5%
CritPt—0.3%
LMArena Hard Prompts1253—
LMCA2.8%—

Math Magistral Small leads

Gemma 3 4B: 15.8 (#286), Magistral Small: 24.9 (#253)

BenchmarkGemma 3 4BMagistral Small
OTIS Mock AIME 2024-20257.5%30%
LMArena Math1239—

Knowledge Magistral Small leads

Gemma 3 4B: 11.8 (#294), Magistral Small: 30.9 (#215)

BenchmarkGemma 3 4BMagistral Small
GPQA Diamond23.2%56.1%
Vectara Hallucination Rate6.4%—
LMArena Expert1223—

Multilingual Not comparable

Gemma 3 4B: 42.7 (#189), Magistral Small: —

BenchmarkGemma 3 4BMagistral Small
LMArena Non-English1273—
LMArena German1281—
LMArena Russian1294—

Instruction Following Not comparable

Gemma 3 4B: 65.4 (#219), Magistral Small: —

BenchmarkGemma 3 4BMagistral Small
LMArena Instruction Following1239—

Long Context Not comparable

Gemma 3 4B: 38.6 (#189), Magistral Small: —

BenchmarkGemma 3 4BMagistral Small
LMArena Longer Query1274—

Writing & Preference Not comparable

Gemma 3 4B: 42.2 (#234), Magistral Small: —

BenchmarkGemma 3 4BMagistral Small
LMArena Text1291—
LMArena Creative Writing1270—
EQ-Bench Creative Writing1068—
LMArena Multi-Turn1255—

Frequently asked questions

Is Gemma 3 4B better than Magistral Small?

Magistral Small is the stronger model overall, scoring 29.9 to 27.9 on the Noometry Index. Gemma 3 4B costs 15× less per token, which makes it the better buy when Magistral Small's lead doesn't matter for your workload.

Which is cheaper, Gemma 3 4B or Magistral Small?

Gemma 3 4B is cheaper. It lists at $0.04 per million input tokens and $0.08 per million output tokens; Magistral Small lists at $0.50 and $1.50.

Is Gemma 3 4B or Magistral Small better for coding?

Magistral Small scores higher on coding benchmarks: 37.8 versus 35.3 in the Noometry coding category.

Which has the bigger context window?

Gemma 3 4B does, with 131K tokens against 128K.

How many benchmarks do Gemma 3 4B and Magistral Small share?

6 benchmarks have published results for both models. Gemma 3 4B has 22 scored results on Noometry and Magistral Small has 10.

Related comparisons

Go deeper