Noometry

Model comparison

Gemma 3n E4b IT vs Mistral Small

Gemma 3n E4b IT is the stronger model overall, scoring 37.3 to 33.4 on the Noometry Index.

Last verified . 18 shared benchmarks.

Gemma 3n E4b IT Google

37.3

Rank #206 Confirmed

Mistral Small Mistral AI

33.4

Rank #243 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Gemma 3n E4b IT scores higher in 4 categories and Mistral Small in 4 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in math, where Gemma 3n E4b IT leads 35.1 to 16.4.
  • The biggest single-benchmark swing is Kagi LLM Benchmark: 31.5% for Gemma 3n E4b IT and 37.8% for Mistral Small.

Side by side

Gemma 3n E4b ITMistral Small
ProviderGoogleMistral AI
Noometry Index37.333.4
Released—2024-02-26
WeightsOpenOpen
Context window—262K
Max output—256K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.60
Results tracked1839

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 3n E4b IT leads

Gemma 3n E4b IT: 37.0 (#198), Mistral Small: 34.0 (#247)

BenchmarkGemma 3n E4b ITMistral Small
LMArena Coding12681362
SciCode—26.5%
BigCodeBench Instruct—36.1%
LiveBench Coding—36.2%
BigCodeBench Complete—46.6%
ALE-Bench—497.62

Agentic & Tool Use Not comparable

Gemma 3n E4b IT: —, Mistral Small: 28.1 (#93)

BenchmarkGemma 3n E4b ITMistral Small
Berkeley Function Calling Leaderboard—37.1%

Reasoning Too close to call

Gemma 3n E4b IT: 19.9 (#247), Mistral Small: 19.8 (#250)

BenchmarkGemma 3n E4b ITMistral Small
Kagi LLM Benchmark31.5%37.8%
LMArena Hard Prompts12841335
CritPt—0%
LiveBench Reasoning—44.8%
DTBench—70.9%
LiveBench Data Analysis—53.7%
LMCA—20.6%
LiveBench—44%

Math Gemma 3n E4b IT leads

Gemma 3n E4b IT: 35.1 (#188), Mistral Small: 16.4 (#293)

BenchmarkGemma 3n E4b ITMistral Small
LMArena Math12511341
OTIS Mock AIME 2024-2025—5.8%
LiveBench Math—39.9%
MATH Level 5—46.8%

Knowledge Gemma 3n E4b IT leads

Gemma 3n E4b IT: 34.2 (#198), Mistral Small: 31.0 (#222)

BenchmarkGemma 3n E4b ITMistral Small
LMArena Expert12461291
GPQA Diamond—47.5%
Vectara Hallucination Rate—5.1%
MMLU—68.7%

Multimodal Not comparable

Gemma 3n E4b IT: —, Mistral Small: 33.5 (#96)

BenchmarkGemma 3n E4b ITMistral Small
LMArena Vision—1142

Multilingual Mistral Small leads

Gemma 3n E4b IT: 43.4 (#183), Mistral Small: 45.5 (#169)

BenchmarkGemma 3n E4b ITMistral Small
LMArena Non-English12851315
LMArena Chinese13091340
LMArena French13301337
LMArena German13111340
LMArena Japanese12721275
LMArena Korean12591259
LMArena Russian12881324
LMArena Spanish13051346

Instruction Following Too close to call

Gemma 3n E4b IT: 66.1 (#210), Mistral Small: 66.4 (#209)

BenchmarkGemma 3n E4b ITMistral Small
LMArena Instruction Following12551310
LiveBench Instruction Following—63.7%

Long Context Mistral Small leads

Gemma 3n E4b IT: 38.7 (#191), Mistral Small: 40.4 (#156)

BenchmarkGemma 3n E4b ITMistral Small
LMArena Longer Query12761327

Writing & Preference Mistral Small leads

Gemma 3n E4b IT: 50.1 (#186), Mistral Small: 52.5 (#171)

BenchmarkGemma 3n E4b ITMistral Small
LMArena Text13061338
LMArena Creative Writing12871305
LMArena Multi-Turn12761344
LiveBench Language—30.5%

Frequently asked questions

Is Gemma 3n E4b IT better than Mistral Small?

Gemma 3n E4b IT is the stronger model overall, scoring 37.3 to 33.4 on the Noometry Index.

Is Gemma 3n E4b IT or Mistral Small better for coding?

Gemma 3n E4b IT scores higher on coding benchmarks: 37.0 versus 34.0 in the Noometry coding category.

How many benchmarks do Gemma 3n E4b IT and Mistral Small share?

18 benchmarks have published results for both models. Gemma 3n E4b IT has 18 scored results on Noometry and Mistral Small has 39.

Related comparisons

Go deeper