Noometry

Model comparison

Grok-3 mini vs Llama 3.1-405B

Grok-3 mini is the stronger model overall, scoring 41.0 to 30.4 on the Noometry Index.

Last verified . 29 shared benchmarks.

Grok-3 mini xAI

41.0

Rank #130 Confirmed

Llama 3.1-405B Meta

30.4

Rank #279 Confirmed

Summary

  • They share 29 benchmarks with published results for both. Grok-3 mini scores higher in 7 categories and Llama 3.1-405B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Grok-3 mini leads 40.3 to 17.3.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 77.8% for Grok-3 mini and 9.7% for Llama 3.1-405B.
  • Llama 3.1-405B has downloadable open weights; the other is API-only.

Side by side

Grok-3 miniLlama 3.1-405B
ProviderxAIMeta
Noometry Index41.030.4
Released2025-04-092024-07-23
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3542

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-3 mini leads

Grok-3 mini: 40.4 (#126), Llama 3.1-405B: 32.7 (#257)

BenchmarkGrok-3 miniLlama 3.1-405B
WeirdML42.6%21.4%
LMArena Coding13801291
Aider Polyglot49.3%—

Agentic & Tool Use Not comparable

Grok-3 mini: —, Llama 3.1-405B: 20.9 (#131)

BenchmarkGrok-3 miniLlama 3.1-405B
TheAgentCompany—7.4%
Cybench—7.5%

Reasoning Llama 3.1-405B leads

Grok-3 mini: 13.8 (#323), Llama 3.1-405B: 16.4 (#293)

BenchmarkGrok-3 miniLlama 3.1-405B
Kagi LLM Benchmark61.3%45%
LMArena Hard Prompts13741270
Epoch Capabilities Index140.35128.76
ARC-AGI-20.4%—
SimpleBench—23%
ARC-AGI-116.5%—
DTBench—61.4%
BIG-Bench Hard—82.9%
ForecastBench—59.9
HellaSwag—89.2%
PIQA—85.9%
WinoGrande—89.2%

Math Grok-3 mini leads

Grok-3 mini: 40.3 (#78), Llama 3.1-405B: 17.3 (#283)

BenchmarkGrok-3 miniLlama 3.1-405B
OTIS Mock AIME 2024-202577.8%9.7%
Omni-MATH31.8%24.9%
LMArena Math13861281
MATH Level 590.9%49.8%
FrontierMath (Feb 2025 set)5.9%—

Knowledge Grok-3 mini leads

Grok-3 mini: 46.3 (#75), Llama 3.1-405B: 30.4 (#221)

BenchmarkGrok-3 miniLlama 3.1-405B
GPQA Diamond76.3%50.9%
MMLU-Pro79.9%72.3%
Confabulations10.8%17.6%
GPQA (HELM)67.5%52.2%
LMArena Expert13951243
ARC (AI2) Challenge—95.3%
MMLU—84.5%
TriviaQA—82.7%

Multilingual Grok-3 mini leads

Grok-3 mini: 48.3 (#141), Llama 3.1-405B: 40.9 (#209)

BenchmarkGrok-3 miniLlama 3.1-405B
LMArena Non-English13521248
LMArena Chinese13871242
LMArena French13571278
LMArena German13501252
LMArena Japanese13431208
LMArena Korean13361184
LMArena Russian13521265
LMArena Spanish13791259

Instruction Following Grok-3 mini leads

Grok-3 mini: 78.7 (#8), Llama 3.1-405B: 66.1 (#209)

BenchmarkGrok-3 miniLlama 3.1-405B
IFEval95.1%81.1%
LMArena Instruction Following13561259

Long Context Grok-3 mini leads

Grok-3 mini: 41.0 (#142), Llama 3.1-405B: 38.4 (#192)

BenchmarkGrok-3 miniLlama 3.1-405B
LMArena Longer Query13721266
Fiction.LiveBench66.7%—

Writing & Preference Grok-3 mini leads

Grok-3 mini: 52.7 (#163), Llama 3.1-405B: 39.1 (#246)

BenchmarkGrok-3 miniLlama 3.1-405B
LMArena Text13701284
LMArena Creative Writing13421262
WildBench65.1%78.3%
LMArena Multi-Turn13551297
Short-Story Creative Writing73.5%—
EQ-Bench Creative Writing—870

Frequently asked questions

Is Grok-3 mini better than Llama 3.1-405B?

Grok-3 mini is the stronger model overall, scoring 41.0 to 30.4 on the Noometry Index.

Is Grok-3 mini or Llama 3.1-405B better for coding?

Grok-3 mini scores higher on coding benchmarks: 40.4 versus 32.7 in the Noometry coding category.

How many benchmarks do Grok-3 mini and Llama 3.1-405B share?

29 benchmarks have published results for both models. Grok-3 mini has 35 scored results on Noometry and Llama 3.1-405B has 42.

Related comparisons

Go deeper