Noometry

Model comparison

Grok 3 vs Llama 3.1-8B

Grok 3 is the stronger model overall, scoring 39.9 to 23.0 on the Noometry Index.

Last verified . 29 shared benchmarks.

Grok 3 xAI

39.9

Rank #157 Confirmed

Llama 3.1-8B Meta

23.0

Rank #352 Confirmed

Summary

  • They share 29 benchmarks with published results for both. Grok 3 scores higher in 8 categories and Llama 3.1-8B in 1 category; 9 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok 3 leads 46.2 to 8.0.
  • The biggest single-benchmark swing is MATH Level 5: 88.7% for Grok 3 and 22.9% for Llama 3.1-8B.
  • Llama 3.1-8B has downloadable open weights; the other is API-only.

Side by side

Grok 3Llama 3.1-8B
ProviderxAIMeta
Noometry Index39.923.0
Released2025-04-092024-07-23
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.05
Output $ / M tokens—$0.08
Results tracked4043

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 3 leads

Grok 3: 41.9 (#115), Llama 3.1-8B: 20.2 (#340)

BenchmarkGrok 3Llama 3.1-8B
WeirdML37.2%1.7%
LMArena Coding14321195
Aider Polyglot53.3%—
SciCode—13.2%
BigCodeBench Instruct—32.8%
BigCodeBench Complete—40.5%
HumanEval+—62.8%
MBPP+—55.6%

Agentic & Tool Use Grok 3 leads

Grok 3: 30.5 (#76), Llama 3.1-8B: 22.5 (#131)

BenchmarkGrok 3Llama 3.1-8B
BALROG29.5%15.1%
Berkeley Function Calling Leaderboard—25.8%

Reasoning Llama 3.1-8B leads

Grok 3: 13.7 (#333), Llama 3.1-8B: 14.9 (#321)

BenchmarkGrok 3Llama 3.1-8B
LMArena Hard Prompts14341175
Epoch Capabilities Index138.33116.57
ARC-AGI-20%—
SimpleBench36.1%—
Kagi LLM Benchmark61.3%—
ARC-AGI-15.5%—
CritPt—0%
Chess Puzzles—0%
DTBench—50.9%
LMCA—5.4%
PIQA—81.2%

Math Grok 3 leads

Grok 3: 38.0 (#145), Llama 3.1-8B: 10.2 (#317)

BenchmarkGrok 3Llama 3.1-8B
OTIS Mock AIME 2024-202555.6%1.7%
Omni-MATH46.4%13.7%
LMArena Math13911179
MATH Level 588.7%22.9%
FrontierMath (Feb 2025 set)3.8%—
FrontierMath Tier 4 (v1)0%—
GSM8K—82.4%

Knowledge Grok 3 leads

Grok 3: 46.2 (#82), Llama 3.1-8B: 8.0 (#307)

BenchmarkGrok 3Llama 3.1-8B
GPQA Diamond75.8%27%
MMLU-Pro78.8%40.6%
GPQA (HELM)65%24.7%
LMArena Expert14211144
Confabulations14.2%—
Vectara Hallucination Rate5.8%—
BoolQ—82.8%
MMLU—56.1%

Multilingual Grok 3 leads

Grok 3: 52.3 (#87), Llama 3.1-8B: 34.0 (#249)

BenchmarkGrok 3Llama 3.1-8B
LMArena Non-English14101148
LMArena Chinese14481151
LMArena French14601177
LMArena German14311144
LMArena Japanese13871061
LMArena Korean13731053
LMArena Russian14161158
LMArena Spanish14171169

Instruction Following Grok 3 leads

Grok 3: 75.0 (#73), Llama 3.1-8B: 58.9 (#258)

BenchmarkGrok 3Llama 3.1-8B
IFEval88.4%74.3%
LMArena Instruction Following14091159

Long Context Grok 3 leads

Grok 3: 38.7 (#192), Llama 3.1-8B: 35.8 (#238)

BenchmarkGrok 3Llama 3.1-8B
LMArena Longer Query14391182
Fiction.LiveBench58.3%—

Writing & Preference Grok 3 leads

Grok 3: 55.8 (#141), Llama 3.1-8B: 29.7 (#290)

BenchmarkGrok 3Llama 3.1-8B
LMArena Text14261187
LMArena Creative Writing14141154
EQ-Bench Creative Writing1186713
WildBench84.9%68.7%
LMArena Multi-Turn14251172
Short-Story Creative Writing76.4%—

Frequently asked questions

Is Grok 3 better than Llama 3.1-8B?

Grok 3 is the stronger model overall, scoring 39.9 to 23.0 on the Noometry Index.

Is Grok 3 or Llama 3.1-8B better for coding?

Grok 3 scores higher on coding benchmarks: 41.9 versus 20.2 in the Noometry coding category.

How many benchmarks do Grok 3 and Llama 3.1-8B share?

29 benchmarks have published results for both models. Grok 3 has 40 scored results on Noometry and Llama 3.1-8B has 43.

Related comparisons

Go deeper