Noometry

Model comparison

Grok-2 (Dec 2024) vs Llama-3.3-70B-Instruct

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.7 to 30.6 on the Noometry Index.

Last verified . 32 shared benchmarks.

Grok-2 (Dec 2024) xAI

33.7

Rank #239 Confirmed

Llama-3.3-70B-Instruct Meta

30.6

Rank #291 Confirmed

Summary

  • They share 32 benchmarks with published results for both. Grok-2 (Dec 2024) scores higher in 6 categories and Llama-3.3-70B-Instruct in 2 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Grok-2 (Dec 2024) leads 38.8 to 26.4.
  • The biggest single-benchmark swing is MATH Level 5: 63.5% for Grok-2 (Dec 2024) and 41.6% for Llama-3.3-70B-Instruct.
  • Llama-3.3-70B-Instruct has downloadable open weights; the other is API-only.

Side by side

Grok-2 (Dec 2024)Llama-3.3-70B-Instruct
ProviderxAIMeta
Noometry Index33.730.6
Released2024-08-132024-12-06
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.32
Results tracked3443

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 33.3 (#258), Llama-3.3-70B-Instruct: 31.0 (#290)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
WeirdML22.2%14.4%
LiveBench Coding46.4%36.6%
LMArena Coding12871268
SciCode—26%
BigCodeBench Instruct—46.9%
BigCodeBench Complete—57.5%

Agentic & Tool Use Not comparable

Grok-2 (Dec 2024): —, Llama-3.3-70B-Instruct: 25.8 (#105)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
Berkeley Function Calling Leaderboard—31.9%
BALROG—23%

Reasoning Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 16.9 (#299), Llama-3.3-70B-Instruct: 14.1 (#327)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
SimpleBench22.7%19.9%
LiveBench Reasoning54.8%50.8%
LMArena Hard Prompts12721257
DTBench65.2%59.5%
LiveBench Data Analysis54.5%49.5%
Epoch Capabilities Index130.48127.33
LiveBench54.3%50.2%
CritPt—0%
LMCA—17.5%
ForecastBench—58.6

Math Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 20.8 (#284), Llama-3.3-70B-Instruct: 15.3 (#298)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
OTIS Mock AIME 2024-202511.5%5.1%
LiveBench Math54.9%42.2%
LMArena Math12831267
MATH Level 563.5%41.6%
FrontierMath (Feb 2025 set)0.7%—

Knowledge Too close to call

Grok-2 (Dec 2024): 29.8 (#233), Llama-3.3-70B-Instruct: 30.6 (#226)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
GPQA Diamond53.8%47.4%
Confabulations20.1%22.8%
LMArena Expert12541225
Vectara Hallucination Rate—4.1%
MMLU—86.3%

Multilingual Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 43.1 (#188), Llama-3.3-70B-Instruct: 39.9 (#220)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
LMArena Non-English12821236
LMArena Chinese12891217
LMArena French13181281
LMArena German12871251
LMArena Japanese12441150
LMArena Korean12371143
LMArena Russian12861252
LMArena Spanish12811270

Instruction Following Llama-3.3-70B-Instruct leads

Grok-2 (Dec 2024): 66.9 (#202), Llama-3.3-70B-Instruct: 71.1 (#157)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
LiveBench Instruction Following69.6%82.7%
LMArena Instruction Following12701242

Long Context Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 38.8 (#190), Llama-3.3-70B-Instruct: 26.4 (#295)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
LMArena Longer Query12761256
Fiction.LiveBench—33.3%

Writing & Preference Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 48.6 (#198), Llama-3.3-70B-Instruct: 47.6 (#207)

BenchmarkGrok-2 (Dec 2024)Llama-3.3-70B-Instruct
LMArena Text13051274
LMArena Creative Writing12841250
LMArena Multi-Turn12901280
LiveBench Language45.6%39.2%
Short-Story Creative Writing63.6%—

Frequently asked questions

Is Grok-2 (Dec 2024) better than Llama-3.3-70B-Instruct?

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.7 to 30.6 on the Noometry Index.

Is Grok-2 (Dec 2024) or Llama-3.3-70B-Instruct better for coding?

Grok-2 (Dec 2024) scores higher on coding benchmarks: 33.3 versus 31.0 in the Noometry coding category.

How many benchmarks do Grok-2 (Dec 2024) and Llama-3.3-70B-Instruct share?

32 benchmarks have published results for both models. Grok-2 (Dec 2024) has 34 scored results on Noometry and Llama-3.3-70B-Instruct has 43.

Related comparisons

Go deeper