Noometry

Model comparison

Grok 3 vs Nemotron 4 340b Instruct

Grok 3 is the stronger model overall, scoring 39.6 to 35.3 on the Noometry Index.

Last verified . 17 shared benchmarks.

Grok 3 xAI

39.6

Rank #150 Confirmed

Nemotron 4 340b Instruct NVIDIA

35.3

Rank #218 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Grok 3 scores higher in 7 categories and Nemotron 4 340b Instruct in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok 3 leads 46.0 to 31.4.
  • Nemotron 4 340b Instruct has downloadable open weights; the other is API-only.

Side by side

Grok 3Nemotron 4 340b Instruct
ProviderxAINVIDIA
Noometry Index39.635.3
Released2025-04-092024-06-14
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4017

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 3 leads

Grok 3: 41.5 (#113), Nemotron 4 340b Instruct: 34.6 (#223)

BenchmarkGrok 3Nemotron 4 340b Instruct
LMArena Coding14321209
Aider Polyglot53.3%—
WeirdML37.2%—

Agentic & Tool Use Not comparable

Grok 3: 30.2 (#69), Nemotron 4 340b Instruct: —

BenchmarkGrok 3Nemotron 4 340b Instruct
BALROG29.5%—

Reasoning Nemotron 4 340b Instruct leads

Grok 3: 13.8 (#322), Nemotron 4 340b Instruct: 22.4 (#180)

BenchmarkGrok 3Nemotron 4 340b Instruct
LMArena Hard Prompts14331202
ARC-AGI-20%—
SimpleBench36.1%—
Kagi LLM Benchmark61.3%—
ARC-AGI-15.5%—
Epoch Capabilities Index138.34—

Math Grok 3 leads

Grok 3: 36.2 (#122), Nemotron 4 340b Instruct: 31.7 (#193)

BenchmarkGrok 3Nemotron 4 340b Instruct
LMArena Math13901216
OTIS Mock AIME 2024-202555.6%—
Omni-MATH46.4%—
MATH Level 588.7%—
FrontierMath (Feb 2025 set)3.8%—
FrontierMath Tier 4 (v1)0%—

Knowledge Grok 3 leads

Grok 3: 46.0 (#76), Nemotron 4 340b Instruct: 31.4 (#212)

BenchmarkGrok 3Nemotron 4 340b Instruct
LMArena Expert14201171
GPQA Diamond75.8%—
MMLU-Pro78.8%—
Confabulations14.2%—
Vectara Hallucination Rate5.8%—
GPQA (HELM)65%—

Multilingual Grok 3 leads

Grok 3: 52.5 (#81), Nemotron 4 340b Instruct: 38.0 (#228)

BenchmarkGrok 3Nemotron 4 340b Instruct
LMArena Non-English14111206
LMArena Chinese14481214
LMArena French14601196
LMArena German14311192
LMArena Japanese13891128
LMArena Korean13741148
LMArena Russian14161219
LMArena Spanish14171195

Instruction Following Grok 3 leads

Grok 3: 75.2 (#69), Nemotron 4 340b Instruct: 63.3 (#227)

BenchmarkGrok 3Nemotron 4 340b Instruct
LMArena Instruction Following14091203
IFEval88.4%—

Long Context Grok 3 leads

Grok 3: 38.7 (#187), Nemotron 4 340b Instruct: 37.1 (#216)

BenchmarkGrok 3Nemotron 4 340b Instruct
LMArena Longer Query14391225
Fiction.LiveBench58.3%—

Writing & Preference Grok 3 leads

Grok 3: 56.0 (#136), Nemotron 4 340b Instruct: 42.9 (#229)

BenchmarkGrok 3Nemotron 4 340b Instruct
LMArena Text14261225
LMArena Creative Writing14141203
LMArena Multi-Turn14241214
Short-Story Creative Writing76.4%—
EQ-Bench Creative Writing1186—
WildBench84.9%—

Frequently asked questions

Is Grok 3 better than Nemotron 4 340b Instruct?

Grok 3 is the stronger model overall, scoring 39.6 to 35.3 on the Noometry Index.

Is Grok 3 or Nemotron 4 340b Instruct better for coding?

Grok 3 scores higher on coding benchmarks: 41.5 versus 34.6 in the Noometry coding category.

How many benchmarks do Grok 3 and Nemotron 4 340b Instruct share?

17 benchmarks have published results for both models. Grok 3 has 40 scored results on Noometry and Nemotron 4 340b Instruct has 17.

Related comparisons

Go deeper