Noometry

Model comparison

GPT-5.4 nano vs Llama 3-70B

GPT-5.4 nano is the stronger model overall, scoring 41.9 to 28.8 on the Noometry Index.

Last verified . 23 shared benchmarks.

GPT-5.4 nano OpenAI

41.9

Rank #125 Confirmed

Llama 3-70B Meta

28.8

Rank #323 Confirmed

Summary

  • They share 23 benchmarks with published results for both. GPT-5.4 nano scores higher in 8 categories and Llama 3-70B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-5.4 nano leads 40.9 to 12.8.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 87.8% for GPT-5.4 nano and 4.3% for Llama 3-70B.
  • Llama 3-70B has downloadable open weights; the other is API-only.

Side by side

GPT-5.4 nanoLlama 3-70B
ProviderOpenAIMeta
Noometry Index41.928.8
Released2026-03-172024-04-18
WeightsProprietaryOpen
Context window400K—
Max output128K—
Input $ / M tokens$0.20—
Output $ / M tokens$1.25—
Results tracked4031

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 nano leads

GPT-5.4 nano: 43.6 (#84), Llama 3-70B: 35.8 (#218)

BenchmarkGPT-5.4 nanoLlama 3-70B
LMArena Coding14051206
SciCode46.9%—
WeirdML49.2%—
BigCodeBench Instruct—43.6%
BigCodeBench Complete—54.5%
ALE-Bench1,005—
HumanEval+—72%
MBPP+—69%

Agentic & Tool Use Not comparable

GPT-5.4 nano: —, Llama 3-70B: 21.1 (#139)

BenchmarkGPT-5.4 nanoLlama 3-70B
Cybench—5%

Reasoning GPT-5.4 nano leads

GPT-5.4 nano: 23.7 (#173), Llama 3-70B: 18.0 (#288)

BenchmarkGPT-5.4 nanoLlama 3-70B
Kagi LLM Benchmark39.7%35.1%
LMArena Hard Prompts13811195
DTBench80.3%54.2%
Epoch Capabilities Index145.81122.93
ForecastBench57.357.1
ARC-AGI-25.7%—
ARC-AGI-151.5%—
CritPt9.3%—
Chess Puzzles30%—
Mystery Game Puzzles9%—
LMCA36.9%—
WinoGrande—83.5%

Math GPT-5.4 nano leads

GPT-5.4 nano: 40.9 (#88), Llama 3-70B: 12.8 (#305)

BenchmarkGPT-5.4 nanoLlama 3-70B
OTIS Mock AIME 2024-202587.8%4.3%
LMArena Math14061218
FrontierMath (Tiers 1-3)44.9%—
FrontierMath Tier 412.2%—
ProofBench5%—
MATH Level 5—22.6%
FrontierMath (Feb 2025 set)25.9%—
FrontierMath Tier 4 (v1)6.3%—

Knowledge GPT-5.4 nano leads

GPT-5.4 nano: 41.9 (#103), Llama 3-70B: 20.8 (#277)

BenchmarkGPT-5.4 nanoLlama 3-70B
GPQA Diamond78.5%40.6%
LMArena Expert13961149
SimpleQA Verified11.7%—
Vectara Hallucination Rate3.1%—
MMLU—79.3%

Multimodal Not comparable

GPT-5.4 nano: 36.7 (#78), Llama 3-70B: —

BenchmarkGPT-5.4 nanoLlama 3-70B
LMArena Vision1196—

Multilingual GPT-5.4 nano leads

GPT-5.4 nano: 48.6 (#140), Llama 3-70B: 33.6 (#251)

BenchmarkGPT-5.4 nanoLlama 3-70B
LMArena Non-English13591142
LMArena Chinese13921114
LMArena French13961232
LMArena German13671169
LMArena Japanese13431017
LMArena Korean13201017
LMArena Russian13631159
LMArena Spanish13711241

Instruction Following GPT-5.4 nano leads

GPT-5.4 nano: 71.9 (#144), Llama 3-70B: 62.5 (#238)

BenchmarkGPT-5.4 nanoLlama 3-70B
LMArena Instruction Following13621194

Long Context GPT-5.4 nano leads

GPT-5.4 nano: 41.6 (#137), Llama 3-70B: 35.6 (#240)

BenchmarkGPT-5.4 nanoLlama 3-70B
LMArena Longer Query13661174

Writing & Preference GPT-5.4 nano leads

GPT-5.4 nano: 55.7 (#142), Llama 3-70B: 42.8 (#231)

BenchmarkGPT-5.4 nanoLlama 3-70B
LMArena Text13721221
LMArena Creative Writing13141210
LMArena Multi-Turn13821223

Frequently asked questions

Is GPT-5.4 nano better than Llama 3-70B?

GPT-5.4 nano is the stronger model overall, scoring 41.9 to 28.8 on the Noometry Index.

Is GPT-5.4 nano or Llama 3-70B better for coding?

GPT-5.4 nano scores higher on coding benchmarks: 43.6 versus 35.8 in the Noometry coding category.

How many benchmarks do GPT-5.4 nano and Llama 3-70B share?

23 benchmarks have published results for both models. GPT-5.4 nano has 40 scored results on Noometry and Llama 3-70B has 31.

Related comparisons

Go deeper