Noometry

Model comparison

GPT-4 vs Llama 3.2 3B

GPT-4 and Llama 3.2 3B score almost the same on the Noometry Index (29.1 vs 28.9), so choose on price, context window or the category you care about most.

Last verified . 16 shared benchmarks.

GPT-4 OpenAI

29.1

Rank #316 Confirmed

Llama 3.2 3B Meta

28.9

Rank #321 Confirmed

Summary

  • They share 16 benchmarks with published results for both. GPT-4 scores higher in 5 categories and Llama 3.2 3B in 3 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Llama 3.2 3B leads 32.4 to 10.8.
  • The biggest single-benchmark swing is BigCodeBench Complete: 57.2% for GPT-4 and 28.3% for Llama 3.2 3B.
  • Llama 3.2 3B is cheaper at $0.05 / $0.33 per million input/output tokens, against $30 / $60 for GPT-4.
  • Llama 3.2 3B accepts more context: 131K tokens versus 8K.
  • Llama 3.2 3B has downloadable open weights; the other is API-only.

Side by side

GPT-4Llama 3.2 3B
ProviderOpenAIMeta
Noometry Index29.128.9
Released2023-03-142024-09-24
WeightsProprietaryOpen
Context window8K131K
Max output8K118K
Input $ / M tokens$30$0.05
Output $ / M tokens$60$0.33
Results tracked3818

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 leads

GPT-4: 31.6 (#283), Llama 3.2 3B: 27.6 (#319)

BenchmarkGPT-4Llama 3.2 3B
BigCodeBench Instruct46%23.4%
LMArena Coding12541098
BigCodeBench Complete57.2%28.3%
WeirdML12.4%—
HumanEval+79.3%—

Agentic & Tool Use Not comparable

GPT-4: —, Llama 3.2 3B: 20.1 (#143)

BenchmarkGPT-4Llama 3.2 3B
Berkeley Function Calling Leaderboard—21.9%
BALROG—10.1%
METR Time Horizons36.1%—

Reasoning Llama 3.2 3B leads

GPT-4: 17.8 (#289), Llama 3.2 3B: 21.0 (#228)

BenchmarkGPT-4Llama 3.2 3B
LMArena Hard Prompts12411095
Chess Puzzles4%—
Mystery Game Puzzles12%—
DTBench62.7%—
LMCA17.1%—
BIG-Bench Hard75.1%—
Epoch Capabilities Index125.89—
ForecastBench57.8—
HellaSwag95.3%—
WinoGrande87.5%—

Math Llama 3.2 3B leads

GPT-4: 10.8 (#309), Llama 3.2 3B: 32.4 (#214)

BenchmarkGPT-4Llama 3.2 3B
LMArena Math12691126
OTIS Mock AIME 2024-20251.1%—
MATH Level 523%—
GSM8K92%—

Knowledge Llama 3.2 3B leads

GPT-4: 18.4 (#282), Llama 3.2 3B: 29.7 (#235)

BenchmarkGPT-4Llama 3.2 3B
LMArena Expert12111090
GPQA Diamond35.7%—
MMLU86.4%—
TriviaQA84.8%—

Multilingual GPT-4 leads

GPT-4: 40.6 (#215), Llama 3.2 3B: 26.2 (#281)

BenchmarkGPT-4Llama 3.2 3B
LMArena Non-English12461019
LMArena Chinese12421017
LMArena German12511056
LMArena Russian1251949
LMArena French1283—
LMArena Japanese1209—
LMArena Korean1184—
LMArena Spanish1261—

Instruction Following GPT-4 leads

GPT-4: 65.3 (#222), Llama 3.2 3B: 56.0 (#275)

BenchmarkGPT-4Llama 3.2 3B
LMArena Instruction Following12411089

Long Context GPT-4 leads

GPT-4: 37.7 (#212), Llama 3.2 3B: 33.4 (#261)

BenchmarkGPT-4Llama 3.2 3B
LMArena Longer Query12441100

Writing & Preference GPT-4 leads

GPT-4: 34.9 (#268), Llama 3.2 3B: 24.7 (#307)

BenchmarkGPT-4Llama 3.2 3B
LMArena Text12631110
LMArena Creative Writing12441094
EQ-Bench Creative Writing752595
LMArena Multi-Turn12571105

Frequently asked questions

Is GPT-4 better than Llama 3.2 3B?

GPT-4 and Llama 3.2 3B score almost the same on the Noometry Index (29.1 vs 28.9), so choose on price, context window or the category you care about most.

Which is cheaper, GPT-4 or Llama 3.2 3B?

Llama 3.2 3B is cheaper. It lists at $0.05 per million input tokens and $0.33 per million output tokens; GPT-4 lists at $30 and $60.

Is GPT-4 or Llama 3.2 3B better for coding?

GPT-4 scores higher on coding benchmarks: 31.6 versus 27.6 in the Noometry coding category.

Which has the bigger context window?

Llama 3.2 3B does, with 131K tokens against 8K.

How many benchmarks do GPT-4 and Llama 3.2 3B share?

16 benchmarks have published results for both models. GPT-4 has 38 scored results on Noometry and Llama 3.2 3B has 18.

Related comparisons

Go deeper