Noometry

Model comparison

GPT-5.2 Codex vs Llama 3.1-8B

GPT-5.2 Codex is the stronger model overall, scoring 42.6 to 23.0 on the Noometry Index. Llama 3.1-8B costs 84× less per token, which makes it the better buy when GPT-5.2 Codex's lead doesn't matter for your workload.

Last verified . 0 shared benchmarks.

GPT-5.2 Codex OpenAI

42.6

Rank #111 Reported

Llama 3.1-8B Meta

23.0

Rank #352 Confirmed

Summary

  • The widest gap is in coding, where GPT-5.2 Codex leads 45.5 to 20.2.
  • Llama 3.1-8B is cheaper at $0.05 / $0.08 per million input/output tokens, against $1.75 / $14 for GPT-5.2 Codex.
  • GPT-5.2 Codex accepts more context: 400K tokens versus 128K.
  • Llama 3.1-8B has downloadable open weights; the other is API-only.

Side by side

GPT-5.2 CodexLlama 3.1-8B
ProviderOpenAIMeta
Noometry Index42.623.0
Released2025-12-182024-07-23
WeightsProprietaryOpen
Context window400K128K
Max output128K4K
Input $ / M tokens$1.75$0.05
Output $ / M tokens$14$0.08
Results tracked543

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.2 Codex leads

GPT-5.2 Codex: 45.5 (#71), Llama 3.1-8B: 20.2 (#340)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
SWE-bench Verified (bash only)72.8%—
LMArena WebDev1339—
SWE-bench Multilingual66.3%—
SciCode—13.2%
WeirdML—1.7%
BigCodeBench Instruct—32.8%
LMArena Coding—1195
BigCodeBench Complete—40.5%
ALE-Bench1,300—
HumanEval+—62.8%
MBPP+—55.6%

Agentic & Tool Use GPT-5.2 Codex leads

GPT-5.2 Codex: 41.0 (#22), Llama 3.1-8B: 22.5 (#131)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
Terminal-Bench66.5%—
Berkeley Function Calling Leaderboard—25.8%
BALROG—15.1%

Reasoning Not comparable

GPT-5.2 Codex: —, Llama 3.1-8B: 14.9 (#321)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
CritPt—0%
Chess Puzzles—0%
LMArena Hard Prompts—1175
DTBench—50.9%
LMCA—5.4%
Epoch Capabilities Index—116.57
PIQA—81.2%

Math Not comparable

GPT-5.2 Codex: —, Llama 3.1-8B: 10.2 (#317)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
OTIS Mock AIME 2024-2025—1.7%
Omni-MATH—13.7%
LMArena Math—1179
MATH Level 5—22.9%
GSM8K—82.4%

Knowledge Not comparable

GPT-5.2 Codex: —, Llama 3.1-8B: 8.0 (#307)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
GPQA Diamond—27%
MMLU-Pro—40.6%
GPQA (HELM)—24.7%
LMArena Expert—1144
BoolQ—82.8%
MMLU—56.1%

Multilingual Not comparable

GPT-5.2 Codex: —, Llama 3.1-8B: 34.0 (#249)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
LMArena Non-English—1148
LMArena Chinese—1151
LMArena French—1177
LMArena German—1144
LMArena Japanese—1061
LMArena Korean—1053
LMArena Russian—1158
LMArena Spanish—1169

Instruction Following Not comparable

GPT-5.2 Codex: —, Llama 3.1-8B: 58.9 (#258)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
IFEval—74.3%
LMArena Instruction Following—1159

Long Context Not comparable

GPT-5.2 Codex: —, Llama 3.1-8B: 35.8 (#238)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
LMArena Longer Query—1182

Writing & Preference Not comparable

GPT-5.2 Codex: —, Llama 3.1-8B: 29.7 (#290)

BenchmarkGPT-5.2 CodexLlama 3.1-8B
LMArena Text—1187
LMArena Creative Writing—1154
EQ-Bench Creative Writing—713
WildBench—68.7%
LMArena Multi-Turn—1172

Frequently asked questions

Is GPT-5.2 Codex better than Llama 3.1-8B?

GPT-5.2 Codex is the stronger model overall, scoring 42.6 to 23.0 on the Noometry Index. Llama 3.1-8B costs 84× less per token, which makes it the better buy when GPT-5.2 Codex's lead doesn't matter for your workload.

Which is cheaper, GPT-5.2 Codex or Llama 3.1-8B?

Llama 3.1-8B is cheaper. It lists at $0.05 per million input tokens and $0.08 per million output tokens; GPT-5.2 Codex lists at $1.75 and $14.

Is GPT-5.2 Codex or Llama 3.1-8B better for coding?

GPT-5.2 Codex scores higher on coding benchmarks: 45.5 versus 20.2 in the Noometry coding category.

Which has the bigger context window?

GPT-5.2 Codex does, with 400K tokens against 128K.

How many benchmarks do GPT-5.2 Codex and Llama 3.1-8B share?

0 benchmarks have published results for both models. GPT-5.2 Codex has 5 scored results on Noometry and Llama 3.1-8B has 43.

Related comparisons

Go deeper