Noometry

Model comparison

Codellama 70b Instruct vs o3

o3 is the stronger model overall, scoring 47.5 to 33.7 on the Noometry Index.

Last verified . 4 shared benchmarks.

Codellama 70b Instruct Meta

33.7

Rank #237 Confirmed

o3 OpenAI

47.5

Rank #61 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Codellama 70b Instruct scores higher in 0 categories and o3 in 5 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where o3 leads 63.5 to 33.4.
  • Codellama 70b Instruct has downloadable open weights; the other is API-only.

Side by side

Codellama 70b Instructo3
ProviderMetaOpenAI
Noometry Index33.747.5
Released—2025-04-16
WeightsOpenProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$2
Output $ / M tokens—$8
Results tracked763

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

Codellama 70b Instruct: 37.6 (#193), o3: 46.8 (#64)

BenchmarkCodellama 70b Instructo3
SWE-bench Verified—62.3%
SWE-bench Verified (bash only)—58.4%
Aider Polyglot—81.3%
GSO—8.8%
WeirdML—52.4%
BigCodeBench Instruct40.7%—
LMArena Coding—1408
BigCodeBench Complete49.6%—
CadEval—74%
ALE-Bench—933.55
HumanEval+65.9%—

Agentic & Tool Use Not comparable

Codellama 70b Instruct: —, o3: 34.5 (#44)

BenchmarkCodellama 70b Instructo3
Berkeley Function Calling Leaderboard—63%
GDPval—30.8%
DeepResearch Bench—45.2%
OSWorld—23%
LMArena Search—1144
METR Time Horizons—65.4%

Reasoning o3 leads

Codellama 70b Instruct: 20.1 (#242), o3: 32.0 (#78)

BenchmarkCodellama 70b Instructo3
LMArena Hard Prompts10521402
ARC-AGI-2—6.5%
SimpleBench—53.1%
Kagi LLM Benchmark—67.6%
ARC-AGI-1—60.8%
CritPt—1.4%
Chess Puzzles—38%
EnigmaEval—13.1%
Mystery Game Puzzles—29%
DTBench—84.8%
LMCA—39.7%
Epoch Capabilities Index—146.86
ForecastBench—62.5

Math Not comparable

Codellama 70b Instruct: —, o3: 50.2 (#58)

BenchmarkCodellama 70b Instructo3
FrontierMath (Tiers 1-3)—33.3%
OTIS Mock AIME 2024-2025—84.4%
Omni-MATH—71.4%
LMArena Math—1426
MATH Level 5—97.8%
FrontierMath (Feb 2025 set)—18.7%
FrontierMath Tier 4 (v1)—2.1%

Knowledge Not comparable

Codellama 70b Instruct: —, o3: 54.6 (#52)

BenchmarkCodellama 70b Instructo3
GPQA Diamond—81.8%
Humanity's Last Exam—20.3%
SimpleQA Verified—49.4%
MMLU-Pro—85.9%
Confabulations—14.4%
GPQA (HELM)—75.3%
LMArena Expert—1402

Multimodal Not comparable

Codellama 70b Instruct: —, o3: 41.4 (#36)

BenchmarkCodellama 70b Instructo3
LMArena Vision—1214
GeoBench—74%
VPCT—52%

Multilingual o3 leads

Codellama 70b Instruct: 24.8 (#288), o3: 51.7 (#105)

BenchmarkCodellama 70b Instructo3
LMArena Non-English9921401
LMArena Chinese—1437
LMArena French—1430
LMArena German—1420
LMArena Japanese—1403
LMArena Korean—1370
LMArena Russian—1406
LMArena Spanish—1395

Instruction Following o3 leads

Codellama 70b Instruct: 51.9 (#293), o3: 72.8 (#127)

BenchmarkCodellama 70b Instructo3
LMArena Instruction Following10241368
IFEval—86.9%

Long Context Not comparable

Codellama 70b Instruct: —, o3: 53.3 (#6)

BenchmarkCodellama 70b Instructo3
Fiction.LiveBench—88.9%
CL-bench—17.8%
LMArena Longer Query—1372

Writing & Preference o3 leads

Codellama 70b Instruct: 33.4 (#277), o3: 63.5 (#64)

BenchmarkCodellama 70b Instructo3
LMArena Text10571410
LMArena Creative Writing—1359
Short-Story Creative Writing—83.9%
EQ-Bench Creative Writing—1676
WildBench—86.1%
LMArena Multi-Turn—1405

Frequently asked questions

Is Codellama 70b Instruct better than o3?

o3 is the stronger model overall, scoring 47.5 to 33.7 on the Noometry Index.

Is Codellama 70b Instruct or o3 better for coding?

o3 scores higher on coding benchmarks: 46.8 versus 37.6 in the Noometry coding category.

How many benchmarks do Codellama 70b Instruct and o3 share?

4 benchmarks have published results for both models. Codellama 70b Instruct has 7 scored results on Noometry and o3 has 63.

Related comparisons

Go deeper