Noometry

Model comparison

Codellama 70b Instruct vs GPT-4 Turbo

Codellama 70b Instruct is the stronger model overall, scoring 33.7 to 30.5 on the Noometry Index.

Last verified . 7 shared benchmarks.

Codellama 70b Instruct Meta

33.7

Rank #237 Confirmed

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Summary

  • They share 7 benchmarks with published results for both. Codellama 70b Instruct scores higher in 2 categories and GPT-4 Turbo in 3 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where GPT-4 Turbo leads 40.5 to 24.8.
  • The biggest single-benchmark swing is BigCodeBench Complete: 49.6% for Codellama 70b Instruct and 58.2% for GPT-4 Turbo.
  • Codellama 70b Instruct has downloadable open weights; the other is API-only.

Side by side

Codellama 70b InstructGPT-4 Turbo
ProviderMetaOpenAI
Noometry Index33.730.5
Released—2023-11-06
WeightsOpenProprietary
Context window—128K
Max output—4K
Input $ / M tokens—$10
Output $ / M tokens—$30
Results tracked736

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Codellama 70b Instruct leads

Codellama 70b Instruct: 37.6 (#193), GPT-4 Turbo: 33.8 (#249)

BenchmarkCodellama 70b InstructGPT-4 Turbo
BigCodeBench Instruct40.7%48.2%
BigCodeBench Complete49.6%58.2%
HumanEval+65.9%86.6%
WeirdML—18%
LMArena Coding—1268
MBPP+—73.3%

Agentic & Tool Use Not comparable

Codellama 70b Instruct: —, GPT-4 Turbo: —

BenchmarkCodellama 70b InstructGPT-4 Turbo
METR Time Horizons—36.7%

Reasoning Codellama 70b Instruct leads

Codellama 70b Instruct: 20.1 (#242), GPT-4 Turbo: 15.3 (#317)

BenchmarkCodellama 70b InstructGPT-4 Turbo
LMArena Hard Prompts10521251
SimpleBench—25.1%
Chess Puzzles—6%
DTBench—61.6%
LMCA—9.8%
Epoch Capabilities Index—127.25
ForecastBench—59.4

Math Not comparable

Codellama 70b Instruct: —, GPT-4 Turbo: 9.0 (#322)

BenchmarkCodellama 70b InstructGPT-4 Turbo
FrontierMath (Tiers 1-3)—0.7%
OTIS Mock AIME 2024-2025—6.7%
LMArena Math—1272
MATH Level 5—46.7%

Knowledge Not comparable

Codellama 70b Instruct: —, GPT-4 Turbo: 24.3 (#268)

BenchmarkCodellama 70b InstructGPT-4 Turbo
GPQA Diamond—46.6%
Confabulations—28.4%
LMArena Expert—1223
MMLU—81.3%

Multimodal Not comparable

Codellama 70b Instruct: —, GPT-4 Turbo: 30.6 (#110)

BenchmarkCodellama 70b InstructGPT-4 Turbo
LMArena Vision—1090

Multilingual GPT-4 Turbo leads

Codellama 70b Instruct: 24.8 (#288), GPT-4 Turbo: 40.5 (#216)

BenchmarkCodellama 70b InstructGPT-4 Turbo
LMArena Non-English9921245
LMArena Chinese—1242
LMArena French—1276
LMArena German—1259
LMArena Japanese—1194
LMArena Korean—1187
LMArena Russian—1259
LMArena Spanish—1260

Instruction Following GPT-4 Turbo leads

Codellama 70b Instruct: 51.9 (#293), GPT-4 Turbo: 65.8 (#216)

BenchmarkCodellama 70b InstructGPT-4 Turbo
LMArena Instruction Following10241249

Long Context Not comparable

Codellama 70b Instruct: —, GPT-4 Turbo: 38.0 (#206)

BenchmarkCodellama 70b InstructGPT-4 Turbo
LMArena Longer Query—1254

Writing & Preference GPT-4 Turbo leads

Codellama 70b Instruct: 33.4 (#277), GPT-4 Turbo: 47.7 (#206)

BenchmarkCodellama 70b InstructGPT-4 Turbo
LMArena Text10571272
LMArena Creative Writing—1269
LMArena Multi-Turn—1267

Frequently asked questions

Is Codellama 70b Instruct better than GPT-4 Turbo?

Codellama 70b Instruct is the stronger model overall, scoring 33.7 to 30.5 on the Noometry Index.

Is Codellama 70b Instruct or GPT-4 Turbo better for coding?

Codellama 70b Instruct scores higher on coding benchmarks: 37.6 versus 33.8 in the Noometry coding category.

How many benchmarks do Codellama 70b Instruct and GPT-4 Turbo share?

7 benchmarks have published results for both models. Codellama 70b Instruct has 7 scored results on Noometry and GPT-4 Turbo has 36.

Related comparisons

Go deeper