Noometry

Model comparison

Codellama 34b Instruct vs GPT-4 Turbo

Codellama 34b Instruct and GPT-4 Turbo score almost the same on the Noometry Index (30.8 vs 30.5), so choose on price, context window or the category you care about most.

Last verified . 14 shared benchmarks.

Codellama 34b Instruct Meta

30.8

Rank #287 Confirmed

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Codellama 34b Instruct scores higher in 2 categories and GPT-4 Turbo in 5 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Codellama 34b Instruct leads 31.0 to 9.0.
  • The biggest single-benchmark swing is BigCodeBench Complete: 37.1% for Codellama 34b Instruct and 58.2% for GPT-4 Turbo.
  • Codellama 34b Instruct has downloadable open weights; the other is API-only.

Side by side

Codellama 34b InstructGPT-4 Turbo
ProviderMetaOpenAI
Noometry Index30.830.5
Released—2023-11-06
WeightsOpenProprietary
Context window—128K
Max output—4K
Input $ / M tokens—$10
Output $ / M tokens—$30
Results tracked1436

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 Turbo leads

Codellama 34b Instruct: 28.5 (#314), GPT-4 Turbo: 33.8 (#249)

BenchmarkCodellama 34b InstructGPT-4 Turbo
BigCodeBench Instruct29%48.2%
LMArena Coding10461268
BigCodeBench Complete37.1%58.2%
HumanEval+43.9%86.6%
MBPP+56.3%73.3%
WeirdML—18%

Agentic & Tool Use Not comparable

Codellama 34b Instruct: —, GPT-4 Turbo: —

BenchmarkCodellama 34b InstructGPT-4 Turbo
METR Time Horizons—36.7%

Reasoning Codellama 34b Instruct leads

Codellama 34b Instruct: 19.6 (#255), GPT-4 Turbo: 15.3 (#317)

BenchmarkCodellama 34b InstructGPT-4 Turbo
LMArena Hard Prompts10321251
SimpleBench—25.1%
Chess Puzzles—6%
DTBench—61.6%
LMCA—9.8%
Epoch Capabilities Index—127.25
ForecastBench—59.4

Math Codellama 34b Instruct leads

Codellama 34b Instruct: 31.0 (#230), GPT-4 Turbo: 9.0 (#322)

BenchmarkCodellama 34b InstructGPT-4 Turbo
LMArena Math10561272
FrontierMath (Tiers 1-3)—0.7%
OTIS Mock AIME 2024-2025—6.7%
MATH Level 5—46.7%

Knowledge Not comparable

Codellama 34b Instruct: —, GPT-4 Turbo: 24.3 (#268)

BenchmarkCodellama 34b InstructGPT-4 Turbo
GPQA Diamond—46.6%
Confabulations—28.4%
LMArena Expert—1223
MMLU—81.3%

Multimodal Not comparable

Codellama 34b Instruct: —, GPT-4 Turbo: 30.6 (#110)

BenchmarkCodellama 34b InstructGPT-4 Turbo
LMArena Vision—1090

Multilingual GPT-4 Turbo leads

Codellama 34b Instruct: 25.8 (#284), GPT-4 Turbo: 40.5 (#216)

BenchmarkCodellama 34b InstructGPT-4 Turbo
LMArena Non-English10111245
LMArena Chinese9761242
LMArena French—1276
LMArena German—1259
LMArena Japanese—1194
LMArena Korean—1187
LMArena Russian—1259
LMArena Spanish—1260

Instruction Following GPT-4 Turbo leads

Codellama 34b Instruct: 52.2 (#291), GPT-4 Turbo: 65.8 (#216)

BenchmarkCodellama 34b InstructGPT-4 Turbo
LMArena Instruction Following10281249

Long Context GPT-4 Turbo leads

Codellama 34b Instruct: 30.9 (#284), GPT-4 Turbo: 38.0 (#206)

BenchmarkCodellama 34b InstructGPT-4 Turbo
LMArena Longer Query10131254

Writing & Preference GPT-4 Turbo leads

Codellama 34b Instruct: 28.2 (#297), GPT-4 Turbo: 47.7 (#206)

BenchmarkCodellama 34b InstructGPT-4 Turbo
LMArena Text10661272
LMArena Creative Writing10321269
LMArena Multi-Turn10151267

Frequently asked questions

Is Codellama 34b Instruct better than GPT-4 Turbo?

Codellama 34b Instruct and GPT-4 Turbo score almost the same on the Noometry Index (30.8 vs 30.5), so choose on price, context window or the category you care about most.

Is Codellama 34b Instruct or GPT-4 Turbo better for coding?

GPT-4 Turbo scores higher on coding benchmarks: 33.8 versus 28.5 in the Noometry coding category.

How many benchmarks do Codellama 34b Instruct and GPT-4 Turbo share?

14 benchmarks have published results for both models. Codellama 34b Instruct has 14 scored results on Noometry and GPT-4 Turbo has 36.

Related comparisons

Go deeper