Noometry

Model comparison

Codellama 70b Instruct vs GPT-3.5-turbo

Codellama 70b Instruct is the stronger model overall, scoring 33.7 to 23.2 on the Noometry Index.

Last verified . 7 shared benchmarks.

Codellama 70b Instruct Meta

33.7

Rank #237 Confirmed

GPT-3.5-turbo OpenAI

23.2

Rank #350 Confirmed

Summary

  • They share 7 benchmarks with published results for both. Codellama 70b Instruct scores higher in 3 categories and GPT-3.5-turbo in 2 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Codellama 70b Instruct leads 37.6 to 23.9.
  • Codellama 70b Instruct has downloadable open weights; the other is API-only.

Side by side

Codellama 70b InstructGPT-3.5-turbo
ProviderMetaOpenAI
Noometry Index33.723.2
Released—2023-03-01
WeightsOpenProprietary
Context window—16K
Max output—4K
Input $ / M tokens—$0.50
Output $ / M tokens—$1.50
Results tracked744

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Codellama 70b Instruct leads

Codellama 70b Instruct: 37.6 (#193), GPT-3.5-turbo: 23.9 (#331)

BenchmarkCodellama 70b InstructGPT-3.5-turbo
BigCodeBench Instruct40.7%39.1%
BigCodeBench Complete49.6%50.6%
HumanEval+65.9%70.7%
WeirdML—3.5%
LMArena Coding—1136
MBPP+—69.7%

Agentic & Tool Use Not comparable

Codellama 70b Instruct: —, GPT-3.5-turbo: —

BenchmarkCodellama 70b InstructGPT-3.5-turbo
METR Time Horizons—21.5%

Reasoning Codellama 70b Instruct leads

Codellama 70b Instruct: 20.1 (#242), GPT-3.5-turbo: 13.8 (#332)

BenchmarkCodellama 70b InstructGPT-3.5-turbo
LMArena Hard Prompts10521108
Chess Puzzles—0%
Mystery Game Puzzles—3%
DTBench—48.5%
LMCA—9.7%
Adversarial NLI—58.1%
BIG-Bench Hard—61.6%
CommonsenseQA 2.0—57%
Epoch Capabilities Index—118.55
ForecastBench—50.4
WinoGrande—81.6%

Math Not comparable

Codellama 70b Instruct: —, GPT-3.5-turbo: 6.3 (#327)

BenchmarkCodellama 70b InstructGPT-3.5-turbo
FrontierMath (Tiers 1-3)—0%
OTIS Mock AIME 2024-2025—2.2%
LMArena Math—1142
MATH Level 5—15.9%
GSM8K—57.8%

Knowledge Not comparable

Codellama 70b Instruct: —, GPT-3.5-turbo: 10.0 (#303)

BenchmarkCodellama 70b InstructGPT-3.5-turbo
GPQA Diamond—28%
LMArena Expert—1070
ARC (AI2) Challenge—87.4%
BoolQ—87%
MMLU—71.4%
OpenBookQA—86%
TriviaQA—85.8%

Multilingual GPT-3.5-turbo leads

Codellama 70b Instruct: 24.8 (#288), GPT-3.5-turbo: 31.5 (#258)

BenchmarkCodellama 70b InstructGPT-3.5-turbo
LMArena Non-English9921108
LMArena Chinese—1075
LMArena French—1118
LMArena German—1090
LMArena Japanese—1043
LMArena Korean—1019
LMArena Russian—1123
LMArena Spanish—1121

Instruction Following GPT-3.5-turbo leads

Codellama 70b Instruct: 51.9 (#293), GPT-3.5-turbo: 57.9 (#262)

BenchmarkCodellama 70b InstructGPT-3.5-turbo
LMArena Instruction Following10241119

Long Context Not comparable

Codellama 70b Instruct: —, GPT-3.5-turbo: 34.0 (#254)

BenchmarkCodellama 70b InstructGPT-3.5-turbo
LMArena Longer Query—1121

Writing & Preference Codellama 70b Instruct leads

Codellama 70b Instruct: 33.4 (#277), GPT-3.5-turbo: 25.3 (#305)

BenchmarkCodellama 70b InstructGPT-3.5-turbo
LMArena Text10571125
LMArena Creative Writing—1092
EQ-Bench Creative Writing—451
LMArena Multi-Turn—1117

Frequently asked questions

Is Codellama 70b Instruct better than GPT-3.5-turbo?

Codellama 70b Instruct is the stronger model overall, scoring 33.7 to 23.2 on the Noometry Index.

Is Codellama 70b Instruct or GPT-3.5-turbo better for coding?

Codellama 70b Instruct scores higher on coding benchmarks: 37.6 versus 23.9 in the Noometry coding category.

How many benchmarks do Codellama 70b Instruct and GPT-3.5-turbo share?

7 benchmarks have published results for both models. Codellama 70b Instruct has 7 scored results on Noometry and GPT-3.5-turbo has 44.

Related comparisons

Go deeper