Noometry

Model comparison

Codestral vs Llama 3.1-70B

Codestral and Llama 3.1-70B score almost the same on the Noometry Index (30.6 vs 29.6), so choose on price, context window or the category you care about most.

Last verified . 2 shared benchmarks.

Codestral Mistral AI

30.6

Rank #290 Reported

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Summary

  • They share 2 benchmarks with published results for both. Codestral scores higher in 0 categories and Llama 3.1-70B in 2 categories; 2 gaps are clear of the uncertainty.
  • Llama 3.1-70B is cheaper at $0.40 / $0.40 per million input/output tokens, against $0.30 / $0.90 for Codestral.
  • Codestral accepts more context: 256K tokens versus 128K.
  • Llama 3.1-70B has downloadable open weights; the other is API-only.

Side by side

CodestralLlama 3.1-70B
ProviderMistral AIMeta
Noometry Index30.629.6
Released2024-05-292024-07-23
WeightsProprietaryOpen
Context window256K128K
Max output8K4K
Input $ / M tokens$0.30$0.40
Output $ / M tokens$0.90$0.40
Results tracked735

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3.1-70B leads

Codestral: 27.3 (#321), Llama 3.1-70B: 30.3 (#296)

BenchmarkCodestralLlama 3.1-70B
BigCodeBench Instruct41.8%46.1%
BigCodeBench Complete52.5%54.8%
Aider Polyglot11.1%—
WeirdML—9%
LMArena Coding—1260
ALE-Bench137.78—
HumanEval+73.8%—
MBPP+61.9%—

Agentic & Tool Use Not comparable

Codestral: —, Llama 3.1-70B: 25.1 (#112)

BenchmarkCodestralLlama 3.1-70B
TheAgentCompany—6.9%
BALROG—27.9%

Reasoning Llama 3.1-70B leads

Codestral: 19.8 (#251), Llama 3.1-70B: 21.6 (#220)

BenchmarkCodestralLlama 3.1-70B
Kagi LLM Benchmark32.5%—
LMArena Hard Prompts—1241
DTBench—60%
LMCA—14.8%
Epoch Capabilities Index—125.92

Math Not comparable

Codestral: —, Llama 3.1-70B: 13.5 (#304)

BenchmarkCodestralLlama 3.1-70B
OTIS Mock AIME 2024-2025—3.6%
Omni-MATH—21%
LMArena Math—1252
MATH Level 5—36.7%

Knowledge Not comparable

Codestral: —, Llama 3.1-70B: 24.2 (#269)

BenchmarkCodestralLlama 3.1-70B
GPQA Diamond—44.2%
MMLU-Pro—65.3%
GPQA (HELM)—42.6%
LMArena Expert—1209
MMLU—80.1%

Multilingual Not comparable

Codestral: —, Llama 3.1-70B: 38.8 (#225)

BenchmarkCodestralLlama 3.1-70B
LMArena Non-English—1219
LMArena Chinese—1215
LMArena French—1261
LMArena German—1222
LMArena Japanese—1132
LMArena Korean—1140
LMArena Russian—1234
LMArena Spanish—1253

Instruction Following Not comparable

Codestral: —, Llama 3.1-70B: 65.3 (#223)

BenchmarkCodestralLlama 3.1-70B
IFEval—82.1%
LMArena Instruction Following—1231

Long Context Not comparable

Codestral: —, Llama 3.1-70B: 37.6 (#214)

BenchmarkCodestralLlama 3.1-70B
LMArena Longer Query—1241

Writing & Preference Not comparable

Codestral: —, Llama 3.1-70B: 35.4 (#267)

BenchmarkCodestralLlama 3.1-70B
LMArena Text—1261
LMArena Creative Writing—1232
EQ-Bench Creative Writing—784
WildBench—75.8%
LMArena Multi-Turn—1256

Frequently asked questions

Is Codestral better than Llama 3.1-70B?

Codestral and Llama 3.1-70B score almost the same on the Noometry Index (30.6 vs 29.6), so choose on price, context window or the category you care about most.

Which is cheaper, Codestral or Llama 3.1-70B?

Llama 3.1-70B is cheaper. It lists at $0.40 per million input tokens and $0.40 per million output tokens; Codestral lists at $0.30 and $0.90.

Is Codestral or Llama 3.1-70B better for coding?

Llama 3.1-70B scores higher on coding benchmarks: 30.3 versus 27.3 in the Noometry coding category.

Which has the bigger context window?

Codestral does, with 256K tokens against 128K.

How many benchmarks do Codestral and Llama 3.1-70B share?

2 benchmarks have published results for both models. Codestral has 7 scored results on Noometry and Llama 3.1-70B has 35.

Related comparisons

Go deeper