Noometry

Model comparison

Llama2 70b Steerlm Chat vs Mistral Small

Mistral Small is the stronger model overall, scoring 33.4 to 31.8 on the Noometry Index.

Last verified . 9 shared benchmarks.

Llama2 70b Steerlm Chat NVIDIA

31.8

Rank #268 Confirmed

Mistral Small Mistral AI

33.4

Rank #243 Confirmed

Summary

  • They share 9 benchmarks with published results for both. Llama2 70b Steerlm Chat scores higher in 2 categories and Mistral Small in 5 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Small leads 52.5 to 31.6.

Side by side

Llama2 70b Steerlm ChatMistral Small
ProviderNVIDIAMistral AI
Noometry Index31.833.4
Released—2024-02-26
WeightsOpenOpen
Context window—262K
Max output—256K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.60
Results tracked939

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small leads

Llama2 70b Steerlm Chat: 29.9 (#300), Mistral Small: 34.0 (#247)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Coding10251362
SciCode—26.5%
BigCodeBench Instruct—36.1%
LiveBench Coding—36.2%
BigCodeBench Complete—46.6%
ALE-Bench—497.62

Agentic & Tool Use Not comparable

Llama2 70b Steerlm Chat: —, Mistral Small: 28.1 (#93)

BenchmarkLlama2 70b Steerlm ChatMistral Small
Berkeley Function Calling Leaderboard—37.1%

Reasoning Too close to call

Llama2 70b Steerlm Chat: 20.0 (#246), Mistral Small: 19.8 (#250)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Hard Prompts10471335
Kagi LLM Benchmark—37.8%
CritPt—0%
LiveBench Reasoning—44.8%
DTBench—70.9%
LiveBench Data Analysis—53.7%
LMCA—20.6%
LiveBench—44%

Math Llama2 70b Steerlm Chat leads

Llama2 70b Steerlm Chat: 31.3 (#226), Mistral Small: 16.4 (#293)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Math10721341
OTIS Mock AIME 2024-2025—5.8%
LiveBench Math—39.9%
MATH Level 5—46.8%

Knowledge Not comparable

Llama2 70b Steerlm Chat: —, Mistral Small: 31.0 (#222)

BenchmarkLlama2 70b Steerlm ChatMistral Small
GPQA Diamond—47.5%
Vectara Hallucination Rate—5.1%
LMArena Expert—1291
MMLU—68.7%

Multimodal Not comparable

Llama2 70b Steerlm Chat: —, Mistral Small: 33.5 (#96)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Vision—1142

Multilingual Mistral Small leads

Llama2 70b Steerlm Chat: 28.8 (#270), Mistral Small: 45.5 (#169)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Non-English10631315
LMArena Chinese—1340
LMArena French—1337
LMArena German—1340
LMArena Japanese—1275
LMArena Korean—1259
LMArena Russian—1324
LMArena Spanish—1346

Instruction Following Mistral Small leads

Llama2 70b Steerlm Chat: 54.2 (#279), Mistral Small: 66.4 (#209)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Instruction Following10601310
LiveBench Instruction Following—63.7%

Long Context Mistral Small leads

Llama2 70b Steerlm Chat: 30.4 (#288), Mistral Small: 40.4 (#156)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Longer Query9981327

Writing & Preference Mistral Small leads

Llama2 70b Steerlm Chat: 31.6 (#283), Mistral Small: 52.5 (#171)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Text10981338
LMArena Creative Writing10911305
LMArena Multi-Turn10581344
LiveBench Language—30.5%

Frequently asked questions

Is Llama2 70b Steerlm Chat better than Mistral Small?

Mistral Small is the stronger model overall, scoring 33.4 to 31.8 on the Noometry Index.

Is Llama2 70b Steerlm Chat or Mistral Small better for coding?

Mistral Small scores higher on coding benchmarks: 34.0 versus 29.9 in the Noometry coding category.

How many benchmarks do Llama2 70b Steerlm Chat and Mistral Small share?

9 benchmarks have published results for both models. Llama2 70b Steerlm Chat has 9 scored results on Noometry and Mistral Small has 39.

Related comparisons

Go deeper