Model comparison

Falcon-180B vs Mistral Small

Mistral Small is the stronger model overall, scoring 33.4 to 32.2 on the Noometry Index.

Last verified . 7 shared benchmarks.

Mistral Small Mistral AI

33.4

Rank #243 Confirmed

Summary

  • They share 7 benchmarks with published results for both. Falcon-180B scores higher in 0 categories and Mistral Small in 4 categories; 3 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Small leads 52.5 to 29.1.

Side by side

Falcon-180B and Mistral Small specifications
Falcon-180BMistral Small
ProviderTechnology Innovation InstituteMistral AI
Noometry Index32.233.4
Released2023-09-062024-02-26
WeightsOpenOpen
Context window—262K
Max output—256K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.60
Results tracked1639

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Falcon-180B: —, Mistral Small: 34.0 (#247)

Coding benchmarks
BenchmarkFalcon-180BMistral Small
SciCode—26.5%
BigCodeBench Instruct—36.1%
LiveBench Coding—36.2%
LMArena Coding—1362
BigCodeBench Complete—46.6%
ALE-Bench—497.62

Agentic & Tool Use Not comparable

Falcon-180B: —, Mistral Small: 28.1 (#93)

Agentic & Tool Use benchmarks
BenchmarkFalcon-180BMistral Small
Berkeley Function Calling Leaderboard—37.1%

Reasoning Too close to call

Falcon-180B: 19.1 (#269), Mistral Small: 19.8 (#250)

Reasoning benchmarks
BenchmarkFalcon-180BMistral Small
LMArena Hard Prompts10071335
Kagi LLM Benchmark—37.8%
CritPt—0%
LiveBench Reasoning—44.8%
DTBench—70.9%
LiveBench Data Analysis—53.7%
LMCA—20.6%
Epoch Capabilities Index112.13—
HellaSwag89%—
LAMBADA79.8%—
LiveBench—44%
PIQA84.9%—
WinoGrande87.1%—

Math Not comparable

Falcon-180B: —, Mistral Small: 16.4 (#293)

Math benchmarks
BenchmarkFalcon-180BMistral Small
OTIS Mock AIME 2024-2025—5.8%
LiveBench Math—39.9%
LMArena Math—1341
MATH Level 5—46.8%
GSM8K54.4%—

Knowledge Not comparable

Falcon-180B: —, Mistral Small: 31.0 (#222)

Knowledge benchmarks
BenchmarkFalcon-180BMistral Small
MMLU70.6%68.7%
GPQA Diamond—47.5%
Vectara Hallucination Rate—5.1%
LMArena Expert—1291
ARC (AI2) Challenge67.8%—
BoolQ89%—
OpenBookQA64.2%—

Multimodal Not comparable

Falcon-180B: —, Mistral Small: 33.5 (#96)

Multimodal benchmarks
BenchmarkFalcon-180BMistral Small
LMArena Vision—1142

Multilingual Mistral Small leads

Falcon-180B: 25.2 (#286), Mistral Small: 45.5 (#169)

Multilingual benchmarks
BenchmarkFalcon-180BMistral Small
LMArena Non-English10001315
LMArena Chinese—1340
LMArena French—1337
LMArena German—1340
LMArena Japanese—1275
LMArena Korean—1259
LMArena Russian—1324
LMArena Spanish—1346

Instruction Following Mistral Small leads

Falcon-180B: 53.4 (#286), Mistral Small: 66.4 (#209)

Instruction Following benchmarks
BenchmarkFalcon-180BMistral Small
LMArena Instruction Following10471310
LiveBench Instruction Following—63.7%

Long Context Not comparable

Falcon-180B: —, Mistral Small: 40.4 (#156)

Long Context benchmarks
BenchmarkFalcon-180BMistral Small
LMArena Longer Query—1327

Writing & Preference Mistral Small leads

Falcon-180B: 29.1 (#295), Mistral Small: 52.5 (#171)

Writing & Preference benchmarks
BenchmarkFalcon-180BMistral Small
LMArena Text10541338
LMArena Creative Writing10891305
LMArena Multi-Turn10131344
LiveBench Language—30.5%

Frequently asked questions

Is Falcon-180B better than Mistral Small?

Mistral Small is the stronger model overall, scoring 33.4 to 32.2 on the Noometry Index.

How many benchmarks do Falcon-180B and Mistral Small share?

7 benchmarks have published results for both models. Falcon-180B has 16 scored results on Noometry and Mistral Small has 39.

Related comparisons

Go deeper