Model comparison

Falcon-7B vs Mistral 7B

Mistral 7B has enough public results to be ranked (#351); Falcon-7B does not yet, so treat this comparison as directional.

Last verified . 11 shared benchmarks.

Summary

  • They share 11 benchmarks with published results for both.

Side by side

Falcon-7B and Mistral 7B specifications
Falcon-7BMistral 7B
ProviderTechnology Innovation InstituteMistral AI
Noometry Index—23.0
Released2023-04-242023-09-27
WeightsOpenOpen
Context window—8K
Max output—8K
Input $ / M tokens—$0.25
Output $ / M tokens—$0.25
Results tracked1237

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Falcon-7B: —, Mistral 7B: 26.4 (#326)

Coding benchmarks
BenchmarkFalcon-7BMistral 7B
BigCodeBench Instruct—19.5%
LMArena Coding—1082
BigCodeBench Complete—27.3%
HumanEval+—36%
MBPP+—42.1%

Reasoning Not comparable

Falcon-7B: —, Mistral 7B: 13.1 (#336)

Reasoning benchmarks
BenchmarkFalcon-7BMistral 7B
BIG-Bench Hard28.8%56.1%
Epoch Capabilities Index95.1112.21
HellaSwag78.1%81%
PIQA80.3%83%
WinoGrande67.2%75.3%
Chess Puzzles—0%
LMArena Hard Prompts—1067
DTBench—42.5%
Adversarial NLI—47.1%
LAMBADA74.9%—

Math Not comparable

Falcon-7B: —, Mistral 7B: 8.1 (#325)

Math benchmarks
BenchmarkFalcon-7BMistral 7B
GSM8K6.8%54.4%
OTIS Mock AIME 2024-2025—0.3%
LMArena Math—1085
MATH Level 5—3.7%

Knowledge Not comparable

Falcon-7B: —, Mistral 7B: 7.4 (#311)

Knowledge benchmarks
BenchmarkFalcon-7BMistral 7B
ARC (AI2) Challenge47.9%78.6%
BoolQ75.3%87.4%
MMLU35%62.5%
OpenBookQA51.6%79.8%
TriviaQA64.6%75.2%
GPQA Diamond—15.2%
LMArena Expert—1036

Multilingual Not comparable

Falcon-7B: —, Mistral 7B: 25.8 (#283)

Multilingual benchmarks
BenchmarkFalcon-7BMistral 7B
LMArena Non-English—1012
LMArena Chinese—1009
LMArena French—1037
LMArena German—987
LMArena Japanese—878
LMArena Russian—1018
LMArena Spanish—1026

Instruction Following Not comparable

Falcon-7B: —, Mistral 7B: 54.2 (#280)

Instruction Following benchmarks
BenchmarkFalcon-7BMistral 7B
LMArena Instruction Following—1060

Long Context Not comparable

Falcon-7B: —, Mistral 7B: 32.2 (#271)

Long Context benchmarks
BenchmarkFalcon-7BMistral 7B
LMArena Longer Query—1060

Writing & Preference Not comparable

Falcon-7B: —, Mistral 7B: 30.7 (#286)

Writing & Preference benchmarks
BenchmarkFalcon-7BMistral 7B
LMArena Text—1090
LMArena Creative Writing—1068
LMArena Multi-Turn—1062

Frequently asked questions

Is Falcon-7B better than Mistral 7B?

Mistral 7B has enough public results to be ranked (#351); Falcon-7B does not yet, so treat this comparison as directional.

How many benchmarks do Falcon-7B and Mistral 7B share?

11 benchmarks have published results for both models. Falcon-7B has 12 scored results on Noometry and Mistral 7B has 37.

Related comparisons

Go deeper