Model comparison

Falcon-7B vs Mistral Medium 3.5

Mistral Medium 3.5 has enough public results to be ranked (#152); Falcon-7B does not yet, so treat this comparison as directional.

Last verified . 1 shared benchmarks.

Summary

  • They share 1 benchmark with published results for both.

Side by side

Falcon-7B and Mistral Medium 3.5 specifications
Falcon-7BMistral Medium 3.5
ProviderTechnology Innovation InstituteMistral AI
Noometry Index—40.2
Released2023-04-24—
WeightsOpenOpen
Context window—262K
Max output—210K
Input $ / M tokens—$1.50
Output $ / M tokens—$7.50
Results tracked1222

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Falcon-7B: —, Mistral Medium 3.5: 36.0 (#213)

Coding benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
LMArena WebDev—1264
LMArena Coding—1461

Reasoning Not comparable

Falcon-7B: —, Mistral Medium 3.5: 17.3 (#295)

Reasoning benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
Epoch Capabilities Index95.1141.35
Kagi LLM Benchmark—41.4%
NYT Connections (extended)—12.9%
LMArena Hard Prompts—1436
BIG-Bench Hard28.8%—
HellaSwag78.1%—
LAMBADA74.9%—
PIQA80.3%—
WinoGrande67.2%—

Math Not comparable

Falcon-7B: —, Mistral Medium 3.5: 39.1 (#113)

Math benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
LMArena Math—1431
GSM8K6.8%—

Knowledge Not comparable

Falcon-7B: —, Mistral Medium 3.5: 40.0 (#126)

Knowledge benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
LMArena Expert—1432
ARC (AI2) Challenge47.9%—
BoolQ75.3%—
MMLU35%—
OpenBookQA51.6%—
TriviaQA64.6%—

Multimodal Not comparable

Falcon-7B: —, Mistral Medium 3.5: 38.3 (#65)

Multimodal benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
LMArena Vision—1223

Multilingual Not comparable

Falcon-7B: —, Mistral Medium 3.5: 51.9 (#100)

Multilingual benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
LMArena Non-English—1404
LMArena Chinese—1442
LMArena French—1448
LMArena German—1451
LMArena Korean—1385
LMArena Russian—1395
LMArena Spanish—1409

Instruction Following Not comparable

Falcon-7B: —, Mistral Medium 3.5: 74.6 (#90)

Instruction Following benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
LMArena Instruction Following—1415

Long Context Not comparable

Falcon-7B: —, Mistral Medium 3.5: 43.2 (#103)

Long Context benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
LMArena Longer Query—1415

Writing & Preference Not comparable

Falcon-7B: —, Mistral Medium 3.5: 58.5 (#117)

Writing & Preference benchmarks
BenchmarkFalcon-7BMistral Medium 3.5
LMArena Text—1421
LMArena Creative Writing—1374
EQ-Bench 4—993
LMArena Multi-Turn—1423

Frequently asked questions

Is Falcon-7B better than Mistral Medium 3.5?

Mistral Medium 3.5 has enough public results to be ranked (#152); Falcon-7B does not yet, so treat this comparison as directional.

How many benchmarks do Falcon-7B and Mistral Medium 3.5 share?

1 benchmark has published results for both models. Falcon-7B has 12 scored results on Noometry and Mistral Medium 3.5 has 22.

Related comparisons

Go deeper