Model comparison

Falcon-7B vs Mixtral 8x22B

Mixtral 8x22B has enough public results to be ranked (#333); Falcon-7B does not yet, so treat this comparison as directional.

Last verified . 2 shared benchmarks.

Summary

  • They share 2 benchmarks with published results for both.

Side by side

Falcon-7B and Mixtral 8x22B specifications
Falcon-7BMixtral 8x22B
ProviderTechnology Innovation InstituteMistral AI
Noometry Index—27.1
Released2023-04-242024-04-17
WeightsOpenOpen
Context window—64K
Max output—64K
Input $ / M tokens—$2
Output $ / M tokens—$6
Results tracked1234

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Falcon-7B: —, Mixtral 8x22B: 24.2 (#329)

Coding benchmarks
BenchmarkFalcon-7BMixtral 8x22B
WeirdML—3.2%
BigCodeBench Instruct—40.6%
LMArena Coding—1166
BigCodeBench Complete—50.2%
HumanEval+—72%
MBPP+—64.3%

Agentic & Tool Use Not comparable

Falcon-7B: —, Mixtral 8x22B: 23.1 (#127)

Agentic & Tool Use benchmarks
BenchmarkFalcon-7BMixtral 8x22B
Cybench—7.5%

Reasoning Not comparable

Falcon-7B: —, Mixtral 8x22B: 19.9 (#248)

Reasoning benchmarks
BenchmarkFalcon-7BMixtral 8x22B
Epoch Capabilities Index95.1122.03
LMArena Hard Prompts—1150
DTBench—55.1%
BIG-Bench Hard28.8%—
ForecastBench—56.3
HellaSwag78.1%—
LAMBADA74.9%—
PIQA80.3%—
WinoGrande67.2%—

Math Not comparable

Falcon-7B: —, Mixtral 8x22B: 22.9 (#275)

Math benchmarks
BenchmarkFalcon-7BMixtral 8x22B
Omni-MATH—16.3%
LMArena Math—1184
MATH Level 5—24.2%
GSM8K6.8%—

Knowledge Not comparable

Falcon-7B: —, Mixtral 8x22B: 15.1 (#293)

Knowledge benchmarks
BenchmarkFalcon-7BMixtral 8x22B
MMLU35%77.8%
GPQA Diamond—34.1%
MMLU-Pro—46%
GPQA (HELM)—33.4%
LMArena Expert—1113
ARC (AI2) Challenge47.9%—
BoolQ75.3%—
OpenBookQA51.6%—
TriviaQA64.6%—

Multilingual Not comparable

Falcon-7B: —, Mixtral 8x22B: 32.8 (#255)

Multilingual benchmarks
BenchmarkFalcon-7BMixtral 8x22B
LMArena Non-English—1128
LMArena Chinese—1116
LMArena French—1166
LMArena German—1141
LMArena Japanese—1037
LMArena Korean—1057
LMArena Russian—1158
LMArena Spanish—1151

Instruction Following Not comparable

Falcon-7B: —, Mixtral 8x22B: 57.7 (#266)

Instruction Following benchmarks
BenchmarkFalcon-7BMixtral 8x22B
IFEval—72.4%
LMArena Instruction Following—1147

Long Context Not comparable

Falcon-7B: —, Mixtral 8x22B: 34.7 (#247)

Long Context benchmarks
BenchmarkFalcon-7BMixtral 8x22B
LMArena Longer Query—1144

Writing & Preference Not comparable

Falcon-7B: —, Mixtral 8x22B: 36.9 (#262)

Writing & Preference benchmarks
BenchmarkFalcon-7BMixtral 8x22B
LMArena Text—1162
LMArena Creative Writing—1141
WildBench—71.1%
LMArena Multi-Turn—1130

Frequently asked questions

Is Falcon-7B better than Mixtral 8x22B?

Mixtral 8x22B has enough public results to be ranked (#333); Falcon-7B does not yet, so treat this comparison as directional.

How many benchmarks do Falcon-7B and Mixtral 8x22B share?

2 benchmarks have published results for both models. Falcon-7B has 12 scored results on Noometry and Mixtral 8x22B has 34.

Related comparisons

Go deeper