Noometry

Model comparison

Falcon-180B vs Qwen1.5-32B

Falcon-180B is the stronger model overall, scoring 31.8 to 30.0 on the Noometry Index.

Last verified . 7 shared benchmarks.

Falcon-180B Technology Innovation Institute

31.8

Rank #254 Confirmed

Qwen1.5-32B Alibaba (Qwen)

30.0

Rank #288 Confirmed

Summary

  • They share 7 benchmarks with published results for both. Falcon-180B scores higher in 0 categories and Qwen1.5-32B in 4 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where Qwen1.5-32B leads 31.6 to 25.3.

Side by side

Falcon-180BQwen1.5-32B
ProviderTechnology Innovation InstituteAlibaba (Qwen)
Noometry Index31.830.0
Released2023-09-062024-02-04
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1621

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Falcon-180B: —, Qwen1.5-32B: 31.1 (#278)

BenchmarkFalcon-180BQwen1.5-32B
BigCodeBench Instruct—32.3%
LMArena Coding—1155
BigCodeBench Complete—42%

Reasoning Qwen1.5-32B leads

Falcon-180B: 18.3 (#266), Qwen1.5-32B: 20.9 (#210)

BenchmarkFalcon-180BQwen1.5-32B
LMArena Hard Prompts10071131
Epoch Capabilities Index111.89—
HellaSwag89%—
LAMBADA79.8%—
PIQA84.9%—
WinoGrande87.1%—

Math Not comparable

Falcon-180B: —, Qwen1.5-32B: 30.5 (#207)

BenchmarkFalcon-180BQwen1.5-32B
LMArena Math—1155
GSM8K54.4%—

Knowledge Not comparable

Falcon-180B: —, Qwen1.5-32B: 13.5 (#291)

BenchmarkFalcon-180BQwen1.5-32B
MMLU70.6%74.4%
GPQA Diamond—30.7%
LMArena Expert—1126
ARC (AI2) Challenge67.8%—
BoolQ89%—
OpenBookQA64.2%—

Multilingual Qwen1.5-32B leads

Falcon-180B: 25.3 (#281), Qwen1.5-32B: 31.6 (#254)

BenchmarkFalcon-180BQwen1.5-32B
LMArena Non-English10001106
LMArena Chinese—1176
LMArena French—1100
LMArena German—1058
LMArena Japanese—1027
LMArena Korean—1007
LMArena Russian—1073
LMArena Spanish—1089

Instruction Following Qwen1.5-32B leads

Falcon-180B: 53.6 (#281), Qwen1.5-32B: 58.0 (#260)

BenchmarkFalcon-180BQwen1.5-32B
LMArena Instruction Following10471116

Long Context Not comparable

Falcon-180B: —, Qwen1.5-32B: 34.7 (#241)

BenchmarkFalcon-180BQwen1.5-32B
LMArena Longer Query—1146

Writing & Preference Qwen1.5-32B leads

Falcon-180B: 29.3 (#290), Qwen1.5-32B: 34.4 (#266)

BenchmarkFalcon-180BQwen1.5-32B
LMArena Text10541137
LMArena Creative Writing10891083
LMArena Multi-Turn10131140

Frequently asked questions

Is Falcon-180B better than Qwen1.5-32B?

Falcon-180B is the stronger model overall, scoring 31.8 to 30.0 on the Noometry Index.

How many benchmarks do Falcon-180B and Qwen1.5-32B share?

7 benchmarks have published results for both models. Falcon-180B has 16 scored results on Noometry and Qwen1.5-32B has 21.

Related comparisons

Go deeper