Noometry

Model comparison

Falcon-180B vs Qwen1.5-72B

Falcon-180B is the stronger model overall, scoring 32.2 to 30.8 on the Noometry Index.

Last verified . 6 shared benchmarks.

Falcon-180B Technology Innovation Institute

32.2

Rank #260 Confirmed

Qwen1.5-72B Alibaba (Qwen)

30.8

Rank #285 Confirmed

Summary

  • They share 6 benchmarks with published results for both. Falcon-180B scores higher in 0 categories and Qwen1.5-72B in 4 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen1.5-72B leads 37.3 to 29.1.

Side by side

Falcon-180BQwen1.5-72B
ProviderTechnology Innovation InstituteAlibaba (Qwen)
Noometry Index32.230.8
Released2023-09-062024-02-04
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1622

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Falcon-180B: —, Qwen1.5-72B: 31.9 (#277)

BenchmarkFalcon-180BQwen1.5-72B
BigCodeBench Instruct—33.2%
LMArena Coding—1165
BigCodeBench Complete—40.3%
HumanEval+—59.1%
MBPP+—61.6%

Reasoning Qwen1.5-72B leads

Falcon-180B: 19.1 (#269), Qwen1.5-72B: 22.2 (#203)

BenchmarkFalcon-180BQwen1.5-72B
LMArena Hard Prompts10071148
Epoch Capabilities Index112.13—
HellaSwag89%—
LAMBADA79.8%—
PIQA84.9%—
WinoGrande87.1%—

Math Not comparable

Falcon-180B: —, Qwen1.5-72B: 33.2 (#205)

BenchmarkFalcon-180BQwen1.5-72B
LMArena Math—1164
GSM8K54.4%—

Knowledge Not comparable

Falcon-180B: —, Qwen1.5-72B: 11.5 (#300)

BenchmarkFalcon-180BQwen1.5-72B
GPQA Diamond—28.8%
LMArena Expert—1136
ARC (AI2) Challenge67.8%—
BoolQ89%—
MMLU70.6%—
OpenBookQA64.2%—

Multilingual Qwen1.5-72B leads

Falcon-180B: 25.2 (#286), Qwen1.5-72B: 33.2 (#253)

BenchmarkFalcon-180BQwen1.5-72B
LMArena Non-English10001135
LMArena Chinese—1186
LMArena French—1159
LMArena German—1084
LMArena Japanese—1061
LMArena Korean—1050
LMArena Russian—1104
LMArena Spanish—1110

Instruction Following Qwen1.5-72B leads

Falcon-180B: 53.4 (#286), Qwen1.5-72B: 59.3 (#256)

BenchmarkFalcon-180BQwen1.5-72B
LMArena Instruction Following10471141

Long Context Not comparable

Falcon-180B: —, Qwen1.5-72B: 35.1 (#243)

BenchmarkFalcon-180BQwen1.5-72B
LMArena Longer Query—1157

Writing & Preference Qwen1.5-72B leads

Falcon-180B: 29.1 (#295), Qwen1.5-72B: 37.3 (#258)

BenchmarkFalcon-180BQwen1.5-72B
LMArena Text10541166
LMArena Creative Writing10891137
LMArena Multi-Turn10131160

Frequently asked questions

Is Falcon-180B better than Qwen1.5-72B?

Falcon-180B is the stronger model overall, scoring 32.2 to 30.8 on the Noometry Index.

How many benchmarks do Falcon-180B and Qwen1.5-72B share?

6 benchmarks have published results for both models. Falcon-180B has 16 scored results on Noometry and Qwen1.5-72B has 22.

Related comparisons

Go deeper