Noometry

Model comparison

Falcon-180B vs Qwen1.5-110B

Qwen1.5-110B is the stronger model overall, scoring 34.2 to 32.2 on the Noometry Index.

Last verified . 6 shared benchmarks.

Falcon-180B Technology Innovation Institute

32.2

Rank #260 Confirmed

Qwen1.5-110B Alibaba (Qwen)

34.2

Rank #234 Confirmed

Summary

  • They share 6 benchmarks with published results for both. Falcon-180B scores higher in 0 categories and Qwen1.5-110B in 4 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen1.5-110B leads 38.0 to 29.1.

Side by side

Falcon-180BQwen1.5-110B
ProviderTechnology Innovation InstituteAlibaba (Qwen)
Noometry Index32.234.2
Released2023-09-062024-04-25
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1620

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Falcon-180B: —, Qwen1.5-110B: 33.0 (#264)

BenchmarkFalcon-180BQwen1.5-110B
BigCodeBench Instruct—35%
LMArena Coding—1184
BigCodeBench Complete—44.4%

Reasoning Qwen1.5-110B leads

Falcon-180B: 19.1 (#269), Qwen1.5-110B: 22.7 (#189)

BenchmarkFalcon-180BQwen1.5-110B
LMArena Hard Prompts10071168
Epoch Capabilities Index112.13—
ForecastBench—57.7
HellaSwag89%—
LAMBADA79.8%—
PIQA84.9%—
WinoGrande87.1%—

Math Not comparable

Falcon-180B: —, Qwen1.5-110B: 33.7 (#201)

BenchmarkFalcon-180BQwen1.5-110B
LMArena Math—1185
GSM8K54.4%—

Knowledge Not comparable

Falcon-180B: —, Qwen1.5-110B: 31.2 (#219)

BenchmarkFalcon-180BQwen1.5-110B
LMArena Expert—1144
ARC (AI2) Challenge67.8%—
BoolQ89%—
MMLU70.6%—
OpenBookQA64.2%—

Multilingual Qwen1.5-110B leads

Falcon-180B: 25.2 (#286), Qwen1.5-110B: 33.6 (#250)

BenchmarkFalcon-180BQwen1.5-110B
LMArena Non-English10001142
LMArena Chinese—1206
LMArena French—1151
LMArena German—1123
LMArena Japanese—1074
LMArena Korean—1044
LMArena Russian—1118
LMArena Spanish—1142

Instruction Following Qwen1.5-110B leads

Falcon-180B: 53.4 (#286), Qwen1.5-110B: 60.3 (#252)

BenchmarkFalcon-180BQwen1.5-110B
LMArena Instruction Following10471158

Long Context Not comparable

Falcon-180B: —, Qwen1.5-110B: 35.1 (#242)

BenchmarkFalcon-180BQwen1.5-110B
LMArena Longer Query—1157

Writing & Preference Qwen1.5-110B leads

Falcon-180B: 29.1 (#295), Qwen1.5-110B: 38.0 (#255)

BenchmarkFalcon-180BQwen1.5-110B
LMArena Text10541175
LMArena Creative Writing10891148
LMArena Multi-Turn10131160

Frequently asked questions

Is Falcon-180B better than Qwen1.5-110B?

Qwen1.5-110B is the stronger model overall, scoring 34.2 to 32.2 on the Noometry Index.

How many benchmarks do Falcon-180B and Qwen1.5-110B share?

6 benchmarks have published results for both models. Falcon-180B has 16 scored results on Noometry and Qwen1.5-110B has 20.

Related comparisons

Go deeper