Noometry

Model comparison

C4ai Aya Expanse 32b vs Falcon-180B

C4ai Aya Expanse 32b is the stronger model overall, scoring 35.9 to 32.2 on the Noometry Index.

Last verified . 6 shared benchmarks.

C4ai Aya Expanse 32b Cohere

35.9

Rank #221 Confirmed

Falcon-180B Technology Innovation Institute

32.2

Rank #260 Confirmed

Summary

  • They share 6 benchmarks with published results for both. C4ai Aya Expanse 32b scores higher in 4 categories and Falcon-180B in 0 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where C4ai Aya Expanse 32b leads 38.4 to 25.2.

Side by side

C4ai Aya Expanse 32bFalcon-180B
ProviderCohereTechnology Innovation Institute
Noometry Index35.932.2
Released2024-10-242023-09-06
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens——
Output $ / M tokens——
Results tracked1816

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

C4ai Aya Expanse 32b: 34.8 (#231), Falcon-180B: —

BenchmarkC4ai Aya Expanse 32bFalcon-180B
LMArena Coding1197—

Reasoning C4ai Aya Expanse 32b leads

C4ai Aya Expanse 32b: 23.3 (#180), Falcon-180B: 19.1 (#269)

BenchmarkC4ai Aya Expanse 32bFalcon-180B
LMArena Hard Prompts11931007
Epoch Capabilities Index—112.13
HellaSwag—89%
LAMBADA—79.8%
PIQA—84.9%
WinoGrande—87.1%

Math Not comparable

C4ai Aya Expanse 32b: 34.0 (#197), Falcon-180B: —

BenchmarkC4ai Aya Expanse 32bFalcon-180B
LMArena Math1200—
GSM8K—54.4%

Knowledge Not comparable

C4ai Aya Expanse 32b: 33.2 (#206), Falcon-180B: —

BenchmarkC4ai Aya Expanse 32bFalcon-180B
Vectara Hallucination Rate10.9%—
LMArena Expert1182—
ARC (AI2) Challenge—67.8%
BoolQ—89%
MMLU—70.6%
OpenBookQA—64.2%

Multilingual C4ai Aya Expanse 32b leads

C4ai Aya Expanse 32b: 38.4 (#230), Falcon-180B: 25.2 (#286)

BenchmarkC4ai Aya Expanse 32bFalcon-180B
LMArena Non-English12131000
LMArena Chinese1211—
LMArena French1248—
LMArena German1199—
LMArena Japanese1163—
LMArena Korean1158—
LMArena Russian1227—
LMArena Spanish1193—

Instruction Following C4ai Aya Expanse 32b leads

C4ai Aya Expanse 32b: 62.6 (#237), Falcon-180B: 53.4 (#286)

BenchmarkC4ai Aya Expanse 32bFalcon-180B
LMArena Instruction Following11961047

Long Context Not comparable

C4ai Aya Expanse 32b: 37.2 (#220), Falcon-180B: —

BenchmarkC4ai Aya Expanse 32bFalcon-180B
LMArena Longer Query1228—

Writing & Preference C4ai Aya Expanse 32b leads

C4ai Aya Expanse 32b: 42.2 (#235), Falcon-180B: 29.1 (#295)

BenchmarkC4ai Aya Expanse 32bFalcon-180B
LMArena Text12241054
LMArena Creative Writing12001089
LMArena Multi-Turn11901013

Frequently asked questions

Is C4ai Aya Expanse 32b better than Falcon-180B?

C4ai Aya Expanse 32b is the stronger model overall, scoring 35.9 to 32.2 on the Noometry Index.

How many benchmarks do C4ai Aya Expanse 32b and Falcon-180B share?

6 benchmarks have published results for both models. C4ai Aya Expanse 32b has 18 scored results on Noometry and Falcon-180B has 16.

Related comparisons

Go deeper