Noometry

Model comparison

C4ai Aya Expanse 8b vs Grok-2 (Dec 2024)

C4ai Aya Expanse 8b is the stronger model overall, scoring 34.9 to 33.7 on the Noometry Index.

Last verified . 14 shared benchmarks.

C4ai Aya Expanse 8b Cohere

34.9

Rank #229 Confirmed

Grok-2 (Dec 2024) xAI

33.7

Rank #239 Confirmed

Summary

  • They share 14 benchmarks with published results for both. C4ai Aya Expanse 8b scores higher in 4 categories and Grok-2 (Dec 2024) in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where C4ai Aya Expanse 8b leads 33.3 to 20.8.
  • C4ai Aya Expanse 8b has downloadable open weights; the other is API-only.

Side by side

C4ai Aya Expanse 8bGrok-2 (Dec 2024)
ProviderCoherexAI
Noometry Index34.933.7
Released—2024-08-13
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1534

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

C4ai Aya Expanse 8b: 33.8 (#252), Grok-2 (Dec 2024): 33.3 (#258)

BenchmarkC4ai Aya Expanse 8bGrok-2 (Dec 2024)
LMArena Coding11601287
WeirdML—22.2%
LiveBench Coding—46.4%

Reasoning C4ai Aya Expanse 8b leads

C4ai Aya Expanse 8b: 22.4 (#197), Grok-2 (Dec 2024): 16.9 (#299)

BenchmarkC4ai Aya Expanse 8bGrok-2 (Dec 2024)
LMArena Hard Prompts11561272
SimpleBench—22.7%
LiveBench Reasoning—54.8%
DTBench—65.2%
LiveBench Data Analysis—54.5%
Epoch Capabilities Index—130.48
LiveBench—54.3%

Math C4ai Aya Expanse 8b leads

C4ai Aya Expanse 8b: 33.3 (#203), Grok-2 (Dec 2024): 20.8 (#284)

BenchmarkC4ai Aya Expanse 8bGrok-2 (Dec 2024)
LMArena Math11681283
OTIS Mock AIME 2024-2025—11.5%
LiveBench Math—54.9%
MATH Level 5—63.5%
FrontierMath (Feb 2025 set)—0.7%

Knowledge C4ai Aya Expanse 8b leads

C4ai Aya Expanse 8b: 33.6 (#201), Grok-2 (Dec 2024): 29.8 (#233)

BenchmarkC4ai Aya Expanse 8bGrok-2 (Dec 2024)
LMArena Expert11531254
GPQA Diamond—53.8%
Confabulations—20.1%
Vectara Hallucination Rate9.5%—

Multilingual Grok-2 (Dec 2024) leads

C4ai Aya Expanse 8b: 36.1 (#242), Grok-2 (Dec 2024): 43.1 (#188)

BenchmarkC4ai Aya Expanse 8bGrok-2 (Dec 2024)
LMArena Non-English11801282
LMArena Chinese11811289
LMArena German11881287
LMArena Japanese11201244
LMArena Russian11971286
LMArena French—1318
LMArena Korean—1237
LMArena Spanish—1281

Instruction Following Grok-2 (Dec 2024) leads

C4ai Aya Expanse 8b: 60.1 (#253), Grok-2 (Dec 2024): 66.9 (#202)

BenchmarkC4ai Aya Expanse 8bGrok-2 (Dec 2024)
LMArena Instruction Following11551270
LiveBench Instruction Following—69.6%

Long Context Grok-2 (Dec 2024) leads

C4ai Aya Expanse 8b: 36.1 (#236), Grok-2 (Dec 2024): 38.8 (#190)

BenchmarkC4ai Aya Expanse 8bGrok-2 (Dec 2024)
LMArena Longer Query11911276

Writing & Preference Grok-2 (Dec 2024) leads

C4ai Aya Expanse 8b: 39.0 (#250), Grok-2 (Dec 2024): 48.6 (#198)

BenchmarkC4ai Aya Expanse 8bGrok-2 (Dec 2024)
LMArena Text11851305
LMArena Creative Writing11681284
LMArena Multi-Turn11601290
Short-Story Creative Writing—63.6%
LiveBench Language—45.6%

Frequently asked questions

Is C4ai Aya Expanse 8b better than Grok-2 (Dec 2024)?

C4ai Aya Expanse 8b is the stronger model overall, scoring 34.9 to 33.7 on the Noometry Index.

Is C4ai Aya Expanse 8b or Grok-2 (Dec 2024) better for coding?

They score almost the same on coding (33.8 vs 33.3); test both on your own repository before choosing.

How many benchmarks do C4ai Aya Expanse 8b and Grok-2 (Dec 2024) share?

14 benchmarks have published results for both models. C4ai Aya Expanse 8b has 15 scored results on Noometry and Grok-2 (Dec 2024) has 34.

Related comparisons

Go deeper