Noometry

Model comparison

Claude 3 Opus vs Longcat Flash Chat

Longcat Flash Chat is the stronger model overall, scoring 41.5 to 29.3 on the Noometry Index.

Last verified . 17 shared benchmarks.

Claude 3 Opus Anthropic

29.3

Rank #300 Confirmed

Longcat Flash Chat Meituan

41.5

Rank #121 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Claude 3 Opus scores higher in 0 categories and Longcat Flash Chat in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Longcat Flash Chat leads 36.6 to 13.9.
  • Longcat Flash Chat has downloadable open weights; the other is API-only.

Side by side

Claude 3 OpusLongcat Flash Chat
ProviderAnthropicMeituan
Noometry Index29.341.5
Released2024-02-29—
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4619

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Longcat Flash Chat leads

Claude 3 Opus: 32.4 (#261), Longcat Flash Chat: 42.9 (#85)

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Coding12641471
WeirdML19.2%—
BigCodeBench Instruct45.5%—
LiveBench Coding38.6%—
BigCodeBench Complete57.4%—
HumanEval+77.4%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

Claude 3 Opus: 24.3 (#110), Longcat Flash Chat: —

BenchmarkClaude 3 OpusLongcat Flash Chat
Cybench10%—
METR Time Horizons29.5%—

Reasoning Longcat Flash Chat leads

Claude 3 Opus: 14.2 (#318), Longcat Flash Chat: 18.3 (#267)

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Hard Prompts12461440
SimpleBench23.5%—
Kagi LLM Benchmark—43.9%
NYT Connections (extended)—17.7%
Chess Puzzles5%—
EnigmaEval0.8%—
LiveBench Reasoning40.6%—
DTBench61.6%—
LiveBench Data Analysis57.9%—
LMCA17%—
Epoch Capabilities Index126.91—
ForecastBench58.4—
LiveBench49.2%—
WinoGrande88.5%—

Math Longcat Flash Chat leads

Claude 3 Opus: 13.9 (#293), Longcat Flash Chat: 36.6 (#107)

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Math12731441
OTIS Mock AIME 2024-20254.7%—
LiveBench Math43.6%—
MATH Level 537.5%—

Knowledge Longcat Flash Chat leads

Claude 3 Opus: 24.5 (#262), Longcat Flash Chat: 40.0 (#111)

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Expert12231454
GPQA Diamond47.2%—
SimpleQA Verified12.6%—
Confabulations22.7%—
MMLU84.6%—

Multimodal Not comparable

Claude 3 Opus: 26.6 (#112), Longcat Flash Chat: —

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Vision1023—

Multilingual Longcat Flash Chat leads

Claude 3 Opus: 41.6 (#202), Longcat Flash Chat: 52.1 (#95)

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Non-English12581404
LMArena Chinese12481465
LMArena French12741457
LMArena German12581409
LMArena Japanese12041373
LMArena Korean11871372
LMArena Russian12801396
LMArena Spanish12461445

Instruction Following Longcat Flash Chat leads

Claude 3 Opus: 64.3 (#223), Longcat Flash Chat: 74.6 (#92)

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Instruction Following12481411
LiveBench Instruction Following63.9%—

Long Context Longcat Flash Chat leads

Claude 3 Opus: 38.2 (#197), Longcat Flash Chat: 43.5 (#89)

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Longer Query12591425

Writing & Preference Longcat Flash Chat leads

Claude 3 Opus: 47.4 (#208), Longcat Flash Chat: 61.2 (#88)

BenchmarkClaude 3 OpusLongcat Flash Chat
LMArena Text12621427
LMArena Creative Writing12351388
LMArena Multi-Turn12751418
LiveBench Language50.4%—

Frequently asked questions

Is Claude 3 Opus better than Longcat Flash Chat?

Longcat Flash Chat is the stronger model overall, scoring 41.5 to 29.3 on the Noometry Index.

Is Claude 3 Opus or Longcat Flash Chat better for coding?

Longcat Flash Chat scores higher on coding benchmarks: 42.9 versus 32.4 in the Noometry coding category.

How many benchmarks do Claude 3 Opus and Longcat Flash Chat share?

17 benchmarks have published results for both models. Claude 3 Opus has 46 scored results on Noometry and Longcat Flash Chat has 19.

Related comparisons

Go deeper