Noometry

Model comparison

GPT-4 Turbo vs Longcat Flash Chat

Longcat Flash Chat is the stronger model overall, scoring 42.1 to 30.5 on the Noometry Index.

Last verified . 17 shared benchmarks.

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Longcat Flash Chat Meituan

42.1

Rank #120 Confirmed

Summary

  • They share 17 benchmarks with published results for both. GPT-4 Turbo scores higher in 0 categories and Longcat Flash Chat in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Longcat Flash Chat leads 39.4 to 9.0.
  • Longcat Flash Chat has downloadable open weights; the other is API-only.

Side by side

GPT-4 TurboLongcat Flash Chat
ProviderOpenAIMeituan
Noometry Index30.542.1
Released2023-11-06—
WeightsProprietaryOpen
Context window128K—
Max output4K—
Input $ / M tokens$10—
Output $ / M tokens$30—
Results tracked3619

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Longcat Flash Chat leads

GPT-4 Turbo: 33.8 (#249), Longcat Flash Chat: 43.5 (#87)

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Coding12681471
WeirdML18%—
BigCodeBench Instruct48.2%—
BigCodeBench Complete58.2%—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Longcat Flash Chat: —

BenchmarkGPT-4 TurboLongcat Flash Chat
METR Time Horizons36.7%—

Reasoning Longcat Flash Chat leads

GPT-4 Turbo: 15.3 (#317), Longcat Flash Chat: 19.0 (#272)

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Hard Prompts12511440
SimpleBench25.1%—
Kagi LLM Benchmark—43.9%
NYT Connections (extended)—17.7%
Chess Puzzles6%—
DTBench61.6%—
LMCA9.8%—
Epoch Capabilities Index127.25—
ForecastBench59.4—

Math Longcat Flash Chat leads

GPT-4 Turbo: 9.0 (#322), Longcat Flash Chat: 39.4 (#107)

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Math12721442
FrontierMath (Tiers 1-3)0.7%—
OTIS Mock AIME 2024-20256.7%—
MATH Level 546.7%—

Knowledge Longcat Flash Chat leads

GPT-4 Turbo: 24.3 (#268), Longcat Flash Chat: 40.6 (#116)

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Expert12231454
GPQA Diamond46.6%—
Confabulations28.4%—
MMLU81.3%—

Multimodal Not comparable

GPT-4 Turbo: 30.6 (#110), Longcat Flash Chat: —

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Vision1090—

Multilingual Longcat Flash Chat leads

GPT-4 Turbo: 40.5 (#216), Longcat Flash Chat: 51.9 (#101)

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Non-English12451404
LMArena Chinese12421465
LMArena French12761456
LMArena German12591408
LMArena Japanese11941373
LMArena Korean11871371
LMArena Russian12591395
LMArena Spanish12601445

Instruction Following Longcat Flash Chat leads

GPT-4 Turbo: 65.8 (#216), Longcat Flash Chat: 74.4 (#96)

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Instruction Following12491411

Long Context Longcat Flash Chat leads

GPT-4 Turbo: 38.0 (#206), Longcat Flash Chat: 43.5 (#93)

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Longer Query12541425

Writing & Preference Longcat Flash Chat leads

GPT-4 Turbo: 47.7 (#206), Longcat Flash Chat: 61.0 (#91)

BenchmarkGPT-4 TurboLongcat Flash Chat
LMArena Text12721427
LMArena Creative Writing12691388
LMArena Multi-Turn12671418

Frequently asked questions

Is GPT-4 Turbo better than Longcat Flash Chat?

Longcat Flash Chat is the stronger model overall, scoring 42.1 to 30.5 on the Noometry Index.

Is GPT-4 Turbo or Longcat Flash Chat better for coding?

Longcat Flash Chat scores higher on coding benchmarks: 43.5 versus 33.8 in the Noometry coding category.

How many benchmarks do GPT-4 Turbo and Longcat Flash Chat share?

17 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Longcat Flash Chat has 19.

Related comparisons

Go deeper