Model comparison
Longcat Flash Chat vs o4-mini
Longcat Flash Chat and o4-mini score almost the same on the Noometry Index (42.1 vs 41.6), so choose on price, context window or the category you care about most.
Last verified . 18 shared benchmarks.
Summary
- They share 18 benchmarks with published results for both. Longcat Flash Chat scores higher in 3 categories and o4-mini in 5 categories; 7 gaps are clear of the uncertainty.
- The widest gap is in writing & preference, where Longcat Flash Chat leads 61.0 to 54.0.
- The biggest single-benchmark swing is Kagi LLM Benchmark: 43.9% for Longcat Flash Chat and 67.6% for o4-mini.
- Longcat Flash Chat has downloadable open weights; the other is API-only.
Side by side
| Longcat Flash Chat | o4-mini | |
|---|---|---|
| Provider | Meituan | OpenAI |
| Noometry Index | 42.1 | 41.6 |
| Released | — | 2025-04-16 |
| Weights | Open | Proprietary |
| Context window | — | 200K |
| Max output | — | 100K |
| Input $ / M tokens | — | $1.10 |
| Output $ / M tokens | — | $4.40 |
| Results tracked | 19 | 60 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding Longcat Flash Chat leads
Longcat Flash Chat: 43.5 (#87), o4-mini: 40.9 (#127)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| LMArena Coding | 1471 | 1368 |
| SWE-bench Verified (bash only) | — | 45% |
| Aider Polyglot | — | 72% |
| GSO | — | 3.6% |
| WeirdML | — | 52.6% |
| CadEval | — | 62% |
| ALE-Bench | — | 826.17 |
| AlgoTune | — | 1.72 |
Agentic & Tool Use Not comparable
Longcat Flash Chat: —, o4-mini: 32.6 (#61)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| Berkeley Function Calling Leaderboard | — | 53.2% |
| GDPval | — | 25.3% |
| METR Time Horizons | — | 63.9% |
Reasoning o4-mini leads
Longcat Flash Chat: 19.0 (#272), o4-mini: 24.6 (#162)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| Kagi LLM Benchmark | 43.9% | 67.6% |
| LMArena Hard Prompts | 1440 | 1351 |
| ARC-AGI-2 | — | 6.1% |
| SimpleBench | — | 38.7% |
| NYT Connections (extended) | 17.7% | — |
| ARC-AGI-1 | — | 58.7% |
| CritPt | — | 0.6% |
| Chess Puzzles | — | 26% |
| EnigmaEval | — | 9.2% |
| Mystery Game Puzzles | — | 5% |
| DTBench | — | 77.6% |
| LMCA | — | 26.5% |
| Epoch Capabilities Index | — | 145.64 |
| ForecastBench | — | 61.8 |
Math o4-mini leads
Longcat Flash Chat: 39.4 (#107), o4-mini: 40.8 (#89)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| LMArena Math | 1442 | 1389 |
| FrontierMath (Tiers 1-3) | — | 36.1% |
| FrontierMath Tier 4 | — | 4.9% |
| OTIS Mock AIME 2024-2025 | — | 81.7% |
| Omni-MATH | — | 72% |
| MATH Level 5 | — | 97.8% |
| FrontierMath (Feb 2025 set) | — | 24.8% |
| FrontierMath Tier 4 (v1) | — | 6.3% |
Knowledge o4-mini leads
Longcat Flash Chat: 40.6 (#116), o4-mini: 43.6 (#91)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| LMArena Expert | 1454 | 1343 |
| GPQA Diamond | — | 79.6% |
| Humanity's Last Exam | — | 18.1% |
| SimpleQA Verified | — | 19.6% |
| MMLU-Pro | — | 82% |
| Confabulations | — | 15.8% |
| Vectara Hallucination Rate | — | 18.6% |
| GPQA (HELM) | — | 73.5% |
Multimodal Not comparable
Longcat Flash Chat: —, o4-mini: 40.2 (#49)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| LMArena Vision | — | 1194 |
| GeoBench | — | 64% |
| VPCT | — | 57.5% |
Multilingual Longcat Flash Chat leads
Longcat Flash Chat: 51.9 (#101), o4-mini: 47.0 (#154)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| LMArena Non-English | 1404 | 1337 |
| LMArena Chinese | 1465 | 1354 |
| LMArena French | 1456 | 1364 |
| LMArena German | 1408 | 1336 |
| LMArena Japanese | 1373 | 1308 |
| LMArena Korean | 1371 | 1312 |
| LMArena Russian | 1395 | 1334 |
| LMArena Spanish | 1445 | 1347 |
Instruction Following Too close to call
Longcat Flash Chat: 74.4 (#96), o4-mini: 75.2 (#68)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| LMArena Instruction Following | 1411 | 1321 |
| IFEval | — | 92.8% |
Long Context o4-mini leads
Longcat Flash Chat: 43.5 (#93), o4-mini: 45.5 (#33)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| LMArena Longer Query | 1425 | 1315 |
| Fiction.LiveBench | — | 77.8% |
Writing & Preference Longcat Flash Chat leads
Longcat Flash Chat: 61.0 (#91), o4-mini: 54.0 (#152)
| Benchmark | Longcat Flash Chat | o4-mini |
|---|---|---|
| LMArena Text | 1427 | 1353 |
| LMArena Creative Writing | 1388 | 1294 |
| LMArena Multi-Turn | 1418 | 1350 |
| Short-Story Creative Writing | — | 75% |
| WildBench | — | 85.4% |
Frequently asked questions
Is Longcat Flash Chat better than o4-mini?
Longcat Flash Chat and o4-mini score almost the same on the Noometry Index (42.1 vs 41.6), so choose on price, context window or the category you care about most.
Is Longcat Flash Chat or o4-mini better for coding?
Longcat Flash Chat scores higher on coding benchmarks: 43.5 versus 40.9 in the Noometry coding category.
How many benchmarks do Longcat Flash Chat and o4-mini share?
18 benchmarks have published results for both models. Longcat Flash Chat has 19 scored results on Noometry and o4-mini has 60.