Noometry

Model comparison

GPT-4o mini vs Kimi K2.5

Kimi K2.5 is the stronger model overall, scoring 48.1 to 25.5 on the Noometry Index. GPT-4o mini costs 3.4× less per token, which makes it the better buy when Kimi K2.5's lead doesn't matter for your workload.

Last verified . 28 shared benchmarks.

GPT-4o mini OpenAI

25.5

Rank #343 Confirmed

Kimi K2.5 Moonshot AI

48.1

Rank #57 Confirmed

Summary

  • They share 28 benchmarks with published results for both. GPT-4o mini scores higher in 0 categories and Kimi K2.5 in 10 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in math, where Kimi K2.5 leads 51.8 to 10.4.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 6.9% for GPT-4o mini and 92.2% for Kimi K2.5.
  • GPT-4o mini is cheaper at $0.15 / $0.60 per million input/output tokens, against $0.45 / $2.25 for Kimi K2.5.
  • Kimi K2.5 accepts more context: 262K tokens versus 128K.
  • Kimi K2.5 has downloadable open weights; the other is API-only.

Side by side

GPT-4o miniKimi K2.5
ProviderOpenAIMoonshot AI
Noometry Index25.548.1
Released2024-07-182026-01-27
WeightsProprietaryOpen
Context window128K262K
Max output16K262K
Input $ / M tokens$0.15$0.45
Output $ / M tokens$0.60$2.25
Results tracked6051

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Kimi K2.5 leads

GPT-4o mini: 22.0 (#335), Kimi K2.5: 48.8 (#53)

BenchmarkGPT-4o miniKimi K2.5
WeirdML11.8%45.6%
LMArena Coding12901474
SWE-bench Verified—73.8%
SWE-bench Verified (bash only)—70.8%
Aider Polyglot3.6%—
LMArena WebDev—1437
SWE-bench Multilingual—67.3%
SciCode—49%
BigCodeBench Instruct46.1%—
LiveBench Coding43.1%—
BigCodeBench Complete57.4%—
ALE-Bench—821.65
HumanEval+83.5%—
MBPP+72.2%—

Agentic & Tool Use Kimi K2.5 leads

GPT-4o mini: 27.5 (#101), Kimi K2.5: 34.2 (#48)

BenchmarkGPT-4o miniKimi K2.5
Terminal-Bench—43.2%
OSWorld—63.3%
BALROG17.4%—
Vending-Bench 2—1,198

Reasoning Kimi K2.5 leads

GPT-4o mini: 8.7 (#347), Kimi K2.5: 31.2 (#80)

BenchmarkGPT-4o miniKimi K2.5
ARC-AGI-20%11.8%
SimpleBench10.7%46.8%
Kagi LLM Benchmark28.8%78.5%
Chess Puzzles0%12%
LMArena Hard Prompts12671453
Epoch Capabilities Index126.56148.03
NYT Connections (extended)—69.9%
ARC-AGI-1—65.3%
CritPt—3.1%
EnigmaEval—3.4%
Thematic Generalization—69.4%
LiveBench Reasoning32.8%—
Mystery Game Puzzles12%—
DTBench54.4%—
LiveBench Data Analysis50%—
LMCA10.4%—
LiveBench41.3%—
PIQA88.7%—

Math Kimi K2.5 leads

GPT-4o mini: 10.4 (#314), Kimi K2.5: 51.8 (#53)

BenchmarkGPT-4o miniKimi K2.5
OTIS Mock AIME 2024-20256.9%92.2%
LMArena Math12671470
FrontierMath (Tiers 1-3)0.7%—
MathArena Final-Answer Competitions—62.3%
Omni-MATH28%—
LiveBench Math36.3%—
MATH Level 552.6%—
FrontierMath (Feb 2025 set)—27.9%
FrontierMath Tier 4 (v1)—4.2%
GSM8K91.3%—

Knowledge Kimi K2.5 leads

GPT-4o mini: 17.7 (#284), Kimi K2.5: 53.6 (#56)

BenchmarkGPT-4o miniKimi K2.5
GPQA Diamond37.7%87.6%
SimpleQA Verified8.3%34.3%
LMArena Expert12351466
Humanity's Last Exam—24.4%
MMLU-Pro60.3%—
Confabulations37.2%—
Vectara Hallucination Rate—14.2%
GPQA (HELM)36.8%—
BoolQ88.7%—
MMLU81.8%—

Multimodal Kimi K2.5 leads

GPT-4o mini: 25.9 (#122), Kimi K2.5: 41.1 (#39)

BenchmarkGPT-4o miniKimi K2.5
LMArena Vision10661269
Video-MME64.8%—
GeoBench64%—
VPCT34%—
LMArena Document—1430

Multilingual Kimi K2.5 leads

GPT-4o mini: 42.0 (#199), Kimi K2.5: 53.9 (#53)

BenchmarkGPT-4o miniKimi K2.5
LMArena Non-English12661433
LMArena Chinese12651495
LMArena French12971454
LMArena German12721441
LMArena Japanese12161421
LMArena Korean11951410
LMArena Russian12751435
LMArena Spanish12761450

Instruction Following Kimi K2.5 leads

GPT-4o mini: 61.9 (#239), Kimi K2.5: 75.3 (#64)

BenchmarkGPT-4o miniKimi K2.5
LMArena Instruction Following12581431
LiveBench Instruction Following56.8%—
IFEval78.2%—

Long Context Kimi K2.5 leads

GPT-4o mini: 39.1 (#186), Kimi K2.5: 52.1 (#7)

BenchmarkGPT-4o miniKimi K2.5
LMArena Longer Query12891445
Fiction.LiveBench—86.1%
CL-bench—19.3%
CL-bench Life—13.2%

Writing & Preference Kimi K2.5 leads

GPT-4o mini: 39.5 (#248), Kimi K2.5: 65.1 (#53)

BenchmarkGPT-4o miniKimi K2.5
LMArena Text12861445
LMArena Creative Writing12681423
EQ-Bench Creative Writing8731579
LMArena Multi-Turn12851444
Short-Story Creative Writing67.2%—
WildBench79.1%—
LiveBench Language28.6%—

Frequently asked questions

Is GPT-4o mini better than Kimi K2.5?

Kimi K2.5 is the stronger model overall, scoring 48.1 to 25.5 on the Noometry Index. GPT-4o mini costs 3.4× less per token, which makes it the better buy when Kimi K2.5's lead doesn't matter for your workload.

Which is cheaper, GPT-4o mini or Kimi K2.5?

GPT-4o mini is cheaper. It lists at $0.15 per million input tokens and $0.60 per million output tokens; Kimi K2.5 lists at $0.45 and $2.25.

Is GPT-4o mini or Kimi K2.5 better for coding?

Kimi K2.5 scores higher on coding benchmarks: 48.8 versus 22.0 in the Noometry coding category.

Which has the bigger context window?

Kimi K2.5 does, with 262K tokens against 128K.

How many benchmarks do GPT-4o mini and Kimi K2.5 share?

28 benchmarks have published results for both models. GPT-4o mini has 60 scored results on Noometry and Kimi K2.5 has 51.

Related comparisons

Go deeper