Noometry

Model comparison

GPT-4o mini vs Kimi K2 (Jul 2025)

Kimi K2 (Jul 2025) is the stronger model overall, scoring 40.9 to 25.4 on the Noometry Index. GPT-4o mini costs 3.8× less per token, which makes it the better buy when Kimi K2 (Jul 2025)'s lead doesn't matter for your workload.

Last verified . 30 shared benchmarks.

GPT-4o mini OpenAI

25.4

Rank #337 Confirmed

Kimi K2 (Jul 2025) Moonshot AI

40.9

Rank #134 Confirmed

Summary

  • They share 30 benchmarks with published results for both. GPT-4o mini scores higher in 0 categories and Kimi K2 (Jul 2025) in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where Kimi K2 (Jul 2025) leads 40.1 to 9.9.
  • The biggest single-benchmark swing is Aider Polyglot: 3.6% for GPT-4o mini and 59.1% for Kimi K2 (Jul 2025).
  • GPT-4o mini is cheaper at $0.15 / $0.60 per million input/output tokens, against $0.57 / $2.30 for Kimi K2 (Jul 2025).
  • Kimi K2 (Jul 2025) accepts more context: 262K tokens versus 128K.
  • Kimi K2 (Jul 2025) has downloadable open weights; the other is API-only.

Side by side

GPT-4o miniKimi K2 (Jul 2025)
ProviderOpenAIMoonshot AI
Noometry Index25.440.9
Released2024-07-182025-07-12
WeightsProprietaryOpen
Context window128K262K
Max output16K262K
Input $ / M tokens$0.15$0.57
Output $ / M tokens$0.60$2.30
Results tracked6042

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Kimi K2 (Jul 2025) leads

GPT-4o mini: 21.6 (#330), Kimi K2 (Jul 2025): 43.1 (#83)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
Aider Polyglot3.6%59.1%
WeirdML11.8%42.8%
LMArena Coding12901399
SWE-bench Verified (bash only)—63.4%
GSO—4.9%
BigCodeBench Instruct46.1%—
LiveBench Coding43.1%—
BigCodeBench Complete57.4%—
ALE-Bench—597.5
HumanEval+83.5%—
MBPP+72.2%—

Agentic & Tool Use Kimi K2 (Jul 2025) leads

GPT-4o mini: 27.2 (#95), Kimi K2 (Jul 2025): 32.2 (#59)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
Terminal-Bench—35.7%
Berkeley Function Calling Leaderboard—59.1%
BALROG17.4%—
METR Time Horizons—59.2%

Reasoning Kimi K2 (Jul 2025) leads

GPT-4o mini: 8.7 (#343), Kimi K2 (Jul 2025): 22.7 (#173)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
SimpleBench10.7%26.3%
Kagi LLM Benchmark28.8%64.4%
LMArena Hard Prompts12671384
Epoch Capabilities Index126.56146.02
ARC-AGI-20%—
Chess Puzzles0%—
LiveBench Reasoning32.8%—
Mystery Game Puzzles12%—
DTBench54.4%—
LiveBench Data Analysis50%—
LMCA10.4%—
ForecastBench—60.2
LiveBench41.3%—
PIQA88.7%—

Math Kimi K2 (Jul 2025) leads

GPT-4o mini: 9.9 (#307), Kimi K2 (Jul 2025): 40.1 (#81)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
Omni-MATH28%65.4%
LMArena Math12671396
FrontierMath (Tiers 1-3)0.7%—
OTIS Mock AIME 2024-20256.9%—
LiveBench Math36.3%—
MATH Level 552.6%—
FrontierMath (Feb 2025 set)—21.4%
FrontierMath Tier 4 (v1)—0%
GSM8K91.3%—

Knowledge Kimi K2 (Jul 2025) leads

GPT-4o mini: 17.7 (#279), Kimi K2 (Jul 2025): 36.9 (#153)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
MMLU-Pro60.3%81.9%
Confabulations37.2%20.4%
GPQA (HELM)36.8%65.3%
LMArena Expert12351363
GPQA Diamond37.7%—
SimpleQA Verified8.3%—
Vectara Hallucination Rate—17.9%
BoolQ88.7%—
MMLU81.8%—

Multimodal Not comparable

GPT-4o mini: 25.3 (#117), Kimi K2 (Jul 2025): —

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
LMArena Vision1066—
Video-MME64.8%—
GeoBench64%—
VPCT34%—

Multilingual Kimi K2 (Jul 2025) leads

GPT-4o mini: 42.2 (#194), Kimi K2 (Jul 2025): 49.8 (#125)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
LMArena Non-English12661372
LMArena Chinese12641416
LMArena French12961379
LMArena German12721386
LMArena Japanese12161349
LMArena Korean11951324
LMArena Russian12751384
LMArena Spanish12761383

Instruction Following Kimi K2 (Jul 2025) leads

GPT-4o mini: 62.1 (#234), Kimi K2 (Jul 2025): 71.3 (#151)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
IFEval78.2%85%
LMArena Instruction Following12591347
LiveBench Instruction Following56.8%—

Long Context Kimi K2 (Jul 2025) leads

GPT-4o mini: 39.1 (#181), Kimi K2 (Jul 2025): 41.2 (#140)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
LMArena Longer Query12891353
Fiction.LiveBench—66.7%
CL-bench—17.6%

Writing & Preference Kimi K2 (Jul 2025) leads

GPT-4o mini: 39.7 (#243), Kimi K2 (Jul 2025): 62.5 (#76)

BenchmarkGPT-4o miniKimi K2 (Jul 2025)
LMArena Text12861380
LMArena Creative Writing12681349
Short-Story Creative Writing67.2%85.6%
EQ-Bench Creative Writing8731666
WildBench79.1%86.2%
LMArena Multi-Turn12851371
LiveBench Language28.6%—

Frequently asked questions

Is GPT-4o mini better than Kimi K2 (Jul 2025)?

Kimi K2 (Jul 2025) is the stronger model overall, scoring 40.9 to 25.4 on the Noometry Index. GPT-4o mini costs 3.8× less per token, which makes it the better buy when Kimi K2 (Jul 2025)'s lead doesn't matter for your workload.

Which is cheaper, GPT-4o mini or Kimi K2 (Jul 2025)?

GPT-4o mini is cheaper. It lists at $0.15 per million input tokens and $0.60 per million output tokens; Kimi K2 (Jul 2025) lists at $0.57 and $2.30.

Is GPT-4o mini or Kimi K2 (Jul 2025) better for coding?

Kimi K2 (Jul 2025) scores higher on coding benchmarks: 43.1 versus 21.6 in the Noometry coding category.

Which has the bigger context window?

Kimi K2 (Jul 2025) does, with 262K tokens against 128K.

How many benchmarks do GPT-4o mini and Kimi K2 (Jul 2025) share?

30 benchmarks have published results for both models. GPT-4o mini has 60 scored results on Noometry and Kimi K2 (Jul 2025) has 42.

Related comparisons

Go deeper