Noometry

Model comparison

ERNIE 5.1 vs Kimi K2 (Jul 2025)

ERNIE 5.1 is the stronger model overall, scoring 43.8 to 41.2 on the Noometry Index.

Last verified . 17 shared benchmarks.

ERNIE 5.1 Baidu

43.8

Rank #83 Confirmed

Kimi K2 (Jul 2025) Moonshot AI

41.2

Rank #140 Confirmed

Summary

  • They share 17 benchmarks with published results for both. ERNIE 5.1 scores higher in 6 categories and Kimi K2 (Jul 2025) in 2 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in multilingual, where ERNIE 5.1 leads 55.5 to 49.6.
  • Kimi K2 (Jul 2025) has downloadable open weights; the other is API-only.

Side by side

ERNIE 5.1Kimi K2 (Jul 2025)
ProviderBaiduMoonshot AI
Noometry Index43.841.2
Released—2025-07-12
WeightsProprietaryOpen
Context window—262K
Max output—262K
Input $ / M tokens—$0.57
Output $ / M tokens—$2.30
Results tracked1942

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding ERNIE 5.1 leads

ERNIE 5.1: 44.0 (#76), Kimi K2 (Jul 2025): 42.4 (#102)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
LMArena Coding14881399
SWE-bench Verified (bash only)—63.4%
Aider Polyglot—59.1%
GSO—4.9%
WeirdML—42.8%
ALE-Bench—597.5

Agentic & Tool Use Not comparable

ERNIE 5.1: —, Kimi K2 (Jul 2025): 32.4 (#64)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
Terminal-Bench—35.7%
Berkeley Function Calling Leaderboard—59.1%
LMArena Search1227—
METR Time Horizons—59.2%

Reasoning Kimi K2 (Jul 2025) leads

ERNIE 5.1: 21.9 (#211), Kimi K2 (Jul 2025): 23.3 (#179)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
LMArena Hard Prompts14811384
SimpleBench—26.3%
Kagi LLM Benchmark—64.4%
NYT Connections (extended)23.4%—
Epoch Capabilities Index—146.01
ForecastBench—60.2

Math Kimi K2 (Jul 2025) leads

ERNIE 5.1: 40.3 (#92), Kimi K2 (Jul 2025): 42.7 (#83)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
LMArena Math14811397
Omni-MATH—65.4%
FrontierMath (Feb 2025 set)—21.4%
FrontierMath Tier 4 (v1)—0%

Knowledge ERNIE 5.1 leads

ERNIE 5.1: 41.9 (#102), Kimi K2 (Jul 2025): 37.3 (#157)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
LMArena Expert14921365
MMLU-Pro—81.9%
Confabulations—20.4%
Vectara Hallucination Rate—17.9%
GPQA (HELM)—65.3%

Multilingual ERNIE 5.1 leads

ERNIE 5.1: 55.5 (#29), Kimi K2 (Jul 2025): 49.6 (#130)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
LMArena Non-English14541372
LMArena Chinese15081415
LMArena French14881379
LMArena German14701387
LMArena Japanese14221349
LMArena Korean14271325
LMArena Russian14591385
LMArena Spanish14731386

Instruction Following ERNIE 5.1 leads

ERNIE 5.1: 76.7 (#37), Kimi K2 (Jul 2025): 71.1 (#156)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
LMArena Instruction Following14601348
IFEval—85%

Long Context ERNIE 5.1 leads

ERNIE 5.1: 44.7 (#59), Kimi K2 (Jul 2025): 41.2 (#145)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
LMArena Longer Query14621353
Fiction.LiveBench—66.7%
CL-bench—17.6%

Writing & Preference ERNIE 5.1 leads

ERNIE 5.1: 65.1 (#52), Kimi K2 (Jul 2025): 62.3 (#78)

BenchmarkERNIE 5.1Kimi K2 (Jul 2025)
LMArena Text14681380
LMArena Creative Writing14411350
LMArena Multi-Turn14711371
Short-Story Creative Writing—85.6%
EQ-Bench Creative Writing—1666
WildBench—86.2%

Frequently asked questions

Is ERNIE 5.1 better than Kimi K2 (Jul 2025)?

ERNIE 5.1 is the stronger model overall, scoring 43.8 to 41.2 on the Noometry Index.

Is ERNIE 5.1 or Kimi K2 (Jul 2025) better for coding?

ERNIE 5.1 scores higher on coding benchmarks: 44.0 versus 42.4 in the Noometry coding category.

How many benchmarks do ERNIE 5.1 and Kimi K2 (Jul 2025) share?

17 benchmarks have published results for both models. ERNIE 5.1 has 19 scored results on Noometry and Kimi K2 (Jul 2025) has 42.

Related comparisons

Go deeper