Noometry

Model comparison

ERNIE 5.1 vs Gemini 2.5 Flash

ERNIE 5.1 is the stronger model overall, scoring 43.8 to 39.3 on the Noometry Index.

Last verified . 17 shared benchmarks.

ERNIE 5.1 Baidu

43.8

Rank #83 Confirmed

Gemini 2.5 Flash Google

39.3

Rank #170 Confirmed

Summary

  • They share 17 benchmarks with published results for both. ERNIE 5.1 scores higher in 7 categories and Gemini 2.5 Flash in 1 category; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where ERNIE 5.1 leads 65.1 to 53.8.

Side by side

ERNIE 5.1Gemini 2.5 Flash
ProviderBaiduGoogle
Noometry Index43.839.3
Released—2025-04-17
WeightsProprietaryProprietary
Context window—1.05M
Max output—66K
Input $ / M tokens—$0.30
Output $ / M tokens—$2.50
Results tracked1954

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding ERNIE 5.1 leads

ERNIE 5.1: 44.0 (#76), Gemini 2.5 Flash: 35.8 (#220)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Coding14881424
SWE-bench Verified (bash only)—28.7%
Aider Polyglot—55.1%
WeirdML—41.9%
ALE-Bench—661.88

Agentic & Tool Use Not comparable

ERNIE 5.1: —, Gemini 2.5 Flash: 30.8 (#74)

BenchmarkERNIE 5.1Gemini 2.5 Flash
Terminal-Bench—17.1%
Berkeley Function Calling Leaderboard—56.2%
TheAgentCompany—41.1%
BALROG—33.5%
LMArena Search1227—
Vending-Bench 2—548.84

Reasoning ERNIE 5.1 leads

ERNIE 5.1: 21.9 (#211), Gemini 2.5 Flash: 18.1 (#286)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Hard Prompts14811422
ARC-AGI-2—2.5%
SimpleBench—41.2%
Kagi LLM Benchmark—56.8%
NYT Connections (extended)23.4%—
ARC-AGI-1—33.3%
CritPt—1.1%
EnigmaEval—2.7%
DTBench—76.5%
LMCA—27.5%
Epoch Capabilities Index—143.03
ForecastBench—60.6

Math Too close to call

ERNIE 5.1: 40.3 (#92), Gemini 2.5 Flash: 39.9 (#98)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Math14811415
OTIS Mock AIME 2024-2025—73.1%
Omni-MATH—38.5%
FrontierMath (Feb 2025 set)—4.8%
FrontierMath Tier 4 (v1)—4.2%

Knowledge ERNIE 5.1 leads

ERNIE 5.1: 41.9 (#102), Gemini 2.5 Flash: 36.4 (#168)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Expert14921426
Humanity's Last Exam—12.1%
MMLU-Pro—63.9%
Confabulations—16.8%
Vectara Hallucination Rate—7.8%
GPQA (HELM)—39%

Multimodal Not comparable

ERNIE 5.1: —, Gemini 2.5 Flash: 41.8 (#32)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Vision—1253
GeoBench—76%
VPCT—46.2%
SpatialViz-Bench—36.9%

Multilingual ERNIE 5.1 leads

ERNIE 5.1: 55.5 (#29), Gemini 2.5 Flash: 52.3 (#88)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Non-English14541409
LMArena Chinese15081450
LMArena French14881433
LMArena German14701418
LMArena Japanese14221405
LMArena Korean14271385
LMArena Russian14591415
LMArena Spanish14731421

Instruction Following Too close to call

ERNIE 5.1: 76.7 (#37), Gemini 2.5 Flash: 75.7 (#54)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Instruction Following14601405
IFEval—89.8%

Long Context Gemini 2.5 Flash leads

ERNIE 5.1: 44.7 (#59), Gemini 2.5 Flash: 47.5 (#17)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Longer Query14621419
Fiction.LiveBench—77.8%

Writing & Preference ERNIE 5.1 leads

ERNIE 5.1: 65.1 (#52), Gemini 2.5 Flash: 53.8 (#157)

BenchmarkERNIE 5.1Gemini 2.5 Flash
LMArena Text14681417
LMArena Creative Writing14411400
LMArena Multi-Turn14711408
Short-Story Creative Writing—76.5%
EQ-Bench Creative Writing—1137
WildBench—81.7%

Frequently asked questions

Is ERNIE 5.1 better than Gemini 2.5 Flash?

ERNIE 5.1 is the stronger model overall, scoring 43.8 to 39.3 on the Noometry Index.

Is ERNIE 5.1 or Gemini 2.5 Flash better for coding?

ERNIE 5.1 scores higher on coding benchmarks: 44.0 versus 35.8 in the Noometry coding category.

How many benchmarks do ERNIE 5.1 and Gemini 2.5 Flash share?

17 benchmarks have published results for both models. ERNIE 5.1 has 19 scored results on Noometry and Gemini 2.5 Flash has 54.

Related comparisons

Go deeper