Noometry

Model comparison

ERNIE 5.0 0110 vs GPT-4o mini

ERNIE 5.0 0110 is the stronger model overall, scoring 41.8 to 25.5 on the Noometry Index.

Last verified . 18 shared benchmarks.

ERNIE 5.0 0110 Baidu

41.8

Rank #129 Confirmed

GPT-4o mini OpenAI

25.5

Rank #343 Confirmed

Summary

  • They share 18 benchmarks with published results for both. ERNIE 5.0 0110 scores higher in 9 categories and GPT-4o mini in 0 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where ERNIE 5.0 0110 leads 39.3 to 10.4.

Side by side

ERNIE 5.0 0110GPT-4o mini
ProviderBaiduOpenAI
Noometry Index41.825.5
Released—2024-07-18
WeightsProprietaryProprietary
Context window—128K
Max output—16K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.60
Results tracked2060

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 43.0 (#94), GPT-4o mini: 22.0 (#335)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Coding14551290
Aider Polyglot—3.6%
WeirdML—11.8%
BigCodeBench Instruct—46.1%
LiveBench Coding—43.1%
BigCodeBench Complete—57.4%
HumanEval+—83.5%
MBPP+—72.2%

Agentic & Tool Use Not comparable

ERNIE 5.0 0110: —, GPT-4o mini: 27.5 (#101)

BenchmarkERNIE 5.0 0110GPT-4o mini
BALROG—17.4%

Reasoning ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 17.0 (#297), GPT-4o mini: 8.7 (#347)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Hard Prompts14451267
ARC-AGI-2—0%
SimpleBench—10.7%
Kagi LLM Benchmark—28.8%
NYT Connections (extended)10.3%—
Chess Puzzles—0%
Thematic Generalization41.7%—
LiveBench Reasoning—32.8%
Mystery Game Puzzles—12%
DTBench—54.4%
LiveBench Data Analysis—50%
LMCA—10.4%
Epoch Capabilities Index—126.56
LiveBench—41.3%
PIQA—88.7%

Math ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 39.3 (#110), GPT-4o mini: 10.4 (#314)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Math14371267
FrontierMath (Tiers 1-3)—0.7%
OTIS Mock AIME 2024-2025—6.9%
Omni-MATH—28%
LiveBench Math—36.3%
MATH Level 5—52.6%
GSM8K—91.3%

Knowledge ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 39.8 (#128), GPT-4o mini: 17.7 (#284)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Expert14281235
GPQA Diamond—37.7%
SimpleQA Verified—8.3%
MMLU-Pro—60.3%
Confabulations—37.2%
GPQA (HELM)—36.8%
BoolQ—88.7%
MMLU—81.8%

Multimodal ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 39.9 (#53), GPT-4o mini: 25.9 (#122)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Vision12491066
Video-MME—64.8%
GeoBench—64%
VPCT—34%

Multilingual ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 54.1 (#49), GPT-4o mini: 42.0 (#199)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Non-English14361266
LMArena Chinese15121265
LMArena French14671297
LMArena German14601272
LMArena Japanese13821216
LMArena Korean14061195
LMArena Russian14461275
LMArena Spanish14731276

Instruction Following ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 74.5 (#92), GPT-4o mini: 61.9 (#239)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Instruction Following14131258
LiveBench Instruction Following—56.8%
IFEval—78.2%

Long Context ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 43.4 (#95), GPT-4o mini: 39.1 (#186)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Longer Query14221289

Writing & Preference ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 63.1 (#66), GPT-4o mini: 39.5 (#248)

BenchmarkERNIE 5.0 0110GPT-4o mini
LMArena Text14451286
LMArena Creative Writing14261268
LMArena Multi-Turn14341285
Short-Story Creative Writing—67.2%
EQ-Bench Creative Writing—873
WildBench—79.1%
LiveBench Language—28.6%

Frequently asked questions

Is ERNIE 5.0 0110 better than GPT-4o mini?

ERNIE 5.0 0110 is the stronger model overall, scoring 41.8 to 25.5 on the Noometry Index.

Is ERNIE 5.0 0110 or GPT-4o mini better for coding?

ERNIE 5.0 0110 scores higher on coding benchmarks: 43.0 versus 22.0 in the Noometry coding category.

How many benchmarks do ERNIE 5.0 0110 and GPT-4o mini share?

18 benchmarks have published results for both models. ERNIE 5.0 0110 has 20 scored results on Noometry and GPT-4o mini has 60.

Related comparisons

Go deeper