Noometry

Model comparison

ERNIE 5.0 0110 vs o3-mini

ERNIE 5.0 0110 is the stronger model overall, scoring 41.8 to 36.7 on the Noometry Index.

Last verified . 17 shared benchmarks.

ERNIE 5.0 0110 Baidu

41.8

Rank #129 Confirmed

o3-mini OpenAI

36.7

Rank #212 Confirmed

Summary

  • They share 17 benchmarks with published results for both. ERNIE 5.0 0110 scores higher in 7 categories and o3-mini in 1 category; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where ERNIE 5.0 0110 leads 63.1 to 50.3.

Side by side

ERNIE 5.0 0110o3-mini
ProviderBaiduOpenAI
Noometry Index41.836.7
Released—2024-12-20
WeightsProprietaryProprietary
Context window—200K
Max output—100K
Input $ / M tokens—$1.10
Output $ / M tokens—$4.40
Results tracked2051

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 43.0 (#94), o3-mini: 40.8 (#132)

BenchmarkERNIE 5.0 0110o3-mini
LMArena Coding14551378
Aider Polyglot—60.4%
SciCode—39.8%
GSO—1.3%
WeirdML—43.7%
LiveBench Coding—82.7%
CadEval—54%

Agentic & Tool Use Not comparable

ERNIE 5.0 0110: —, o3-mini: 29.6 (#84)

BenchmarkERNIE 5.0 0110o3-mini
Cybench—22.5%

Reasoning Too close to call

ERNIE 5.0 0110: 17.0 (#297), o3-mini: 16.3 (#305)

BenchmarkERNIE 5.0 0110o3-mini
LMArena Hard Prompts14451366
ARC-AGI-2—3%
SimpleBench—22.8%
NYT Connections (extended)10.3%—
ARC-AGI-1—34.5%
CritPt—0.3%
Chess Puzzles—17%
Thematic Generalization41.7%—
LiveBench Reasoning—89.6%
Mystery Game Puzzles—7%
DTBench—68.8%
LiveBench Data Analysis—70.6%
LMCA—19%
Epoch Capabilities Index—140.34
ForecastBench—59.6
LiveBench—75.9%

Math ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 39.3 (#110), o3-mini: 28.1 (#244)

BenchmarkERNIE 5.0 0110o3-mini
LMArena Math14371396
FrontierMath (Tiers 1-3)—18.6%
FrontierMath Tier 4—0%
OTIS Mock AIME 2024-2025—76.9%
LiveBench Math—77.3%
MATH Level 5—96.5%
FrontierMath (Feb 2025 set)—12.4%
FrontierMath Tier 4 (v1)—4.2%

Knowledge ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 39.8 (#128), o3-mini: 38.3 (#146)

BenchmarkERNIE 5.0 0110o3-mini
LMArena Expert14281364
GPQA Diamond—77%
SimpleQA Verified—15.3%
Confabulations—17.9%

Multimodal Not comparable

ERNIE 5.0 0110: 39.9 (#53), o3-mini: —

BenchmarkERNIE 5.0 0110o3-mini
LMArena Vision1249—

Multilingual ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 54.1 (#49), o3-mini: 45.7 (#164)

BenchmarkERNIE 5.0 0110o3-mini
LMArena Non-English14361319
LMArena Chinese15121379
LMArena French14671334
LMArena German14601303
LMArena Japanese13821286
LMArena Korean14061314
LMArena Russian14461304
LMArena Spanish14731321

Instruction Following Too close to call

ERNIE 5.0 0110: 74.5 (#92), o3-mini: 75.1 (#72)

BenchmarkERNIE 5.0 0110o3-mini
LMArena Instruction Following14131337
LiveBench Instruction Following—84.4%

Long Context ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 43.4 (#95), o3-mini: 33.8 (#256)

BenchmarkERNIE 5.0 0110o3-mini
LMArena Longer Query14221343
Fiction.LiveBench—50%

Writing & Preference ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 63.1 (#66), o3-mini: 50.3 (#182)

BenchmarkERNIE 5.0 0110o3-mini
LMArena Text14451337
LMArena Creative Writing14261286
LMArena Multi-Turn14341320
Short-Story Creative Writing—61.7%
LiveBench Language—50.7%

Frequently asked questions

Is ERNIE 5.0 0110 better than o3-mini?

ERNIE 5.0 0110 is the stronger model overall, scoring 41.8 to 36.7 on the Noometry Index.

Is ERNIE 5.0 0110 or o3-mini better for coding?

ERNIE 5.0 0110 scores higher on coding benchmarks: 43.0 versus 40.8 in the Noometry coding category.

How many benchmarks do ERNIE 5.0 0110 and o3-mini share?

17 benchmarks have published results for both models. ERNIE 5.0 0110 has 20 scored results on Noometry and o3-mini has 51.

Related comparisons

Go deeper