Noometry

Model comparison

ERNIE 5.1 vs Gemini 1.5 Flash 8B

ERNIE 5.1 is the stronger model overall, scoring 43.8 to 29.9 on the Noometry Index.

Last verified . 17 shared benchmarks.

ERNIE 5.1 Baidu

43.8

Rank #83 Confirmed

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

Summary

  • They share 17 benchmarks with published results for both. ERNIE 5.1 scores higher in 8 categories and Gemini 1.5 Flash 8B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where ERNIE 5.1 leads 40.3 to 14.2.

Side by side

ERNIE 5.1Gemini 1.5 Flash 8B
ProviderBaiduGoogle
Noometry Index43.829.9
Released—2024-10-03
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked1921

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding ERNIE 5.1 leads

ERNIE 5.1: 44.0 (#76), Gemini 1.5 Flash 8B: 35.5 (#225)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Coding14881218

Agentic & Tool Use Not comparable

ERNIE 5.1: —, Gemini 1.5 Flash 8B: —

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Search1227—

Reasoning ERNIE 5.1 leads

ERNIE 5.1: 21.9 (#211), Gemini 1.5 Flash 8B: 20.0 (#244)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Hard Prompts14811209
NYT Connections (extended)23.4%—
DTBench—50%

Math ERNIE 5.1 leads

ERNIE 5.1: 40.3 (#92), Gemini 1.5 Flash 8B: 14.2 (#302)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Math14811207
OTIS Mock AIME 2024-2025—4.6%

Knowledge ERNIE 5.1 leads

ERNIE 5.1: 41.9 (#102), Gemini 1.5 Flash 8B: 16.0 (#289)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Expert14921185
GPQA Diamond—33%

Multimodal Not comparable

ERNIE 5.1: —, Gemini 1.5 Flash 8B: 28.2 (#115)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Vision—1044

Multilingual ERNIE 5.1 leads

ERNIE 5.1: 55.5 (#29), Gemini 1.5 Flash 8B: 38.5 (#229)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Non-English14541215
LMArena Chinese15081231
LMArena French14881234
LMArena German14701206
LMArena Japanese14221150
LMArena Korean14271140
LMArena Russian14591236
LMArena Spanish14731212

Instruction Following ERNIE 5.1 leads

ERNIE 5.1: 76.7 (#37), Gemini 1.5 Flash 8B: 62.8 (#236)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Instruction Following14601199

Long Context ERNIE 5.1 leads

ERNIE 5.1: 44.7 (#59), Gemini 1.5 Flash 8B: 37.0 (#225)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Longer Query14621219

Writing & Preference ERNIE 5.1 leads

ERNIE 5.1: 65.1 (#52), Gemini 1.5 Flash 8B: 42.8 (#232)

BenchmarkERNIE 5.1Gemini 1.5 Flash 8B
LMArena Text14681226
LMArena Creative Writing14411218
LMArena Multi-Turn14711185

Frequently asked questions

Is ERNIE 5.1 better than Gemini 1.5 Flash 8B?

ERNIE 5.1 is the stronger model overall, scoring 43.8 to 29.9 on the Noometry Index.

Is ERNIE 5.1 or Gemini 1.5 Flash 8B better for coding?

ERNIE 5.1 scores higher on coding benchmarks: 44.0 versus 35.5 in the Noometry coding category.

How many benchmarks do ERNIE 5.1 and Gemini 1.5 Flash 8B share?

17 benchmarks have published results for both models. ERNIE 5.1 has 19 scored results on Noometry and Gemini 1.5 Flash 8B has 21.

Related comparisons

Go deeper