Noometry

Model comparison

ERNIE 5.0 0110 vs Llama 3.1-8B

ERNIE 5.0 0110 is the stronger model overall, scoring 41.8 to 23.0 on the Noometry Index.

Last verified . 17 shared benchmarks.

ERNIE 5.0 0110 Baidu

41.8

Rank #129 Confirmed

Llama 3.1-8B Meta

23.0

Rank #352 Confirmed

Summary

  • They share 17 benchmarks with published results for both. ERNIE 5.0 0110 scores higher in 8 categories and Llama 3.1-8B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where ERNIE 5.0 0110 leads 63.1 to 29.7.
  • Llama 3.1-8B has downloadable open weights; the other is API-only.

Side by side

ERNIE 5.0 0110Llama 3.1-8B
ProviderBaiduMeta
Noometry Index41.823.0
Released—2024-07-23
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.05
Output $ / M tokens—$0.08
Results tracked2043

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 43.0 (#94), Llama 3.1-8B: 20.2 (#340)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Coding14551195
SciCode—13.2%
WeirdML—1.7%
BigCodeBench Instruct—32.8%
BigCodeBench Complete—40.5%
HumanEval+—62.8%
MBPP+—55.6%

Agentic & Tool Use Not comparable

ERNIE 5.0 0110: —, Llama 3.1-8B: 22.5 (#131)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
Berkeley Function Calling Leaderboard—25.8%
BALROG—15.1%

Reasoning ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 17.0 (#297), Llama 3.1-8B: 14.9 (#321)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Hard Prompts14451175
NYT Connections (extended)10.3%—
CritPt—0%
Chess Puzzles—0%
Thematic Generalization41.7%—
DTBench—50.9%
LMCA—5.4%
Epoch Capabilities Index—116.57
PIQA—81.2%

Math ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 39.3 (#110), Llama 3.1-8B: 10.2 (#317)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Math14371179
OTIS Mock AIME 2024-2025—1.7%
Omni-MATH—13.7%
MATH Level 5—22.9%
GSM8K—82.4%

Knowledge ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 39.8 (#128), Llama 3.1-8B: 8.0 (#307)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Expert14281144
GPQA Diamond—27%
MMLU-Pro—40.6%
GPQA (HELM)—24.7%
BoolQ—82.8%
MMLU—56.1%

Multimodal Not comparable

ERNIE 5.0 0110: 39.9 (#53), Llama 3.1-8B: —

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Vision1249—

Multilingual ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 54.1 (#49), Llama 3.1-8B: 34.0 (#249)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Non-English14361148
LMArena Chinese15121151
LMArena French14671177
LMArena German14601144
LMArena Japanese13821061
LMArena Korean14061053
LMArena Russian14461158
LMArena Spanish14731169

Instruction Following ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 74.5 (#92), Llama 3.1-8B: 58.9 (#258)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Instruction Following14131159
IFEval—74.3%

Long Context ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 43.4 (#95), Llama 3.1-8B: 35.8 (#238)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Longer Query14221182

Writing & Preference ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 63.1 (#66), Llama 3.1-8B: 29.7 (#290)

BenchmarkERNIE 5.0 0110Llama 3.1-8B
LMArena Text14451187
LMArena Creative Writing14261154
LMArena Multi-Turn14341172
EQ-Bench Creative Writing—713
WildBench—68.7%

Frequently asked questions

Is ERNIE 5.0 0110 better than Llama 3.1-8B?

ERNIE 5.0 0110 is the stronger model overall, scoring 41.8 to 23.0 on the Noometry Index.

Is ERNIE 5.0 0110 or Llama 3.1-8B better for coding?

ERNIE 5.0 0110 scores higher on coding benchmarks: 43.0 versus 20.2 in the Noometry coding category.

How many benchmarks do ERNIE 5.0 0110 and Llama 3.1-8B share?

17 benchmarks have published results for both models. ERNIE 5.0 0110 has 20 scored results on Noometry and Llama 3.1-8B has 43.

Related comparisons

Go deeper