Noometry

Model comparison

ERNIE 5.0 0110 vs Mixtral 8x22B

ERNIE 5.0 0110 is the stronger model overall, scoring 41.2 to 26.7 on the Noometry Index.

Last verified . 17 shared benchmarks.

ERNIE 5.0 0110 Baidu

41.2

Rank #129 Confirmed

Mixtral 8x22B Mistral AI

26.7

Rank #330 Confirmed

Summary

  • They share 17 benchmarks with published results for both. ERNIE 5.0 0110 scores higher in 7 categories and Mixtral 8x22B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where ERNIE 5.0 0110 leads 63.4 to 37.1.
  • Mixtral 8x22B has downloadable open weights; the other is API-only.

Side by side

ERNIE 5.0 0110Mixtral 8x22B
ProviderBaiduMistral AI
Noometry Index41.226.7
Released—2024-04-17
WeightsProprietaryOpen
Context window—64K
Max output—64K
Input $ / M tokens—$2
Output $ / M tokens—$6
Results tracked2034

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 42.3 (#94), Mixtral 8x22B: 23.8 (#324)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Coding14551166
WeirdML—3.2%
BigCodeBench Instruct—40.6%
BigCodeBench Complete—50.2%
HumanEval+—72%
MBPP+—64.3%

Agentic & Tool Use Not comparable

ERNIE 5.0 0110: —, Mixtral 8x22B: 22.8 (#119)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
Cybench—7.5%

Reasoning Mixtral 8x22B leads

ERNIE 5.0 0110: 16.3 (#296), Mixtral 8x22B: 19.1 (#246)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Hard Prompts14461150
NYT Connections (extended)10.3%—
Thematic Generalization41.7%—
DTBench—55.1%
Epoch Capabilities Index—122
ForecastBench—56.3

Math ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 36.5 (#111), Mixtral 8x22B: 21.2 (#271)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Math14371184
Omni-MATH—16.3%
MATH Level 5—24.2%

Knowledge ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 39.2 (#125), Mixtral 8x22B: 15.1 (#288)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Expert14271113
GPQA Diamond—34.1%
MMLU-Pro—46%
GPQA (HELM)—33.4%
MMLU—77.8%

Multimodal Not comparable

ERNIE 5.0 0110: 39.1 (#49), Mixtral 8x22B: —

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Vision1248—

Multilingual ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 54.3 (#47), Mixtral 8x22B: 32.9 (#250)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Non-English14361128
LMArena Chinese15121116
LMArena French14661165
LMArena German14621141
LMArena Japanese13831037
LMArena Korean14061056
LMArena Russian14461158
LMArena Spanish14731150

Instruction Following ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 74.7 (#88), Mixtral 8x22B: 57.9 (#261)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Instruction Following14131147
IFEval—72.4%

Long Context ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 43.4 (#90), Mixtral 8x22B: 34.6 (#242)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Longer Query14221144

Writing & Preference ERNIE 5.0 0110 leads

ERNIE 5.0 0110: 63.4 (#64), Mixtral 8x22B: 37.1 (#257)

BenchmarkERNIE 5.0 0110Mixtral 8x22B
LMArena Text14451162
LMArena Creative Writing14261141
LMArena Multi-Turn14331130
WildBench—71.1%

Frequently asked questions

Is ERNIE 5.0 0110 better than Mixtral 8x22B?

ERNIE 5.0 0110 is the stronger model overall, scoring 41.2 to 26.7 on the Noometry Index.

Is ERNIE 5.0 0110 or Mixtral 8x22B better for coding?

ERNIE 5.0 0110 scores higher on coding benchmarks: 42.3 versus 23.8 in the Noometry coding category.

How many benchmarks do ERNIE 5.0 0110 and Mixtral 8x22B share?

17 benchmarks have published results for both models. ERNIE 5.0 0110 has 20 scored results on Noometry and Mixtral 8x22B has 34.

Related comparisons

Go deeper