Noometry

Model comparison

Mistral Nemo vs Yi-34B

Yi-34B is the stronger model overall, scoring 27.8 to 26.4 on the Noometry Index.

Last verified . 4 shared benchmarks.

Mistral Nemo Mistral AI

26.4

Rank #337 Confirmed

Yi-34B 01.AI

27.8

Rank #329 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Mistral Nemo scores higher in 2 categories and Yi-34B in 2 categories; 3 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Yi-34B leads 34.1 to 28.5.
  • The biggest single-benchmark swing is GPQA Diamond: 29.9% for Mistral Nemo and 14.7% for Yi-34B.

Side by side

Mistral NemoYi-34B
ProviderMistral AI01.AI
Noometry Index26.427.8
Released2024-07-012023-11-02
WeightsOpenOpen
Context window128K—
Max output128K—
Input $ / M tokens$0.15—
Output $ / M tokens$0.15—
Results tracked1023

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Mistral Nemo: —, Yi-34B: 32.3 (#274)

BenchmarkMistral NemoYi-34B
LMArena Coding—1112

Agentic & Tool Use Not comparable

Mistral Nemo: 23.5 (#125), Yi-34B: —

BenchmarkMistral NemoYi-34B
Berkeley Function Calling Leaderboard27.6%—
BALROG17.6%—

Reasoning Too close to call

Mistral Nemo: 20.7 (#232), Yi-34B: 21.2 (#226)

BenchmarkMistral NemoYi-34B
Epoch Capabilities Index118.68117.39
LMArena Hard Prompts—1104
DTBench48.6%—
BIG-Bench Hard—71.7%
PIQA83.5%—

Math Mistral Nemo leads

Mistral Nemo: 25.5 (#268), Yi-34B: 21.6 (#282)

BenchmarkMistral NemoYi-34B
MATH Level 510.8%5.1%
GSM8K84.2%76%
LMArena Math—1114

Knowledge Mistral Nemo leads

Mistral Nemo: 12.3 (#298), Yi-34B: 7.5 (#309)

BenchmarkMistral NemoYi-34B
GPQA Diamond29.9%14.7%
LMArena Expert—1061
BoolQ82.5%—
MMLU—76.3%

Multilingual Not comparable

Mistral Nemo: —, Yi-34B: 29.7 (#264)

BenchmarkMistral NemoYi-34B
LMArena Non-English—1079
LMArena Chinese—1176
LMArena French—1081
LMArena German—1042
LMArena Japanese—993
LMArena Korean—959
LMArena Russian—1050
LMArena Spanish—1070

Instruction Following Not comparable

Mistral Nemo: —, Yi-34B: 56.2 (#274)

BenchmarkMistral NemoYi-34B
LMArena Instruction Following—1091

Long Context Not comparable

Mistral Nemo: —, Yi-34B: 33.2 (#264)

BenchmarkMistral NemoYi-34B
LMArena Longer Query—1094

Writing & Preference Yi-34B leads

Mistral Nemo: 28.5 (#296), Yi-34B: 34.1 (#273)

BenchmarkMistral NemoYi-34B
LMArena Text—1129
LMArena Creative Writing—1108
EQ-Bench Creative Writing881—
LMArena Multi-Turn—1113

Frequently asked questions

Is Mistral Nemo better than Yi-34B?

Yi-34B is the stronger model overall, scoring 27.8 to 26.4 on the Noometry Index.

How many benchmarks do Mistral Nemo and Yi-34B share?

4 benchmarks have published results for both models. Mistral Nemo has 10 scored results on Noometry and Yi-34B has 23.

Related comparisons

Go deeper