Noometry

Model comparison

Mistral Small vs Yi-34B

Mistral Small is the stronger model overall, scoring 33.4 to 27.8 on the Noometry Index.

Last verified . 20 shared benchmarks.

Mistral Small Mistral AI

33.4

Rank #243 Confirmed

Yi-34B 01.AI

27.8

Rank #329 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Mistral Small scores higher in 6 categories and Yi-34B in 2 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Mistral Small leads 31.0 to 7.5.
  • The biggest single-benchmark swing is MATH Level 5: 46.8% for Mistral Small and 5.1% for Yi-34B.

Side by side

Mistral SmallYi-34B
ProviderMistral AI01.AI
Noometry Index33.427.8
Released2024-02-262023-11-02
WeightsOpenOpen
Context window262K—
Max output256K—
Input $ / M tokens$0.15—
Output $ / M tokens$0.60—
Results tracked3923

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small leads

Mistral Small: 34.0 (#247), Yi-34B: 32.3 (#274)

BenchmarkMistral SmallYi-34B
LMArena Coding13621112
SciCode26.5%—
BigCodeBench Instruct36.1%—
LiveBench Coding36.2%—
BigCodeBench Complete46.6%—
ALE-Bench497.62—

Agentic & Tool Use Not comparable

Mistral Small: 28.1 (#93), Yi-34B: —

BenchmarkMistral SmallYi-34B
Berkeley Function Calling Leaderboard37.1%—

Reasoning Yi-34B leads

Mistral Small: 19.8 (#250), Yi-34B: 21.2 (#226)

BenchmarkMistral SmallYi-34B
LMArena Hard Prompts13351104
Kagi LLM Benchmark37.8%—
CritPt0%—
LiveBench Reasoning44.8%—
DTBench70.9%—
LiveBench Data Analysis53.7%—
LMCA20.6%—
BIG-Bench Hard—71.7%
Epoch Capabilities Index—117.39
LiveBench44%—

Math Yi-34B leads

Mistral Small: 16.4 (#293), Yi-34B: 21.6 (#282)

BenchmarkMistral SmallYi-34B
LMArena Math13411114
MATH Level 546.8%5.1%
OTIS Mock AIME 2024-20255.8%—
LiveBench Math39.9%—
GSM8K—76%

Knowledge Mistral Small leads

Mistral Small: 31.0 (#222), Yi-34B: 7.5 (#309)

BenchmarkMistral SmallYi-34B
GPQA Diamond47.5%14.7%
LMArena Expert12911061
MMLU68.7%76.3%
Vectara Hallucination Rate5.1%—

Multimodal Not comparable

Mistral Small: 33.5 (#96), Yi-34B: —

BenchmarkMistral SmallYi-34B
LMArena Vision1142—

Multilingual Mistral Small leads

Mistral Small: 45.5 (#169), Yi-34B: 29.7 (#264)

BenchmarkMistral SmallYi-34B
LMArena Non-English13151079
LMArena Chinese13401176
LMArena French13371081
LMArena German13401042
LMArena Japanese1275993
LMArena Korean1259959
LMArena Russian13241050
LMArena Spanish13461070

Instruction Following Mistral Small leads

Mistral Small: 66.4 (#209), Yi-34B: 56.2 (#274)

BenchmarkMistral SmallYi-34B
LMArena Instruction Following13101091
LiveBench Instruction Following63.7%—

Long Context Mistral Small leads

Mistral Small: 40.4 (#156), Yi-34B: 33.2 (#264)

BenchmarkMistral SmallYi-34B
LMArena Longer Query13271094

Writing & Preference Mistral Small leads

Mistral Small: 52.5 (#171), Yi-34B: 34.1 (#273)

BenchmarkMistral SmallYi-34B
LMArena Text13381129
LMArena Creative Writing13051108
LMArena Multi-Turn13441113
LiveBench Language30.5%—

Frequently asked questions

Is Mistral Small better than Yi-34B?

Mistral Small is the stronger model overall, scoring 33.4 to 27.8 on the Noometry Index.

Is Mistral Small or Yi-34B better for coding?

Mistral Small scores higher on coding benchmarks: 34.0 versus 32.3 in the Noometry coding category.

How many benchmarks do Mistral Small and Yi-34B share?

20 benchmarks have published results for both models. Mistral Small has 39 scored results on Noometry and Yi-34B has 23.

Related comparisons

Go deeper