Noometry

Model comparison

o3-mini vs Yi-1.5-34B

o3-mini is the stronger model overall, scoring 36.7 to 30.6 on the Noometry Index.

Last verified . 19 shared benchmarks.

o3-mini OpenAI

36.7

Rank #212 Confirmed

Yi-1.5-34B 01.AI

30.6

Rank #289 Confirmed

Summary

  • They share 19 benchmarks with published results for both. o3-mini scores higher in 6 categories and Yi-1.5-34B in 2 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where o3-mini leads 38.3 to 14.8.
  • The biggest single-benchmark swing is MATH Level 5: 96.5% for o3-mini and 25.5% for Yi-1.5-34B.
  • Yi-1.5-34B has downloadable open weights; the other is API-only.

Side by side

o3-miniYi-1.5-34B
ProviderOpenAI01.AI
Noometry Index36.730.6
Released2024-12-202024-05-13
WeightsProprietaryOpen
Context window200K—
Max output100K—
Input $ / M tokens$1.10—
Output $ / M tokens$4.40—
Results tracked5121

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3-mini leads

o3-mini: 40.8 (#132), Yi-1.5-34B: 32.4 (#272)

Benchmarko3-miniYi-1.5-34B
LMArena Coding13781169
Aider Polyglot60.4%—
SciCode39.8%—
GSO1.3%—
WeirdML43.7%—
BigCodeBench Instruct—33.9%
LiveBench Coding82.7%—
BigCodeBench Complete—43.8%
CadEval54%—

Agentic & Tool Use Not comparable

o3-mini: 29.6 (#84), Yi-1.5-34B: —

Benchmarko3-miniYi-1.5-34B
Cybench22.5%—

Reasoning Yi-1.5-34B leads

o3-mini: 16.3 (#305), Yi-1.5-34B: 22.5 (#191)

Benchmarko3-miniYi-1.5-34B
LMArena Hard Prompts13661160
ARC-AGI-23%—
SimpleBench22.8%—
ARC-AGI-134.5%—
CritPt0.3%—
Chess Puzzles17%—
LiveBench Reasoning89.6%—
Mystery Game Puzzles7%—
DTBench68.8%—
LiveBench Data Analysis70.6%—
LMCA19%—
Epoch Capabilities Index140.34—
ForecastBench59.6—
LiveBench75.9%—

Math Too close to call

o3-mini: 28.1 (#244), Yi-1.5-34B: 27.5 (#249)

Benchmarko3-miniYi-1.5-34B
LMArena Math13961182
MATH Level 596.5%25.5%
FrontierMath (Tiers 1-3)18.6%—
FrontierMath Tier 40%—
OTIS Mock AIME 2024-202576.9%—
LiveBench Math77.3%—
FrontierMath (Feb 2025 set)12.4%—
FrontierMath Tier 4 (v1)4.2%—

Knowledge o3-mini leads

o3-mini: 38.3 (#146), Yi-1.5-34B: 14.8 (#295)

Benchmarko3-miniYi-1.5-34B
GPQA Diamond77%32%
LMArena Expert13641144
SimpleQA Verified15.3%—
Confabulations17.9%—

Multilingual o3-mini leads

o3-mini: 45.7 (#164), Yi-1.5-34B: 32.3 (#256)

Benchmarko3-miniYi-1.5-34B
LMArena Non-English13191121
LMArena Chinese13791213
LMArena French13341156
LMArena German13031111
LMArena Japanese12861021
LMArena Korean13141005
LMArena Russian13041091
LMArena Spanish13211121

Instruction Following o3-mini leads

o3-mini: 75.1 (#72), Yi-1.5-34B: 59.2 (#257)

Benchmarko3-miniYi-1.5-34B
LMArena Instruction Following13371139
LiveBench Instruction Following84.4%—

Long Context Too close to call

o3-mini: 33.8 (#256), Yi-1.5-34B: 34.6 (#248)

Benchmarko3-miniYi-1.5-34B
LMArena Longer Query13431143
Fiction.LiveBench50%—

Writing & Preference o3-mini leads

o3-mini: 50.3 (#182), Yi-1.5-34B: 37.4 (#257)

Benchmarko3-miniYi-1.5-34B
LMArena Text13371173
LMArena Creative Writing12861135
LMArena Multi-Turn13201153
Short-Story Creative Writing61.7%—
LiveBench Language50.7%—

Frequently asked questions

Is o3-mini better than Yi-1.5-34B?

o3-mini is the stronger model overall, scoring 36.7 to 30.6 on the Noometry Index.

Is o3-mini or Yi-1.5-34B better for coding?

o3-mini scores higher on coding benchmarks: 40.8 versus 32.4 in the Noometry coding category.

How many benchmarks do o3-mini and Yi-1.5-34B share?

19 benchmarks have published results for both models. o3-mini has 51 scored results on Noometry and Yi-1.5-34B has 21.

Related comparisons

Go deeper