Noometry

Model comparison

Llama 4 Scout vs Yi-1.5-34B

Yi-1.5-34B is the stronger model overall, scoring 30.6 to 27.7 on the Noometry Index.

Last verified . 20 shared benchmarks.

Llama 4 Scout Meta

27.7

Rank #330 Confirmed

Yi-1.5-34B 01.AI

30.6

Rank #289 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Llama 4 Scout scores higher in 3 categories and Yi-1.5-34B in 5 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Llama 4 Scout leads 31.9 to 14.8.
  • The biggest single-benchmark swing is MATH Level 5: 62.3% for Llama 4 Scout and 25.5% for Yi-1.5-34B.

Side by side

Llama 4 ScoutYi-1.5-34B
ProviderMeta01.AI
Noometry Index27.730.6
Released2025-04-052024-05-13
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$0.10—
Output $ / M tokens$0.30—
Results tracked4321

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Yi-1.5-34B leads

Llama 4 Scout: 20.2 (#339), Yi-1.5-34B: 32.4 (#272)

BenchmarkLlama 4 ScoutYi-1.5-34B
LMArena Coding12861169
BigCodeBench Complete43.1%43.8%
SWE-bench Verified (bash only)9.1%—
SciCode17%—
BigCodeBench Instruct—33.9%

Agentic & Tool Use Not comparable

Llama 4 Scout: 24.6 (#119), Yi-1.5-34B: —

BenchmarkLlama 4 ScoutYi-1.5-34B
Berkeley Function Calling Leaderboard28.1%—

Reasoning Yi-1.5-34B leads

Llama 4 Scout: 9.1 (#345), Yi-1.5-34B: 22.5 (#191)

BenchmarkLlama 4 ScoutYi-1.5-34B
LMArena Hard Prompts12661160
ARC-AGI-20%—
Kagi LLM Benchmark36.9%—
ARC-AGI-10.5%—
CritPt0%—
DTBench57.9%—
LMCA12%—
Epoch Capabilities Index129.64—
ForecastBench57.5—

Math Yi-1.5-34B leads

Llama 4 Scout: 19.6 (#286), Yi-1.5-34B: 27.5 (#249)

BenchmarkLlama 4 ScoutYi-1.5-34B
LMArena Math12871182
MATH Level 562.3%25.5%
OTIS Mock AIME 2024-20257.8%—
Omni-MATH37.3%—
FrontierMath (Feb 2025 set)0%—

Knowledge Llama 4 Scout leads

Llama 4 Scout: 31.9 (#217), Yi-1.5-34B: 14.8 (#295)

BenchmarkLlama 4 ScoutYi-1.5-34B
GPQA Diamond51.8%32%
LMArena Expert12351144
MMLU-Pro74.2%—
Vectara Hallucination Rate7.7%—
GPQA (HELM)50.7%—

Multimodal Not comparable

Llama 4 Scout: 32.2 (#102), Yi-1.5-34B: —

BenchmarkLlama 4 ScoutYi-1.5-34B
LMArena Vision1118—
SpatialViz-Bench34.2%—

Multilingual Llama 4 Scout leads

Llama 4 Scout: 41.0 (#212), Yi-1.5-34B: 32.3 (#256)

BenchmarkLlama 4 ScoutYi-1.5-34B
LMArena Non-English12521121
LMArena Chinese12551213
LMArena French12821156
LMArena German12721111
LMArena Japanese12061021
LMArena Korean12071005
LMArena Russian12631091
LMArena Spanish12781121

Instruction Following Llama 4 Scout leads

Llama 4 Scout: 65.8 (#217), Yi-1.5-34B: 59.2 (#257)

BenchmarkLlama 4 ScoutYi-1.5-34B
LMArena Instruction Following12481139
IFEval81.8%—

Long Context Yi-1.5-34B leads

Llama 4 Scout: 27.5 (#294), Yi-1.5-34B: 34.6 (#248)

BenchmarkLlama 4 ScoutYi-1.5-34B
LMArena Longer Query12651143
Fiction.LiveBench36%—

Writing & Preference Too close to call

Llama 4 Scout: 37.0 (#261), Yi-1.5-34B: 37.4 (#257)

BenchmarkLlama 4 ScoutYi-1.5-34B
LMArena Text12791173
LMArena Creative Writing12491135
LMArena Multi-Turn12801153
EQ-Bench Creative Writing783—
WildBench78%—

Frequently asked questions

Is Llama 4 Scout better than Yi-1.5-34B?

Yi-1.5-34B is the stronger model overall, scoring 30.6 to 27.7 on the Noometry Index.

Is Llama 4 Scout or Yi-1.5-34B better for coding?

Yi-1.5-34B scores higher on coding benchmarks: 32.4 versus 20.2 in the Noometry coding category.

How many benchmarks do Llama 4 Scout and Yi-1.5-34B share?

20 benchmarks have published results for both models. Llama 4 Scout has 43 scored results on Noometry and Yi-1.5-34B has 21.

Related comparisons

Go deeper