Noometry

Model comparison

Claude 3.7 Sonnet vs Yi-34B

Claude 3.7 Sonnet is the stronger model overall, scoring 39.5 to 27.8 on the Noometry Index.

Last verified . 20 shared benchmarks.

Claude 3.7 Sonnet Anthropic

39.5

Rank #164 Confirmed

Yi-34B 01.AI

27.8

Rank #329 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Claude 3.7 Sonnet scores higher in 7 categories and Yi-34B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Claude 3.7 Sonnet leads 39.8 to 7.5.
  • The biggest single-benchmark swing is MATH Level 5: 91.2% for Claude 3.7 Sonnet and 5.1% for Yi-34B.
  • Yi-34B has downloadable open weights; the other is API-only.

Side by side

Claude 3.7 SonnetYi-34B
ProviderAnthropic01.AI
Noometry Index39.527.8
Released2025-02-242023-11-02
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked5823

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 40.6 (#136), Yi-34B: 32.3 (#274)

BenchmarkClaude 3.7 SonnetYi-34B
LMArena Coding13611112
SWE-bench Verified61%—
SWE-bench Verified (bash only)52.8%—
Aider Polyglot64.9%—
GSO3.8%—
LiveBench Coding74.5%—
CadEval54%—

Agentic & Tool Use Not comparable

Claude 3.7 Sonnet: 34.1 (#50), Yi-34B: —

BenchmarkClaude 3.7 SonnetYi-34B
TheAgentCompany30.9%—
Cybench20%—
DeepResearch Bench43.6%—
OSWorld35.8%—
METR Time Horizons60%—

Reasoning Yi-34B leads

Claude 3.7 Sonnet: 18.6 (#277), Yi-34B: 21.2 (#226)

BenchmarkClaude 3.7 SonnetYi-34B
LMArena Hard Prompts13331104
Epoch Capabilities Index141.16117.39
ARC-AGI-20.9%—
SimpleBench46.4%—
ARC-AGI-128.6%—
EnigmaEval4.2%—
LiveBench Reasoning87.8%—
LiveBench Data Analysis74%—
BIG-Bench Hard—71.7%
ForecastBench61.8—
LiveBench76.1%—

Math Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 37.5 (#153), Yi-34B: 21.6 (#282)

BenchmarkClaude 3.7 SonnetYi-34B
LMArena Math13371114
MATH Level 591.2%5.1%
OTIS Mock AIME 2024-202557.8%—
Omni-MATH33%—
LiveBench Math79%—
FrontierMath (Feb 2025 set)4.1%—
GSM8K—76%

Knowledge Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 39.8 (#130), Yi-34B: 7.5 (#309)

BenchmarkClaude 3.7 SonnetYi-34B
GPQA Diamond79.7%14.7%
LMArena Expert13211061
Humanity's Last Exam8%—
MMLU-Pro78.4%—
Confabulations14.7%—
GPQA (HELM)60.8%—
MMLU—76.3%

Multimodal Not comparable

Claude 3.7 Sonnet: 33.7 (#95), Yi-34B: —

BenchmarkClaude 3.7 SonnetYi-34B
LMArena Vision1169—
GeoBench68%—
VPCT39%—
SpatialViz-Bench33.9%—

Multilingual Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 44.1 (#179), Yi-34B: 29.7 (#264)

BenchmarkClaude 3.7 SonnetYi-34B
LMArena Non-English12961079
LMArena Chinese12991176
LMArena French13031081
LMArena German13011042
LMArena Japanese1267993
LMArena Korean1249959
LMArena Russian13111050
LMArena Spanish12981070

Instruction Following Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 72.9 (#125), Yi-34B: 56.2 (#274)

BenchmarkClaude 3.7 SonnetYi-34B
LMArena Instruction Following13521091
LiveBench Instruction Following81.3%—
IFEval83.4%—

Long Context Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 50.3 (#10), Yi-34B: 33.2 (#264)

BenchmarkClaude 3.7 SonnetYi-34B
LMArena Longer Query13731094
Fiction.LiveBench83.3%—

Writing & Preference Claude 3.7 Sonnet leads

Claude 3.7 Sonnet: 54.4 (#150), Yi-34B: 34.1 (#273)

BenchmarkClaude 3.7 SonnetYi-34B
LMArena Text13141129
LMArena Creative Writing13321108
LMArena Multi-Turn13391113
Short-Story Creative Writing81.1%—
EQ-Bench Creative Writing1412—
WildBench81.4%—
LiveBench Language59.9%—

Frequently asked questions

Is Claude 3.7 Sonnet better than Yi-34B?

Claude 3.7 Sonnet is the stronger model overall, scoring 39.5 to 27.8 on the Noometry Index.

Is Claude 3.7 Sonnet or Yi-34B better for coding?

Claude 3.7 Sonnet scores higher on coding benchmarks: 40.6 versus 32.3 in the Noometry coding category.

How many benchmarks do Claude 3.7 Sonnet and Yi-34B share?

20 benchmarks have published results for both models. Claude 3.7 Sonnet has 58 scored results on Noometry and Yi-34B has 23.

Related comparisons

Go deeper