Noometry

Model comparison

Claude 3.5 Haiku vs gpt-oss-20b

gpt-oss-20b is the stronger model overall, scoring 32.5 to 29.2 on the Noometry Index.

Last verified . 29 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

gpt-oss-20b OpenAI

32.5

Rank #255 Confirmed

Summary

  • They share 29 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 4 categories and gpt-oss-20b in 5 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where gpt-oss-20b leads 39.4 to 14.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.3% for Claude 3.5 Haiku and 65.3% for gpt-oss-20b.
  • gpt-oss-20b has downloadable open weights; the other is API-only.

Side by side

Claude 3.5 Haikugpt-oss-20b
ProviderAnthropicOpenAI
Noometry Index29.232.5
Released2024-10-222025-08-05
WeightsProprietaryOpen
Context window—131K
Max output—16K
Input $ / M tokens—$0.018
Output $ / M tokens—$0.09
Results tracked4934

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding gpt-oss-20b leads

Claude 3.5 Haiku: 32.9 (#265), gpt-oss-20b: 37.6 (#192)

BenchmarkClaude 3.5 Haikugpt-oss-20b
SciCode27.4%34.4%
WeirdML30.7%40.9%
LMArena Coding12861306
Aider Polyglot28%—
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
BigCodeBench Complete59%—
CadEval32%—
ALE-Bench—566.05

Agentic & Tool Use Claude 3.5 Haiku leads

Claude 3.5 Haiku: 28.0 (#95), gpt-oss-20b: 9.3 (#154)

BenchmarkClaude 3.5 Haikugpt-oss-20b
Terminal-Bench—3.4%
BALROG19.3%—

Reasoning gpt-oss-20b leads

Claude 3.5 Haiku: 17.7 (#290), gpt-oss-20b: 19.3 (#261)

BenchmarkClaude 3.5 Haikugpt-oss-20b
CritPt0%1.4%
LMArena Hard Prompts12511274
DTBench56.7%68%
Epoch Capabilities Index127.15137.82
Kagi LLM Benchmark—53.2%
Chess Puzzles—4%
LiveBench Reasoning28.1%—
LiveBench Data Analysis48.5%—
LMCA—14.5%
LiveBench43.5%—

Math gpt-oss-20b leads

Claude 3.5 Haiku: 14.7 (#300), gpt-oss-20b: 39.4 (#103)

BenchmarkClaude 3.5 Haikugpt-oss-20b
OTIS Mock AIME 2024-20254.3%65.3%
Omni-MATH22.4%56.5%
LMArena Math12441317
LiveBench Math35.5%—
MATH Level 546.4%—
FrontierMath (Feb 2025 set)0.3%—

Knowledge gpt-oss-20b leads

Claude 3.5 Haiku: 18.7 (#281), gpt-oss-20b: 34.6 (#195)

BenchmarkClaude 3.5 Haikugpt-oss-20b
GPQA Diamond38.1%60.8%
MMLU-Pro60.5%74%
GPQA (HELM)36.3%59.4%
LMArena Expert12081258
Confabulations36.7%—
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.8 (#117), gpt-oss-20b: —

BenchmarkClaude 3.5 Haikugpt-oss-20b
LMArena Vision1092—
GeoBench34%—

Multilingual gpt-oss-20b leads

Claude 3.5 Haiku: 40.0 (#218), gpt-oss-20b: 42.2 (#197)

BenchmarkClaude 3.5 Haikugpt-oss-20b
LMArena Non-English12381268
LMArena Chinese12291314
LMArena German12371255
LMArena Japanese11751244
LMArena Korean11731236
LMArena Russian12531278
LMArena Spanish12611267
LMArena French1264—

Instruction Following Claude 3.5 Haiku leads

Claude 3.5 Haiku: 62.9 (#234), gpt-oss-20b: 61.8 (#240)

BenchmarkClaude 3.5 Haikugpt-oss-20b
IFEval79.2%73.2%
LMArena Instruction Following12411236
LiveBench Instruction Following61.9%—

Long Context Too close to call

Claude 3.5 Haiku: 38.3 (#200), gpt-oss-20b: 37.9 (#209)

BenchmarkClaude 3.5 Haikugpt-oss-20b
LMArena Longer Query12611250

Writing & Preference Claude 3.5 Haiku leads

Claude 3.5 Haiku: 42.7 (#234), gpt-oss-20b: 35.5 (#265)

BenchmarkClaude 3.5 Haikugpt-oss-20b
LMArena Text12551287
LMArena Creative Writing12331201
EQ-Bench Creative Writing1146666
WildBench76%73.7%
LMArena Multi-Turn12651268
Short-Story Creative Writing73.5%—
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than gpt-oss-20b?

gpt-oss-20b is the stronger model overall, scoring 32.5 to 29.2 on the Noometry Index.

Is Claude 3.5 Haiku or gpt-oss-20b better for coding?

gpt-oss-20b scores higher on coding benchmarks: 37.6 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and gpt-oss-20b share?

29 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and gpt-oss-20b has 34.

Related comparisons

Go deeper