Noometry

Anthropic, proprietary

Claude 3.5 Haiku

Claude 3.5 Haiku by Anthropic ranks 315th of 354 ranked models on the Noometry Index as of October 2026, with a score of 29.2. Its strongest category is agentic & tool use, where it ranks 95th.

Last verified

Noometry rank
#315 of 354
Index score
29.2
Evidence
Confirmed 49 results
Provider
Anthropic
Released
October 22, 2024
Weights
Proprietary
Reasoning
Unknown
Context window
—
Max output
—
Input price
Not listed
Output price
Not listed
Blended price
Not listed
Output speed
Not measured
Value
Not ranked
Knowledge cutoff
Unknown

Category scores

Each category score combines every public result we have in that category.

  1. Coding 32.9
  2. Agentic & Tool Use 28.0
  3. Reasoning 17.7
  4. Math 14.7
  5. Knowledge 18.7
  6. Multimodal 26.8
  7. Multilingual 40.0
  8. Instruction Following 62.9
  9. Long Context 38.3
  10. Writing & Preference 42.7
CategoryScoreRankResults
Coding32.9#2658
Agentic & Tool Use28.0#951
Reasoning17.7#2905
Math14.7#3005
Knowledge18.7#2815
Multimodal26.8#1172
Multilingual40.0#2181
Instruction Following62.9#2343
Long Context38.3#2001
Writing & Preference42.7#2347

Strengths and weaknesses

Categories where Claude 3.5 Haiku places highest and lowest among the models ranked in each, with its score against that category's median.

Strongest categories

CategoryScorevs medianRank
Agentic & Tool Use28.0−2.3#95 of 154, top 62%
Long Context38.3−2.7#200 of 296, top 68%
Multilingual40.0−7.4#218 of 297, top 74%

Weakest categories

CategoryScorevs medianRank
Math14.7−21.9#300 of 327, top 92%
Multimodal26.8−11.7#117 of 128, top 92%
Knowledge18.7−18.6#281 of 314, top 90%

Closest competitors

The models ranked just above and below Claude 3.5 Haiku. When scores are this close, price and speed are often the better way to choose.

ModelRankScoreBlended $/MSpeed
DBRX#31129.4——Compare
Gemma 2 27B#31229.4$0.65—Compare
Gemma 1.1 2b IT#31329.3——Compare
Phi 3 Small 8k Instruct#31429.3——Compare
GPT-4#31629.1$37.50—Compare
Llama 2-7B#31729.1——Compare
Granite 4.0 Micro#31829.0$0.0408—Compare
Claude 3 Sonnet#31929.0——Compare

Sponsored placements are available on pages like this one. Advertise on Noometry

Benchmark results

Every published result we track, with its source. Bold rows are the ones used for ranking; where several exist we prefer independent runs over self-reported numbers.

Coding

BenchmarkScorePositionSettingSourceDate
Aider Polyglot28%#33 of 44, top 75%Epoch AI
SciCode27.4%#109 of 121, top 91%Epoch AI
WeirdML30.7%#96 of 119, top 81%Epoch AI
BigCodeBench Instruct46.1%#12 of 64, top 19%BigCodeBench2024-10-22
LiveBench Coding51.4%#17 of 39, top 44%Epoch AI
LMArena Coding1286#208 of 294, top 71%LMArena2026-10-08
BigCodeBench Complete59%#7 of 66, top 11%BigCodeBench2024-10-22
CadEval32%#10 of 14, top 72%Epoch AI

Agentic & Tool Use

BenchmarkScorePositionSettingSourceDate
BALROG19.3%#25 of 35, top 72%Epoch AI

Reasoning

BenchmarkScorePositionSettingSourceDate
CritPt0%#101 of 134, top 76%Epoch AI
LiveBench Reasoning28.1%#31 of 39, top 80%Epoch AI
LMArena Hard Prompts1251#220 of 297, top 75%LMArena2026-10-08
DTBench56.7%#121 of 151, top 81%Epoch AI
LiveBench Data Analysis48.5%#26 of 39, top 67%Epoch AI
Epoch Capabilities Index127.15#150 of 213, top 71%Epoch AI2024-10-22
LiveBench43.5%#28 of 39, top 72%Epoch AI

Math

BenchmarkScorePositionSettingSourceDate
OTIS Mock AIME 2024-20254.3%#151 of 173, top 88%Epoch AI2025-02-25
Omni-MATH22.4%#48 of 57, top 85%HELM Capabilities
LiveBench Math35.5%#31 of 39, top 80%Epoch AI
LMArena Math1244#217 of 285, top 77%LMArena2026-10-08
MATH Level 546.4%#49 of 79, top 63%Epoch AI2025-03-12
FrontierMath (Feb 2025 set)0.3%#64 of 68, top 95%Epoch AI2025-03-07

Knowledge

BenchmarkScorePositionSettingSourceDate
GPQA Diamond38.1%#152 of 186, top 82%Epoch AI2025-03-12
MMLU-Pro60.5%#42 of 58, top 73%HELM Capabilities
Confabulations (lower is better)36.7%#49 of 51, top 97%Lech Mazur benchmarks
GPQA (HELM)36.3%#48 of 57, top 85%HELM Capabilities
LMArena Expert1208#219 of 273, top 81%LMArena2026-10-08
MMLU74.3%#35 of 81, top 44%Epoch AI

Multimodal

BenchmarkScorePositionSettingSourceDate
LMArena Vision1092#108 of 122, top 89%LMArena2026-10-09
GeoBench34%#25 of 25, top 100%Epoch AI

Multilingual

BenchmarkScorePositionSettingSourceDate
LMArena Non-English1238#218 of 297, top 74%LMArena2026-10-08
LMArena Chinese1229#218 of 285, top 77%LMArena2026-10-08
LMArena French1264#178 of 223, top 80%LMArena2026-10-08
LMArena German1237#181 of 231, top 79%LMArena2026-10-08
LMArena Japanese1175#168 of 211, top 80%LMArena2026-10-08
LMArena Korean1173#173 of 213, top 82%LMArena2026-10-08
LMArena Russian1253#211 of 283, top 75%LMArena2026-10-08
LMArena Spanish1261#176 of 226, top 78%LMArena2026-10-08

Instruction Following

BenchmarkScorePositionSettingSourceDate
LiveBench Instruction Following61.9%#26 of 39, top 67%Epoch AI
IFEval79.2%#44 of 57, top 78%HELM Capabilities
LMArena Instruction Following1241#220 of 298, top 74%LMArena2026-10-08

Long Context

BenchmarkScorePositionSettingSourceDate
LMArena Longer Query1261#215 of 291, top 74%LMArena2026-10-08

Writing & Preference

BenchmarkScorePositionSettingSourceDate
LMArena Text1255#225 of 297, top 76%LMArena2026-10-08
LMArena Creative Writing1233#219 of 295, top 75%LMArena2026-10-08
Short-Story Creative Writing73.5%#27 of 39, top 70%Epoch AI
EQ-Bench Creative Writing1146#88 of 115, top 77%EQ-Bench
WildBench76%#43 of 57, top 76%HELM Capabilities
LMArena Multi-Turn1265#216 of 295, top 74%LMArena2026-10-08
LiveBench Language35.4%#22 of 39, top 57%Epoch AI

Compare Claude 3.5 Haiku

Other Anthropic models

Frequently asked questions

How good is Claude 3.5 Haiku?

Claude 3.5 Haiku by Anthropic ranks 315th of 354 ranked models on the Noometry Index as of October 2026, with a score of 29.2. Its strongest category is agentic & tool use, where it ranks 95th.

Is Claude 3.5 Haiku open source?

No. Claude 3.5 Haiku is proprietary and available only through Anthropic's API and partner platforms.

What are Claude 3.5 Haiku's strengths and weaknesses?

Relative to other ranked models, Claude 3.5 Haiku places best in agentic & tool use, long context, multilingual and lowest in math, multimodal, knowledge.

What is Claude 3.5 Haiku best at?

Its best category is agentic & tool use, where it ranks 95th on Noometry.