Noometry

Anthropic, proprietary

Claude Instant

Claude Instant by Anthropic has 7 published benchmark results on Noometry, not yet enough to be ranked.

Last verified

Noometry rank
Unranked
Index score
Not ranked
Evidence
Sparse 7 results
Provider
Anthropic
Released
August 9, 2023
Weights
Proprietary
Reasoning
Unknown
Context window
—
Max output
—
Input price
Not listed
Output price
Not listed
Blended price
Not listed
Output speed
Not measured
Value
Not ranked
Knowledge cutoff
Unknown

Category scores

Each category score combines every public result we have in that category.

  1. Reasoning 19.6
CategoryScoreRankResults
Reasoning19.6—1

Sponsored placements are available on pages like this one. Advertise on Noometry

Benchmark results

Every published result we track, with its source. Bold rows are the ones used for ranking; where several exist we prefer independent runs over self-reported numbers.

Coding

BenchmarkScorePositionSettingSourceDate
HumanEval+50.6%#34 of 45, top 76%mar 2024EvalPlus

Reasoning

BenchmarkScorePositionSettingSourceDate
DTBench45.8%#146 of 151, top 97%Epoch AI
Epoch Capabilities Index120.28#167 of 213, top 79%Epoch AI2023-08-09

Math

BenchmarkScorePositionSettingSourceDate
GSM8K86.7%#6 of 38, top 16%Epoch AI
GSM8K80.9%Epoch AI

Knowledge

BenchmarkScorePositionSettingSourceDate
ARC (AI2) Challenge85.7%Epoch AI
ARC (AI2) Challenge86.3%#9 of 39, top 24%Epoch AI
MMLU73.4%#38 of 81, top 47%Epoch AI
MMLU73.2%Epoch AI
TriviaQA78.7%Epoch AI
TriviaQA78.9%#13 of 25, top 52%Epoch AI

Compare Claude Instant

Other Anthropic models

Frequently asked questions

How good is Claude Instant?

Claude Instant by Anthropic has 7 published benchmark results on Noometry, not yet enough to be ranked.

Is Claude Instant open source?

No. Claude Instant is proprietary and available only through Anthropic's API and partner platforms.