Noometry

Reasoning benchmark

BIG-Bench Hard leaderboard

As of October 2026, Gemini 1.5 Pro (May 2024) has the highest published BIG-Bench Hard score on Noometry at 89.2%, out of 27 models with results.

Last verified

About BIG-Bench Hard

23 challenging BIG-Bench tasks where early models fell short of human raters.

Category
Reasoning
Introduced
2022
Format
Mixed
Unit
Percent (random guessing ≈ 25%)
Official site
github.com

Top 15 models

  1. Gemini 1.5 Pro (May 2024) 89.2%
  2. DeepSeek-V3 87.5%
  3. Llama 3.1-405B 82.9%
  4. phi-3-medium 14B 81.4%
  5. Qwen2.5 72B Instruct 79.8%
  6. Phi 3 Small 8k Instruct 79.1%
  7. DeepSeek-V2 (MoE-236B, May 2024) 78.8%
  8. GPT-4 75.1%
  9. Phi 3 Mini 4k Instruct 71.7%
  10. Yi-34B 71.7%
  11. Llama 2-70B 64.9%
  12. GPT-3.5-turbo 61.6%
  13. Phi-2 59.4%
  14. Nemotron-4 15B 58.7%
  15. Llama 2-13B 58.2%

Sponsored placements are available on pages like this one. Advertise on Noometry

All results

#ModelProviderScoreSettingSourceDate
1Gemini 1.5 Pro (May 2024) Google89.2%Epoch AI
2DeepSeek-V3 DeepSeek87.5%Epoch AI
3Llama 3.1-405B Meta82.9%Epoch AI
4phi-3-medium 14B Microsoft81.4%Epoch AI
5Qwen2.5 72B Instruct Alibaba (Qwen)79.8%Epoch AI
6Phi 3 Small 8k Instruct Microsoft79.1%Epoch AI
7DeepSeek-V2 (MoE-236B, May 2024) DeepSeek78.8%Epoch AI
8GPT-4 OpenAI75.1%Epoch AI
9Phi 3 Mini 4k Instruct Microsoft71.7%Epoch AI
10Yi-34B 01.AI71.7%Epoch AI
11Llama 2-70B Meta64.9%Epoch AI
12GPT-3.5-turbo OpenAI61.6%Epoch AI
13Phi-2 Microsoft59.4%Epoch AI
14Nemotron-4 15B NVIDIA58.7%Epoch AI
15Llama 2-13B Meta58.2%Epoch AI
16Mistral 7B Mistral AI56.1%Epoch AI
17Gemma 7B Google55.1%Epoch AI
18Qwen-14B Alibaba (Qwen)55%Epoch AI
19Yi 6B 01.AI47.2%Epoch AI
20Qwen-7B Alibaba (Qwen)45%Epoch AI
21Llama 2-34B Meta44.1%Epoch AI
22Llama 2-7B Meta39.2%Epoch AI
23Llama 13b Meta37.9%Epoch AI
24Falcon-40B Technology Innovation Institute37.1%Epoch AI
25Gemma 2B Google35.2%Epoch AI
26INTELLECT-1 Hugging Face34.9%Epoch AI
27Falcon-7B Technology Innovation Institute28.8%Epoch AI

Compare the leaders

Other reasoning benchmarks

Frequently asked questions

What does BIG-Bench Hard measure?

23 challenging BIG-Bench tasks where early models fell short of human raters.

Which model has the highest BIG-Bench Hard score?

As of October 2026, Gemini 1.5 Pro (May 2024) has the highest published BIG-Bench Hard score on Noometry at 89.2%, out of 27 models with results.

What is the best open-weight model on BIG-Bench Hard?

DeepSeek-V3 has the highest BIG-Bench Hard accuracy among open-weight models at 87.5%, ranking 2 of 27 overall.