Noometry

Model comparison

GPT-4 vs Phi 3 Mini 128k Instruct

GPT-4 and Phi 3 Mini 128k Instruct score almost the same on the Noometry Index (29.1 vs 29.7), so choose on price, context window or the category you care about most.

Last verified . 19 shared benchmarks.

GPT-4 OpenAI

29.1

Rank #316 Confirmed

Phi 3 Mini 128k Instruct Microsoft

29.7

Rank #305 Confirmed

Summary

  • They share 19 benchmarks with published results for both. GPT-4 scores higher in 5 categories and Phi 3 Mini 128k Instruct in 3 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Phi 3 Mini 128k Instruct leads 31.6 to 10.8.
  • The biggest single-benchmark swing is BigCodeBench Complete: 57.2% for GPT-4 and 40.6% for Phi 3 Mini 128k Instruct.
  • Phi 3 Mini 128k Instruct has downloadable open weights; the other is API-only.

Side by side

GPT-4Phi 3 Mini 128k Instruct
ProviderOpenAIMicrosoft
Noometry Index29.129.7
Released2023-03-142024-04-23
WeightsProprietaryOpen
Context window8K—
Max output8K—
Input $ / M tokens$30—
Output $ / M tokens$60—
Results tracked3819

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 leads

GPT-4: 31.6 (#283), Phi 3 Mini 128k Instruct: 28.8 (#312)

BenchmarkGPT-4Phi 3 Mini 128k Instruct
BigCodeBench Instruct46%29.6%
LMArena Coding12541039
BigCodeBench Complete57.2%40.6%
WeirdML12.4%—
HumanEval+79.3%—

Agentic & Tool Use Not comparable

GPT-4: —, Phi 3 Mini 128k Instruct: —

BenchmarkGPT-4Phi 3 Mini 128k Instruct
METR Time Horizons36.1%—

Reasoning Phi 3 Mini 128k Instruct leads

GPT-4: 17.8 (#289), Phi 3 Mini 128k Instruct: 19.6 (#256)

BenchmarkGPT-4Phi 3 Mini 128k Instruct
LMArena Hard Prompts12411028
Chess Puzzles4%—
Mystery Game Puzzles12%—
DTBench62.7%—
LMCA17.1%—
BIG-Bench Hard75.1%—
Epoch Capabilities Index125.89—
ForecastBench57.8—
HellaSwag95.3%—
WinoGrande87.5%—

Math Phi 3 Mini 128k Instruct leads

GPT-4: 10.8 (#309), Phi 3 Mini 128k Instruct: 31.6 (#222)

BenchmarkGPT-4Phi 3 Mini 128k Instruct
LMArena Math12691089
OTIS Mock AIME 2024-20251.1%—
MATH Level 523%—
GSM8K92%—

Knowledge Phi 3 Mini 128k Instruct leads

GPT-4: 18.4 (#282), Phi 3 Mini 128k Instruct: 26.8 (#254)

BenchmarkGPT-4Phi 3 Mini 128k Instruct
LMArena Expert1211984
GPQA Diamond35.7%—
MMLU86.4%—
TriviaQA84.8%—

Multilingual GPT-4 leads

GPT-4: 40.6 (#215), Phi 3 Mini 128k Instruct: 25.2 (#285)

BenchmarkGPT-4Phi 3 Mini 128k Instruct
LMArena Non-English12461000
LMArena Chinese12421016
LMArena French12831039
LMArena German12511006
LMArena Japanese1209899
LMArena Korean1184856
LMArena Russian12511004
LMArena Spanish12611059

Instruction Following GPT-4 leads

GPT-4: 65.3 (#222), Phi 3 Mini 128k Instruct: 51.8 (#294)

BenchmarkGPT-4Phi 3 Mini 128k Instruct
LMArena Instruction Following12411023

Long Context GPT-4 leads

GPT-4: 37.7 (#212), Phi 3 Mini 128k Instruct: 30.4 (#289)

BenchmarkGPT-4Phi 3 Mini 128k Instruct
LMArena Longer Query1244996

Writing & Preference GPT-4 leads

GPT-4: 34.9 (#268), Phi 3 Mini 128k Instruct: 27.1 (#301)

BenchmarkGPT-4Phi 3 Mini 128k Instruct
LMArena Text12631050
LMArena Creative Writing12441024
LMArena Multi-Turn1257989
EQ-Bench Creative Writing752—

Frequently asked questions

Is GPT-4 better than Phi 3 Mini 128k Instruct?

GPT-4 and Phi 3 Mini 128k Instruct score almost the same on the Noometry Index (29.1 vs 29.7), so choose on price, context window or the category you care about most.

Is GPT-4 or Phi 3 Mini 128k Instruct better for coding?

GPT-4 scores higher on coding benchmarks: 31.6 versus 28.8 in the Noometry coding category.

How many benchmarks do GPT-4 and Phi 3 Mini 128k Instruct share?

19 benchmarks have published results for both models. GPT-4 has 38 scored results on Noometry and Phi 3 Mini 128k Instruct has 19.

Related comparisons

Go deeper