Noometry

Model comparison

Phi 3 Small 8k Instruct vs Phi-4

Phi-4 is the stronger model overall, scoring 31.2 to 29.3 on the Noometry Index.

Last verified . 25 shared benchmarks.

Phi 3 Small 8k Instruct Microsoft

29.3

Rank #314 Confirmed

Phi-4 Microsoft

31.2

Rank #279 Confirmed

Summary

  • They share 25 benchmarks with published results for both. Phi 3 Small 8k Instruct scores higher in 1 category and Phi-4 in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Phi-4 leads 40.5 to 31.1.
  • The biggest single-benchmark swing is LiveBench Reasoning: 15.9% for Phi 3 Small 8k Instruct and 47.8% for Phi-4.

Side by side

Phi 3 Small 8k InstructPhi-4
ProviderMicrosoftMicrosoft
Noometry Index29.331.2
Released2024-04-232024-12-11
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.07
Output $ / M tokens—$0.14
Results tracked3237

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Phi-4 leads

Phi 3 Small 8k Instruct: 27.9 (#318), Phi-4: 34.4 (#239)

BenchmarkPhi 3 Small 8k InstructPhi-4
LiveBench Coding20.3%30.7%
LMArena Coding11011231
BigCodeBench Instruct—45.5%
BigCodeBench Complete—55.4%

Agentic & Tool Use Not comparable

Phi 3 Small 8k Instruct: —, Phi-4: 22.8 (#128)

BenchmarkPhi 3 Small 8k InstructPhi-4
Berkeley Function Calling Leaderboard—28.8%
BALROG—11.6%

Reasoning Phi-4 leads

Phi 3 Small 8k Instruct: 14.8 (#323), Phi-4: 17.7 (#291)

BenchmarkPhi 3 Small 8k InstructPhi-4
LiveBench Reasoning15.9%47.8%
LMArena Hard Prompts11001220
LiveBench Data Analysis30.3%45.2%
LiveBench24%41.6%
Chess Puzzles—1%
Adversarial NLI58.1%—
BIG-Bench Hard79.1%—
Epoch Capabilities Index—130.42
HellaSwag77%—
WinoGrande81.5%—

Math Phi 3 Small 8k Instruct leads

Phi 3 Small 8k Instruct: 27.6 (#248), Phi-4: 20.8 (#285)

BenchmarkPhi 3 Small 8k InstructPhi-4
LiveBench Math17.6%42%
LMArena Math11511246
OTIS Mock AIME 2024-2025—13.8%
MATH Level 5—64.9%

Knowledge Phi-4 leads

Phi 3 Small 8k Instruct: 29.1 (#240), Phi-4: 32.6 (#209)

BenchmarkPhi 3 Small 8k InstructPhi-4
LMArena Expert10671203
MMLU75.7%84.8%
GPQA Diamond—56.1%
Confabulations—29.4%
Vectara Hallucination Rate—3.7%
ARC (AI2) Challenge90.7%—
OpenBookQA88%—
TriviaQA58.1%—

Multilingual Phi-4 leads

Phi 3 Small 8k Instruct: 28.5 (#272), Phi-4: 37.2 (#237)

BenchmarkPhi 3 Small 8k InstructPhi-4
LMArena Non-English10581197
LMArena Chinese10611212
LMArena French11351224
LMArena German10801222
LMArena Japanese9661158
LMArena Korean8941151
LMArena Russian11111209
LMArena Spanish11111234

Instruction Following Phi-4 leads

Phi 3 Small 8k Instruct: 51.9 (#292), Phi-4: 60.4 (#251)

BenchmarkPhi 3 Small 8k InstructPhi-4
LiveBench Instruction Following47.2%58.4%
LMArena Instruction Following10871201

Long Context Phi-4 leads

Phi 3 Small 8k Instruct: 33.0 (#267), Phi-4: 36.9 (#226)

BenchmarkPhi 3 Small 8k InstructPhi-4
LMArena Longer Query10881217

Writing & Preference Phi-4 leads

Phi 3 Small 8k Instruct: 31.1 (#284), Phi-4: 40.5 (#244)

BenchmarkPhi 3 Small 8k InstructPhi-4
LMArena Text11101217
LMArena Creative Writing10831182
LMArena Multi-Turn10681206
LiveBench Language12.9%25.6%
Short-Story Creative Writing—62.6%

Frequently asked questions

Is Phi 3 Small 8k Instruct better than Phi-4?

Phi-4 is the stronger model overall, scoring 31.2 to 29.3 on the Noometry Index.

Is Phi 3 Small 8k Instruct or Phi-4 better for coding?

Phi-4 scores higher on coding benchmarks: 34.4 versus 27.9 in the Noometry coding category.

How many benchmarks do Phi 3 Small 8k Instruct and Phi-4 share?

25 benchmarks have published results for both models. Phi 3 Small 8k Instruct has 32 scored results on Noometry and Phi-4 has 37.

Related comparisons

Go deeper