Noometry

Model comparison

GPT-4.1 nano vs phi-3-medium 14B

phi-3-medium 14B is the stronger model overall, scoring 29.7 to 27.9 on the Noometry Index.

Last verified . 3 shared benchmarks.

GPT-4.1 nano OpenAI

27.9

Rank #327 Confirmed

phi-3-medium 14B Microsoft

29.7

Rank #306 Reported

Summary

  • They share 3 benchmarks with published results for both. GPT-4.1 nano scores higher in 1 category and phi-3-medium 14B in 2 categories; 2 gaps are clear of the uncertainty.
  • The widest gap is in coding, where phi-3-medium 14B leads 36.8 to 24.1.
  • The biggest single-benchmark swing is MATH Level 5: 70% for GPT-4.1 nano and 17.6% for phi-3-medium 14B.
  • phi-3-medium 14B has downloadable open weights; the other is API-only.

Side by side

GPT-4.1 nanophi-3-medium 14B
ProviderOpenAIMicrosoft
Noometry Index27.929.7
Released2025-04-142024-04-23
WeightsProprietaryOpen
Context window1.05M—
Max output33K—
Input $ / M tokens$0.10—
Output $ / M tokens$0.40—
Results tracked3813

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding phi-3-medium 14B leads

GPT-4.1 nano: 24.1 (#330), phi-3-medium 14B: 36.8 (#201)

BenchmarkGPT-4.1 nanophi-3-medium 14B
Aider Polyglot8.9%—
SciCode25.9%—
WeirdML19%—
BigCodeBench Instruct—37.6%
LMArena Coding1306—
BigCodeBench Complete—48.7%

Agentic & Tool Use Not comparable

GPT-4.1 nano: 26.5 (#104), phi-3-medium 14B: —

BenchmarkGPT-4.1 nanophi-3-medium 14B
Berkeley Function Calling Leaderboard33%—

Reasoning Not comparable

GPT-4.1 nano: 8.5 (#349), phi-3-medium 14B: —

BenchmarkGPT-4.1 nanophi-3-medium 14B
Epoch Capabilities Index129.62121.23
ARC-AGI-20%—
Kagi LLM Benchmark33.3%—
ARC-AGI-10%—
CritPt0%—
LMArena Hard Prompts1286—
DTBench52.5%—
LMCA5.5%—
Adversarial NLI—55.8%
BIG-Bench Hard—81.4%
HellaSwag—82.4%
WinoGrande—81.5%

Math Too close to call

GPT-4.1 nano: 26.9 (#252), phi-3-medium 14B: 27.3 (#250)

BenchmarkGPT-4.1 nanophi-3-medium 14B
MATH Level 570%17.6%
OTIS Mock AIME 2024-202528.9%—
Omni-MATH36.7%—
LMArena Math1274—
FrontierMath (Feb 2025 set)1%—

Knowledge GPT-4.1 nano leads

GPT-4.1 nano: 21.8 (#273), phi-3-medium 14B: 9.1 (#306)

BenchmarkGPT-4.1 nanophi-3-medium 14B
GPQA Diamond48.9%27.6%
SimpleQA Verified6%—
MMLU-Pro55%—
GPQA (HELM)50.7%—
LMArena Expert1272—
ARC (AI2) Challenge—91.6%
MMLU—78%
OpenBookQA—87.4%
TriviaQA—73.9%

Multimodal Not comparable

GPT-4.1 nano: 29.2 (#113), phi-3-medium 14B: —

BenchmarkGPT-4.1 nanophi-3-medium 14B
LMArena Vision1063—

Multilingual Not comparable

GPT-4.1 nano: 41.6 (#205), phi-3-medium 14B: —

BenchmarkGPT-4.1 nanophi-3-medium 14B
LMArena Non-English1260—
LMArena Chinese1270—
LMArena German1288—
LMArena Japanese1198—
LMArena Russian1261—

Instruction Following Not comparable

GPT-4.1 nano: 67.8 (#193), phi-3-medium 14B: —

BenchmarkGPT-4.1 nanophi-3-medium 14B
IFEval84.3%—
LMArena Instruction Following1267—

Long Context Not comparable

GPT-4.1 nano: 23.7 (#296), phi-3-medium 14B: —

BenchmarkGPT-4.1 nanophi-3-medium 14B
Fiction.LiveBench25%—
LMArena Longer Query1283—

Writing & Preference Not comparable

GPT-4.1 nano: 40.5 (#243), phi-3-medium 14B: —

BenchmarkGPT-4.1 nanophi-3-medium 14B
LMArena Text1285—
LMArena Creative Writing1260—
EQ-Bench Creative Writing946—
WildBench81.2%—
LMArena Multi-Turn1277—

Frequently asked questions

Is GPT-4.1 nano better than phi-3-medium 14B?

phi-3-medium 14B is the stronger model overall, scoring 29.7 to 27.9 on the Noometry Index.

Is GPT-4.1 nano or phi-3-medium 14B better for coding?

phi-3-medium 14B scores higher on coding benchmarks: 36.8 versus 24.1 in the Noometry coding category.

How many benchmarks do GPT-4.1 nano and phi-3-medium 14B share?

3 benchmarks have published results for both models. GPT-4.1 nano has 38 scored results on Noometry and phi-3-medium 14B has 13.

Related comparisons

Go deeper