Noometry

Model comparison

GPT-4.1 vs Nemotron 3 Ultra

Nemotron 3 Ultra is the stronger model overall, scoring 44.7 to 35.7 on the Noometry Index.

Last verified . 24 shared benchmarks.

GPT-4.1 OpenAI

35.7

Rank #215 Confirmed

Nemotron 3 Ultra NVIDIA

44.7

Rank #71 Confirmed

Summary

  • They share 24 benchmarks with published results for both. GPT-4.1 scores higher in 0 categories and Nemotron 3 Ultra in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Nemotron 3 Ultra leads 30.3 to 11.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 38.3% for GPT-4.1 and 86.7% for Nemotron 3 Ultra.
  • Nemotron 3 Ultra is cheaper at $0.60 / $2.40 per million input/output tokens, against $2 / $8 for GPT-4.1.
  • GPT-4.1 accepts more context: 1.05M tokens versus 262K.
  • Nemotron 3 Ultra has downloadable open weights; the other is API-only.

Side by side

GPT-4.1Nemotron 3 Ultra
ProviderOpenAINVIDIA
Noometry Index35.744.7
Released2025-04-142026-06-04
WeightsProprietaryOpen
Context window1.05M262K
Max output33K128K
Input $ / M tokens$2$0.60
Output $ / M tokens$8$2.40
Results tracked5229

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Nemotron 3 Ultra leads

GPT-4.1: 34.2 (#228), Nemotron 3 Ultra: 37.7 (#174)

BenchmarkGPT-4.1Nemotron 3 Ultra
WeirdML39%43.5%
LMArena Coding13911468
SWE-bench Verified48.5%—
FrontierCode—13.6%
SWE-bench Verified (bash only)39.6%—
Aider Polyglot52.4%—
SciCode—40.3%
CadEval42%—
ALE-Bench558.1—

Agentic & Tool Use Not comparable

GPT-4.1: 34.4 (#40), Nemotron 3 Ultra: —

BenchmarkGPT-4.1Nemotron 3 Ultra
Berkeley Function Calling Leaderboard54%—

Reasoning Nemotron 3 Ultra leads

GPT-4.1: 11.7 (#334), Nemotron 3 Ultra: 30.3 (#78)

BenchmarkGPT-4.1Nemotron 3 Ultra
Chess Puzzles6%12%
LMArena Hard Prompts13841452
DTBench68.3%90.1%
LMCA25.6%36.9%
Epoch Capabilities Index136.79146.2
ARC-AGI-20.4%—
SimpleBench27%—
Kagi LLM Benchmark52.3%—
ARC-AGI-15.5%—
CritPt—3.1%
EnigmaEval2.2%—
Mystery Game Puzzles—20%
ForecastBench61.5—

Math Nemotron 3 Ultra leads

GPT-4.1: 21.4 (#269), Nemotron 3 Ultra: 34.4 (#160)

BenchmarkGPT-4.1Nemotron 3 Ultra
OTIS Mock AIME 2024-202538.3%86.7%
LMArena Math13691456
FrontierMath (Tiers 1-3)6%—
ProofBench—2%
Omni-MATH47.1%—
MATH Level 583%—
FrontierMath (Feb 2025 set)5.5%—
FrontierMath Tier 4 (v1)0%—

Knowledge Nemotron 3 Ultra leads

GPT-4.1: 37.0 (#152), Nemotron 3 Ultra: 52.5 (#56)

BenchmarkGPT-4.1Nemotron 3 Ultra
GPQA Diamond66.9%85.4%
LMArena Expert13641472
Humanity's Last Exam5.4%—
SimpleQA Verified31.1%—
MMLU-Pro81.1%—
Vectara Hallucination Rate5.6%—
GPQA (HELM)65.9%—

Multimodal Not comparable

GPT-4.1: 37.5 (#66), Nemotron 3 Ultra: —

BenchmarkGPT-4.1Nemotron 3 Ultra
LMArena Vision1210—
GeoBench72%—

Multilingual Nemotron 3 Ultra leads

GPT-4.1: 49.6 (#128), Nemotron 3 Ultra: 53.7 (#61)

BenchmarkGPT-4.1Nemotron 3 Ultra
LMArena Non-English13701427
LMArena Chinese13821498
LMArena French13831475
LMArena German13801471
LMArena Korean13391387
LMArena Russian13771415
LMArena Spanish13751452
LMArena Japanese1320—

Instruction Following Nemotron 3 Ultra leads

GPT-4.1: 71.5 (#148), Nemotron 3 Ultra: 74.9 (#84)

BenchmarkGPT-4.1Nemotron 3 Ultra
LMArena Instruction Following13671417
IFEval83.8%—

Long Context Nemotron 3 Ultra leads

GPT-4.1: 40.0 (#158), Nemotron 3 Ultra: 43.9 (#77)

BenchmarkGPT-4.1Nemotron 3 Ultra
LMArena Longer Query13851437
Fiction.LiveBench63.9%—

Writing & Preference Nemotron 3 Ultra leads

GPT-4.1: 57.8 (#120), Nemotron 3 Ultra: 66.3 (#35)

BenchmarkGPT-4.1Nemotron 3 Ultra
LMArena Text13831444
LMArena Creative Writing13631392
EQ-Bench Creative Writing14201692
LMArena Multi-Turn13981412
WildBench85.4%—

Frequently asked questions

Is GPT-4.1 better than Nemotron 3 Ultra?

Nemotron 3 Ultra is the stronger model overall, scoring 44.7 to 35.7 on the Noometry Index.

Which is cheaper, GPT-4.1 or Nemotron 3 Ultra?

Nemotron 3 Ultra is cheaper. It lists at $0.60 per million input tokens and $2.40 per million output tokens; GPT-4.1 lists at $2 and $8.

Is GPT-4.1 or Nemotron 3 Ultra better for coding?

Nemotron 3 Ultra scores higher on coding benchmarks: 37.7 versus 34.2 in the Noometry coding category.

Which has the bigger context window?

GPT-4.1 does, with 1.05M tokens against 262K.

How many benchmarks do GPT-4.1 and Nemotron 3 Ultra share?

24 benchmarks have published results for both models. GPT-4.1 has 52 scored results on Noometry and Nemotron 3 Ultra has 29.

Related comparisons

Go deeper