Noometry

Model comparison

Gemini 2.5 Pro vs GPT-6 Astra

GPT-6 Astra is the stronger model overall, scoring 70.3 to 44.9 on the Noometry Index. Gemini 2.5 Pro costs 5.8× less per token, which makes it the better buy when GPT-6 Astra's lead doesn't matter for your workload.

Last verified . 37 shared benchmarks.

Gemini 2.5 Pro Google

44.9

Rank #67 Confirmed

GPT-6 Astra OpenAI

70.3

Rank #1 Confirmed

Summary

  • They share 37 benchmarks with published results for both. Gemini 2.5 Pro scores higher in 2 categories and GPT-6 Astra in 8 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in math, where GPT-6 Astra leads 93.4 to 31.9.
  • The biggest single-benchmark swing is FrontierMath Tier 4: 0% for Gemini 2.5 Pro and 97.6% for GPT-6 Astra.
  • Gemini 2.5 Pro is cheaper at $1.25 / $10 per million input/output tokens, against $10 / $50 for GPT-6 Astra.
  • GPT-6 Astra accepts more context: 1.05M tokens versus 1.05M.

Side by side

Gemini 2.5 ProGPT-6 Astra
ProviderGoogleOpenAI
Noometry Index44.970.3
Released2025-03-252026-09-03
WeightsProprietaryProprietary
Context window1.05M1.05M
Max output66K128K
Input $ / M tokens$1.25$10
Output $ / M tokens$10$50
Results tracked7851

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-6 Astra leads

Gemini 2.5 Pro: 42.8 (#86), GPT-6 Astra: 70.5 (#3)

BenchmarkGemini 2.5 ProGPT-6 Astra
LMArena WebDev12271792
SciCode42.8%56.5%
WeirdML54%93.6%
LMArena Coding14521488
ALE-Bench785.522,951
SWE-bench Verified57.6%—
DeepSWE—74.1%
FrontierCode—53.3%
SWE-bench Verified (bash only)53.6%—
Aider Polyglot83.1%—
FrontierSWE—65.5%
GSO3.9%—
LiveBench Coding85.9%—
MirrorCode—46.7%
CadEval64%—
AlgoTune1.51—

Agentic & Tool Use GPT-6 Astra leads

Gemini 2.5 Pro: 29.1 (#81), GPT-6 Astra: 52.3 (#3)

BenchmarkGemini 2.5 ProGPT-6 Astra
Remote Labor Index0.8%20.8%
BALROG43.3%68.3%
Vending-Bench 2573.6415,515
Terminal-Bench32.6%—
APEX-Agents—64.7%
GDPval23.3%—
TheAgentCompany30.3%—
τ²-bench Banking13.7%—
DeepResearch Bench42.8%—
LMArena Search1142—
METR Time Horizons55.4%—

Reasoning GPT-6 Astra leads

Gemini 2.5 Pro: 28.8 (#85), GPT-6 Astra: 85.1 (#1)

BenchmarkGemini 2.5 ProGPT-6 Astra
ARC-AGI-24.9%95%
ARC-AGI-141%98.5%
CritPt2%31.7%
Chess Puzzles20%72%
LMArena Hard Prompts14551464
DTBench82.4%97.3%
LMCA34.8%64.4%
Epoch Capabilities Index145.32166.6
SimpleBench62.4%—
Kagi LLM Benchmark70.3%—
NYT Connections (extended)—98.1%
EnigmaEval5.6%—
EBR-Bench—76.2%
LiveBench Reasoning89.8%—
Mystery Game Puzzles—84%
LiveBench Data Analysis79.9%—
Bench to the Future 3—0.14
ForecastBench61.3—
LiveBench82.3%—

Math GPT-6 Astra leads

Gemini 2.5 Pro: 31.9 (#191), GPT-6 Astra: 93.4 (#1)

BenchmarkGemini 2.5 ProGPT-6 Astra
FrontierMath (Tiers 1-3)24.6%93.7%
FrontierMath Tier 40%97.6%
OTIS Mock AIME 2024-202584.7%100%
LMArena Math14501463
ProofBench—99%
Omni-MATH41.6%—
LiveBench Math90.2%—
MATH Level 595.9%—
FrontierMath (Feb 2025 set)14.1%—
FrontierMath Erdős—2.9%
FrontierMath Tier 4 (v1)4.2%—

Knowledge GPT-6 Astra leads

Gemini 2.5 Pro: 55.8 (#41), GPT-6 Astra: 75.3 (#1)

BenchmarkGemini 2.5 ProGPT-6 Astra
GPQA Diamond85.3%95.8%
Humanity's Last Exam21.6%54.8%
Vectara Hallucination Rate7%8.7%
LMArena Expert14521486
SimpleQA Verified—75.6%
MMLU-Pro86.3%—
Confabulations10.6%—
GPQA (HELM)74.9%—

Multimodal GPT-6 Astra leads

Gemini 2.5 Pro: 44.5 (#16), GPT-6 Astra: 55.1 (#3)

BenchmarkGemini 2.5 ProGPT-6 Astra
LMArena Vision12631280
LMArena Document14211468
GeoBench86%—
VPCT48%—
Blueprint-Bench 2—49.7%
Furniture Assembly—80%
SpatialViz-Bench44.7%—

Multilingual Gemini 2.5 Pro leads

Gemini 2.5 Pro: 55.5 (#30), GPT-6 Astra: 54.0 (#53)

BenchmarkGemini 2.5 ProGPT-6 Astra
LMArena Non-English14521431
LMArena Chinese15071495
LMArena French14731467
LMArena Russian14601434
LMArena Spanish14731412
LMArena German1487—
LMArena Japanese1462—
LMArena Korean1435—

Instruction Following GPT-6 Astra leads

Gemini 2.5 Pro: 75.2 (#71), GPT-6 Astra: 76.6 (#37)

BenchmarkGemini 2.5 ProGPT-6 Astra
LMArena Instruction Following14361454
LiveBench Instruction Following80.6%—
IFEval84%—

Long Context Gemini 2.5 Pro leads

Gemini 2.5 Pro: 59.7 (#5), GPT-6 Astra: 44.7 (#57)

BenchmarkGemini 2.5 ProGPT-6 Astra
LMArena Longer Query14491460
Fiction.LiveBench91.7%—

Writing & Preference GPT-6 Astra leads

Gemini 2.5 Pro: 63.9 (#60), GPT-6 Astra: 75.7 (#7)

BenchmarkGemini 2.5 ProGPT-6 Astra
LMArena Text14581443
LMArena Creative Writing14531423
EQ-Bench Creative Writing14212173
LMArena Multi-Turn14531457
Short-Story Creative Writing83.8%—
WildBench85.7%—
LiveBench Language67.8%—

Frequently asked questions

Is Gemini 2.5 Pro better than GPT-6 Astra?

GPT-6 Astra is the stronger model overall, scoring 70.3 to 44.9 on the Noometry Index. Gemini 2.5 Pro costs 5.8× less per token, which makes it the better buy when GPT-6 Astra's lead doesn't matter for your workload.

Which is cheaper, Gemini 2.5 Pro or GPT-6 Astra?

Gemini 2.5 Pro is cheaper. It lists at $1.25 per million input tokens and $10 per million output tokens; GPT-6 Astra lists at $10 and $50.

Is Gemini 2.5 Pro or GPT-6 Astra better for coding?

GPT-6 Astra scores higher on coding benchmarks: 70.5 versus 42.8 in the Noometry coding category.

Which has the bigger context window?

GPT-6 Astra does, with 1.05M tokens against 1.05M.

How many benchmarks do Gemini 2.5 Pro and GPT-6 Astra share?

37 benchmarks have published results for both models. Gemini 2.5 Pro has 78 scored results on Noometry and GPT-6 Astra has 51.

Related comparisons

Go deeper