Noometry

Model comparison

Gemini 2.5 Flash vs GPT-5.6 Sol

GPT-5.6 Sol is the stronger model overall, scoring 65.0 to 39.3 on the Noometry Index. Gemini 2.5 Flash costs 9.4× less per token, which makes it the better buy when GPT-5.6 Sol's lead doesn't matter for your workload.

Last verified . 34 shared benchmarks.

Gemini 2.5 Flash Google

39.3

Rank #170 Confirmed

GPT-5.6 Sol OpenAI

65.0

Rank #7 Confirmed

Summary

  • They share 34 benchmarks with published results for both. Gemini 2.5 Flash scores higher in 1 category and GPT-5.6 Sol in 9 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GPT-5.6 Sol leads 74.8 to 18.1.
  • The biggest single-benchmark swing is ARC-AGI-2: 2.5% for Gemini 2.5 Flash and 92.5% for GPT-5.6 Sol.
  • Gemini 2.5 Flash is cheaper at $0.30 / $2.50 per million input/output tokens, against $4 / $20 for GPT-5.6 Sol.
  • GPT-5.6 Sol accepts more context: 1.05M tokens versus 1.05M.

Side by side

Gemini 2.5 FlashGPT-5.6 Sol
ProviderGoogleOpenAI
Noometry Index39.365.0
Released2025-04-172026-07-09
WeightsProprietaryProprietary
Context window1.05M1.05M
Max output66K128K
Input $ / M tokens$0.30$4
Output $ / M tokens$2.50$20
Results tracked5465

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.6 Sol leads

Gemini 2.5 Flash: 35.8 (#220), GPT-5.6 Sol: 65.1 (#7)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
WeirdML41.9%89.4%
LMArena Coding14241498
ALE-Bench661.882,177
DeepSWE—72.7%
FrontierCode—47.5%
SWE-bench Verified (bash only)28.7%—
Aider Polyglot55.1%—
CursorBench—41.7%
LMArena WebDev—1618
FrontierSWE—32.2%
SciCode—57.1%
GSO—76.5%
MirrorCode—20%

Agentic & Tool Use GPT-5.6 Sol leads

Gemini 2.5 Flash: 30.8 (#74), GPT-5.6 Sol: 50.3 (#7)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
BALROG33.5%60%
Vending-Bench 2548.849,619
Terminal-Bench17.1%—
APEX-Agents—51.4%
Berkeley Function Calling Leaderboard56.2%—
OSWorld 2.0—27.3%
TheAgentCompany41.1%—
τ²-bench Banking—46.9%
PostTrainBench—36.2%
GBAEval—52.6%
GDP.pdf—30.7%
LMArena Search—1257

Reasoning GPT-5.6 Sol leads

Gemini 2.5 Flash: 18.1 (#286), GPT-5.6 Sol: 74.8 (#8)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
ARC-AGI-22.5%92.5%
SimpleBench41.2%71.7%
Kagi LLM Benchmark56.8%67%
ARC-AGI-133.3%97.5%
CritPt1.1%32.3%
EnigmaEval2.7%37.1%
LMArena Hard Prompts14221484
DTBench76.5%96%
LMCA27.5%59.2%
Epoch Capabilities Index143.03161.66
NYT Connections (extended)—93.8%
Chess Puzzles—64%
EBR-Bench—44.8%
Mystery Game Puzzles—58%
Surface Evolver Bench—93.1%
Bench to the Future 3—0.14
ForecastBench60.6—

Math GPT-5.6 Sol leads

Gemini 2.5 Flash: 39.9 (#98), GPT-5.6 Sol: 85.6 (#9)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
OTIS Mock AIME 2024-202573.1%100%
LMArena Math14151474
FrontierMath (Tiers 1-3)—89.1%
FrontierMath Tier 4—82.9%
ProofBench—83%
Omni-MATH38.5%—
FrontierMath (Feb 2025 set)4.8%—
FrontierMath Erdős—0%
FrontierMath Tier 4 (v1)4.2%—

Knowledge GPT-5.6 Sol leads

Gemini 2.5 Flash: 36.4 (#168), GPT-5.6 Sol: 64.3 (#18)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
Vectara Hallucination Rate7.8%12.4%
LMArena Expert14261516
GPQA Diamond—93.5%
Humanity's Last Exam12.1%—
SimpleQA Verified—69.7%
MMLU-Pro63.9%—
Confabulations16.8%—
GPQA (HELM)39%—

Multimodal GPT-5.6 Sol leads

Gemini 2.5 Flash: 41.8 (#32), GPT-5.6 Sol: 48.6 (#9)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
LMArena Vision12531281
GeoBench76%—
VPCT46.2%—
Blueprint-Bench 2—33.6%
Furniture Assembly—56.7%
LMArena Document—1483
SpatialViz-Bench36.9%—

Multilingual GPT-5.6 Sol leads

Gemini 2.5 Flash: 52.3 (#88), GPT-5.6 Sol: 55.3 (#32)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
LMArena Non-English14091452
LMArena Chinese14501527
LMArena French14331477
LMArena German14181476
LMArena Japanese14051471
LMArena Korean13851442
LMArena Russian14151468
LMArena Spanish14211441

Instruction Following GPT-5.6 Sol leads

Gemini 2.5 Flash: 75.7 (#54), GPT-5.6 Sol: 77.7 (#16)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
LMArena Instruction Following14051482
IFEval89.8%—

Long Context Gemini 2.5 Flash leads

Gemini 2.5 Flash: 47.5 (#17), GPT-5.6 Sol: 45.4 (#42)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
LMArena Longer Query14191480
Fiction.LiveBench77.8%—

Writing & Preference GPT-5.6 Sol leads

Gemini 2.5 Flash: 53.8 (#157), GPT-5.6 Sol: 73.3 (#12)

BenchmarkGemini 2.5 FlashGPT-5.6 Sol
LMArena Text14171457
LMArena Creative Writing14001448
EQ-Bench Creative Writing11371972
LMArena Multi-Turn14081460
Short-Story Creative Writing76.5%—
WildBench81.7%—
EQ-Bench 4—1250

Frequently asked questions

Is Gemini 2.5 Flash better than GPT-5.6 Sol?

GPT-5.6 Sol is the stronger model overall, scoring 65.0 to 39.3 on the Noometry Index. Gemini 2.5 Flash costs 9.4× less per token, which makes it the better buy when GPT-5.6 Sol's lead doesn't matter for your workload.

Which is cheaper, Gemini 2.5 Flash or GPT-5.6 Sol?

Gemini 2.5 Flash is cheaper. It lists at $0.30 per million input tokens and $2.50 per million output tokens; GPT-5.6 Sol lists at $4 and $20.

Is Gemini 2.5 Flash or GPT-5.6 Sol better for coding?

GPT-5.6 Sol scores higher on coding benchmarks: 65.1 versus 35.8 in the Noometry coding category.

Which has the bigger context window?

GPT-5.6 Sol does, with 1.05M tokens against 1.05M.

How many benchmarks do Gemini 2.5 Flash and GPT-5.6 Sol share?

34 benchmarks have published results for both models. Gemini 2.5 Flash has 54 scored results on Noometry and GPT-5.6 Sol has 65.

Related comparisons

Go deeper