Noometry

Model comparison

GPT-4 Turbo vs GPT-6 Sol

GPT-6 Sol is the stronger model overall, scoring 49.2 to 30.3 on the Noometry Index.

Last verified . 13 shared benchmarks.

GPT-4 Turbo OpenAI

30.3

Rank #281 Confirmed

GPT-6 Sol OpenAI

49.2

Rank #46 Confirmed

Summary

  • They share 13 benchmarks with published results for both. GPT-4 Turbo scores higher in 0 categories and GPT-6 Sol in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GPT-6 Sol leads 49.3 to 14.9.
  • GPT-6 Sol is cheaper at $2 / $10 per million input/output tokens, against $10 / $30 for GPT-4 Turbo.
  • GPT-6 Sol accepts more context: 1.05M tokens versus 128K.

Side by side

GPT-4 TurboGPT-6 Sol
ProviderOpenAIOpenAI
Noometry Index30.349.2
Released2023-11-062026-09-22
WeightsProprietaryProprietary
Context window128K1.05M
Max output4K128K
Input $ / M tokens$10$2
Output $ / M tokens$30$10
Results tracked3622

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-6 Sol leads

GPT-4 Turbo: 33.4 (#247), GPT-6 Sol: 59.6 (#11)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Coding12681453
DeepSWE—68.8%
FrontierCode—49.3%
LMArena WebDev—1692
WeirdML18%—
BigCodeBench Instruct48.2%—
BigCodeBench Complete58.2%—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, GPT-6 Sol: 33.5 (#49)

BenchmarkGPT-4 TurboGPT-6 Sol
GDP.pdf—28%
METR Time Horizons36.7%—

Reasoning GPT-6 Sol leads

GPT-4 Turbo: 14.9 (#311), GPT-6 Sol: 49.3 (#34)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Hard Prompts12511420
SimpleBench25.1%—
NYT Connections (extended)—90.1%
Chess Puzzles6%—
EBR-Bench—53.3%
DTBench61.6%—
LMCA9.8%—
Epoch Capabilities Index127.25—
ForecastBench59.4—

Math GPT-6 Sol leads

GPT-4 Turbo: 8.6 (#316), GPT-6 Sol: 35.8 (#135)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Math12721404
FrontierMath (Tiers 1-3)0.7%—
OTIS Mock AIME 2024-20256.7%—
MATH Level 546.7%—

Knowledge GPT-6 Sol leads

GPT-4 Turbo: 24.3 (#263), GPT-6 Sol: 43.0 (#87)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Expert12231449
GPQA Diamond46.6%—
Confabulations28.4%—
Vectara Hallucination Rate—6.5%
MMLU81.3%—

Multimodal GPT-6 Sol leads

GPT-4 Turbo: 30.0 (#104), GPT-6 Sol: 47.6 (#9)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Vision1090—
Furniture Assembly—58.3%

Multilingual GPT-6 Sol leads

GPT-4 Turbo: 40.7 (#211), GPT-6 Sol: 51.0 (#113)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Non-English12451389
LMArena Chinese12421416
LMArena Russian12591407
LMArena Spanish12601377
LMArena French1275—
LMArena German1259—
LMArena Japanese1194—
LMArena Korean1187—

Instruction Following GPT-6 Sol leads

GPT-4 Turbo: 66.0 (#211), GPT-6 Sol: 74.4 (#97)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Instruction Following12501406

Long Context GPT-6 Sol leads

GPT-4 Turbo: 38.0 (#200), GPT-6 Sol: 43.2 (#100)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Longer Query12541415

Writing & Preference GPT-6 Sol leads

GPT-4 Turbo: 47.9 (#201), GPT-6 Sol: 72.3 (#15)

BenchmarkGPT-4 TurboGPT-6 Sol
LMArena Text12721396
LMArena Creative Writing12691392
LMArena Multi-Turn12671405
EQ-Bench Creative Writing—2125

Frequently asked questions

Is GPT-4 Turbo better than GPT-6 Sol?

GPT-6 Sol is the stronger model overall, scoring 49.2 to 30.3 on the Noometry Index.

Which is cheaper, GPT-4 Turbo or GPT-6 Sol?

GPT-6 Sol is cheaper. It lists at $2 per million input tokens and $10 per million output tokens; GPT-4 Turbo lists at $10 and $30.

Is GPT-4 Turbo or GPT-6 Sol better for coding?

GPT-6 Sol scores higher on coding benchmarks: 59.6 versus 33.4 in the Noometry coding category.

Which has the bigger context window?

GPT-6 Sol does, with 1.05M tokens against 128K.

How many benchmarks do GPT-4 Turbo and GPT-6 Sol share?

13 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and GPT-6 Sol has 22.

Related comparisons

Go deeper