Noometry

Model comparison

GPT-4 Turbo vs Step 3.5 Flash

Step 3.5 Flash is the stronger model overall, scoring 42.3 to 30.5 on the Noometry Index.

Last verified . 17 shared benchmarks.

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Step 3.5 Flash StepFun

42.3

Rank #116 Confirmed

Summary

  • They share 17 benchmarks with published results for both. GPT-4 Turbo scores higher in 0 categories and Step 3.5 Flash in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Step 3.5 Flash leads 42.6 to 9.0.
  • Step 3.5 Flash is cheaper at $0.10 / $0.30 per million input/output tokens, against $10 / $30 for GPT-4 Turbo.
  • Step 3.5 Flash accepts more context: 256K tokens versus 128K.
  • Step 3.5 Flash has downloadable open weights; the other is API-only.

Side by side

GPT-4 TurboStep 3.5 Flash
ProviderOpenAIStepFun
Noometry Index30.542.3
Released2023-11-062026-01-29
WeightsProprietaryOpen
Context window128K256K
Max output4K256K
Input $ / M tokens$10$0.10
Output $ / M tokens$30$0.30
Results tracked3619

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Step 3.5 Flash leads

GPT-4 Turbo: 33.8 (#249), Step 3.5 Flash: 42.4 (#105)

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Coding12681436
WeirdML18%—
BigCodeBench Instruct48.2%—
BigCodeBench Complete58.2%—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Step 3.5 Flash: —

BenchmarkGPT-4 TurboStep 3.5 Flash
METR Time Horizons36.7%—

Reasoning Step 3.5 Flash leads

GPT-4 Turbo: 15.3 (#317), Step 3.5 Flash: 22.2 (#202)

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Hard Prompts12511411
SimpleBench25.1%—
NYT Connections (extended)—28.4%
Chess Puzzles6%—
DTBench61.6%—
LMCA9.8%—
Epoch Capabilities Index127.25—
ForecastBench59.4—

Math Step 3.5 Flash leads

GPT-4 Turbo: 9.0 (#322), Step 3.5 Flash: 42.6 (#84)

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Math12721408
FrontierMath (Tiers 1-3)0.7%—
MathArena Final-Answer Competitions—66.8%
OTIS Mock AIME 2024-20256.7%—
MATH Level 546.7%—

Knowledge Step 3.5 Flash leads

GPT-4 Turbo: 24.3 (#268), Step 3.5 Flash: 39.6 (#132)

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Expert12231421
GPQA Diamond46.6%—
Confabulations28.4%—
MMLU81.3%—

Multimodal Not comparable

GPT-4 Turbo: 30.6 (#110), Step 3.5 Flash: —

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Vision1090—

Multilingual Step 3.5 Flash leads

GPT-4 Turbo: 40.5 (#216), Step 3.5 Flash: 50.5 (#119)

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Non-English12451385
LMArena Chinese12421447
LMArena French12761421
LMArena German12591405
LMArena Japanese11941354
LMArena Korean11871352
LMArena Russian12591385
LMArena Spanish12601419

Instruction Following Step 3.5 Flash leads

GPT-4 Turbo: 65.8 (#216), Step 3.5 Flash: 73.1 (#124)

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Instruction Following12491385

Long Context Step 3.5 Flash leads

GPT-4 Turbo: 38.0 (#206), Step 3.5 Flash: 42.8 (#117)

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Longer Query12541402

Writing & Preference Step 3.5 Flash leads

GPT-4 Turbo: 47.7 (#206), Step 3.5 Flash: 58.8 (#113)

BenchmarkGPT-4 TurboStep 3.5 Flash
LMArena Text12721403
LMArena Creative Writing12691357
LMArena Multi-Turn12671405

Frequently asked questions

Is GPT-4 Turbo better than Step 3.5 Flash?

Step 3.5 Flash is the stronger model overall, scoring 42.3 to 30.5 on the Noometry Index.

Which is cheaper, GPT-4 Turbo or Step 3.5 Flash?

Step 3.5 Flash is cheaper. It lists at $0.10 per million input tokens and $0.30 per million output tokens; GPT-4 Turbo lists at $10 and $30.

Is GPT-4 Turbo or Step 3.5 Flash better for coding?

Step 3.5 Flash scores higher on coding benchmarks: 42.4 versus 33.8 in the Noometry coding category.

Which has the bigger context window?

Step 3.5 Flash does, with 256K tokens against 128K.

How many benchmarks do GPT-4 Turbo and Step 3.5 Flash share?

17 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Step 3.5 Flash has 19.

Related comparisons

Go deeper