Noometry

Model comparison

Llama 3.1-70B vs Step 3.5 Flash

Step 3.5 Flash is the stronger model overall, scoring 42.3 to 29.6 on the Noometry Index.

Last verified . 17 shared benchmarks.

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Step 3.5 Flash StepFun

42.3

Rank #116 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Llama 3.1-70B scores higher in 0 categories and Step 3.5 Flash in 8 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Step 3.5 Flash leads 42.6 to 13.5.
  • Step 3.5 Flash is cheaper at $0.10 / $0.30 per million input/output tokens, against $0.40 / $0.40 for Llama 3.1-70B.
  • Step 3.5 Flash accepts more context: 256K tokens versus 128K.

Side by side

Llama 3.1-70BStep 3.5 Flash
ProviderMetaStepFun
Noometry Index29.642.3
Released2024-07-232026-01-29
WeightsOpenOpen
Context window128K256K
Max output4K256K
Input $ / M tokens$0.40$0.10
Output $ / M tokens$0.40$0.30
Results tracked3519

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Step 3.5 Flash leads

Llama 3.1-70B: 30.3 (#296), Step 3.5 Flash: 42.4 (#105)

BenchmarkLlama 3.1-70BStep 3.5 Flash
LMArena Coding12601436
WeirdML9%—
BigCodeBench Instruct46.1%—
BigCodeBench Complete54.8%—

Agentic & Tool Use Not comparable

Llama 3.1-70B: 25.1 (#112), Step 3.5 Flash: —

BenchmarkLlama 3.1-70BStep 3.5 Flash
TheAgentCompany6.9%—
BALROG27.9%—

Reasoning Too close to call

Llama 3.1-70B: 21.6 (#220), Step 3.5 Flash: 22.2 (#202)

BenchmarkLlama 3.1-70BStep 3.5 Flash
LMArena Hard Prompts12411411
NYT Connections (extended)—28.4%
DTBench60%—
LMCA14.8%—
Epoch Capabilities Index125.92—

Math Step 3.5 Flash leads

Llama 3.1-70B: 13.5 (#304), Step 3.5 Flash: 42.6 (#84)

BenchmarkLlama 3.1-70BStep 3.5 Flash
LMArena Math12521408
MathArena Final-Answer Competitions—66.8%
OTIS Mock AIME 2024-20253.6%—
Omni-MATH21%—
MATH Level 536.7%—

Knowledge Step 3.5 Flash leads

Llama 3.1-70B: 24.2 (#269), Step 3.5 Flash: 39.6 (#132)

BenchmarkLlama 3.1-70BStep 3.5 Flash
LMArena Expert12091421
GPQA Diamond44.2%—
MMLU-Pro65.3%—
GPQA (HELM)42.6%—
MMLU80.1%—

Multilingual Step 3.5 Flash leads

Llama 3.1-70B: 38.8 (#225), Step 3.5 Flash: 50.5 (#119)

BenchmarkLlama 3.1-70BStep 3.5 Flash
LMArena Non-English12191385
LMArena Chinese12151447
LMArena French12611421
LMArena German12221405
LMArena Japanese11321354
LMArena Korean11401352
LMArena Russian12341385
LMArena Spanish12531419

Instruction Following Step 3.5 Flash leads

Llama 3.1-70B: 65.3 (#223), Step 3.5 Flash: 73.1 (#124)

BenchmarkLlama 3.1-70BStep 3.5 Flash
LMArena Instruction Following12311385
IFEval82.1%—

Long Context Step 3.5 Flash leads

Llama 3.1-70B: 37.6 (#214), Step 3.5 Flash: 42.8 (#117)

BenchmarkLlama 3.1-70BStep 3.5 Flash
LMArena Longer Query12411402

Writing & Preference Step 3.5 Flash leads

Llama 3.1-70B: 35.4 (#267), Step 3.5 Flash: 58.8 (#113)

BenchmarkLlama 3.1-70BStep 3.5 Flash
LMArena Text12611403
LMArena Creative Writing12321357
LMArena Multi-Turn12561405
EQ-Bench Creative Writing784—
WildBench75.8%—

Frequently asked questions

Is Llama 3.1-70B better than Step 3.5 Flash?

Step 3.5 Flash is the stronger model overall, scoring 42.3 to 29.6 on the Noometry Index.

Which is cheaper, Llama 3.1-70B or Step 3.5 Flash?

Step 3.5 Flash is cheaper. It lists at $0.10 per million input tokens and $0.30 per million output tokens; Llama 3.1-70B lists at $0.40 and $0.40.

Is Llama 3.1-70B or Step 3.5 Flash better for coding?

Step 3.5 Flash scores higher on coding benchmarks: 42.4 versus 30.3 in the Noometry coding category.

Which has the bigger context window?

Step 3.5 Flash does, with 256K tokens against 128K.

How many benchmarks do Llama 3.1-70B and Step 3.5 Flash share?

17 benchmarks have published results for both models. Llama 3.1-70B has 35 scored results on Noometry and Step 3.5 Flash has 19.

Related comparisons

Go deeper