Noometry

Model comparison

Mixtral 8x22B vs Step 3.5 Flash

Step 3.5 Flash is the stronger model overall, scoring 41.7 to 26.7 on the Noometry Index.

Last verified . 17 shared benchmarks.

Mixtral 8x22B Mistral AI

26.7

Rank #330 Confirmed

Step 3.5 Flash StepFun

41.7

Rank #116 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Mixtral 8x22B scores higher in 0 categories and Step 3.5 Flash in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Step 3.5 Flash leads 39.0 to 15.1.
  • Step 3.5 Flash is cheaper at $0.10 / $0.30 per million input/output tokens, against $2 / $6 for Mixtral 8x22B.
  • Step 3.5 Flash accepts more context: 256K tokens versus 64K.

Side by side

Mixtral 8x22BStep 3.5 Flash
ProviderMistral AIStepFun
Noometry Index26.741.7
Released2024-04-172026-01-29
WeightsOpenOpen
Context window64K256K
Max output64K256K
Input $ / M tokens$2$0.10
Output $ / M tokens$6$0.30
Results tracked3419

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Step 3.5 Flash leads

Mixtral 8x22B: 23.8 (#324), Step 3.5 Flash: 41.7 (#106)

BenchmarkMixtral 8x22BStep 3.5 Flash
LMArena Coding11661436
WeirdML3.2%—
BigCodeBench Instruct40.6%—
BigCodeBench Complete50.2%—
HumanEval+72%—
MBPP+64.3%—

Agentic & Tool Use Not comparable

Mixtral 8x22B: 22.8 (#119), Step 3.5 Flash: —

BenchmarkMixtral 8x22BStep 3.5 Flash
Cybench7.5%—

Reasoning Step 3.5 Flash leads

Mixtral 8x22B: 19.1 (#246), Step 3.5 Flash: 21.2 (#202)

BenchmarkMixtral 8x22BStep 3.5 Flash
LMArena Hard Prompts11501411
NYT Connections (extended)—28.4%
DTBench55.1%—
Epoch Capabilities Index122—
ForecastBench56.3—

Math Step 3.5 Flash leads

Mixtral 8x22B: 21.2 (#271), Step 3.5 Flash: 40.6 (#77)

BenchmarkMixtral 8x22BStep 3.5 Flash
LMArena Math11841408
MathArena Final-Answer Competitions—66.8%
Omni-MATH16.3%—
MATH Level 524.2%—

Knowledge Step 3.5 Flash leads

Mixtral 8x22B: 15.1 (#288), Step 3.5 Flash: 39.0 (#127)

BenchmarkMixtral 8x22BStep 3.5 Flash
LMArena Expert11131422
GPQA Diamond34.1%—
MMLU-Pro46%—
GPQA (HELM)33.4%—
MMLU77.8%—

Multilingual Step 3.5 Flash leads

Mixtral 8x22B: 32.9 (#250), Step 3.5 Flash: 50.7 (#115)

BenchmarkMixtral 8x22BStep 3.5 Flash
LMArena Non-English11281386
LMArena Chinese11161448
LMArena French11651421
LMArena German11411404
LMArena Japanese10371353
LMArena Korean10561350
LMArena Russian11581385
LMArena Spanish11501419

Instruction Following Step 3.5 Flash leads

Mixtral 8x22B: 57.9 (#261), Step 3.5 Flash: 73.3 (#119)

BenchmarkMixtral 8x22BStep 3.5 Flash
LMArena Instruction Following11471385
IFEval72.4%—

Long Context Step 3.5 Flash leads

Mixtral 8x22B: 34.6 (#242), Step 3.5 Flash: 42.8 (#112)

BenchmarkMixtral 8x22BStep 3.5 Flash
LMArena Longer Query11441402

Writing & Preference Step 3.5 Flash leads

Mixtral 8x22B: 37.1 (#257), Step 3.5 Flash: 59.0 (#108)

BenchmarkMixtral 8x22BStep 3.5 Flash
LMArena Text11621403
LMArena Creative Writing11411357
LMArena Multi-Turn11301405
WildBench71.1%—

Frequently asked questions

Is Mixtral 8x22B better than Step 3.5 Flash?

Step 3.5 Flash is the stronger model overall, scoring 41.7 to 26.7 on the Noometry Index.

Which is cheaper, Mixtral 8x22B or Step 3.5 Flash?

Step 3.5 Flash is cheaper. It lists at $0.10 per million input tokens and $0.30 per million output tokens; Mixtral 8x22B lists at $2 and $6.

Is Mixtral 8x22B or Step 3.5 Flash better for coding?

Step 3.5 Flash scores higher on coding benchmarks: 41.7 versus 23.8 in the Noometry coding category.

Which has the bigger context window?

Step 3.5 Flash does, with 256K tokens against 64K.

How many benchmarks do Mixtral 8x22B and Step 3.5 Flash share?

17 benchmarks have published results for both models. Mixtral 8x22B has 34 scored results on Noometry and Step 3.5 Flash has 19.

Related comparisons

Go deeper