Noometry

Model comparison

Mixtral 8x22B vs Step 5 Preview

Step 5 Preview is the stronger model overall, scoring 47.9 to 27.1 on the Noometry Index.

Last verified . 13 shared benchmarks.

Mixtral 8x22B Mistral AI

27.1

Rank #333 Confirmed

Step 5 Preview StepFun

47.9

Rank #58 Confirmed

Summary

  • They share 13 benchmarks with published results for both. Mixtral 8x22B scores higher in 0 categories and Step 5 Preview in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Step 5 Preview leads 51.8 to 24.2.
  • Step 5 Preview is cheaper at $1 / $2.70 per million input/output tokens, against $2 / $6 for Mixtral 8x22B.
  • Step 5 Preview accepts more context: 1.02M tokens versus 64K.
  • Mixtral 8x22B has downloadable open weights; the other is API-only.

Side by side

Mixtral 8x22BStep 5 Preview
ProviderMistral AIStepFun
Noometry Index27.147.9
Released2024-04-172026-09-16
WeightsOpenProprietary
Context window64K1.02M
Max output64K66K
Input $ / M tokens$2$1
Output $ / M tokens$6$2.70
Results tracked3418

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Step 5 Preview leads

Mixtral 8x22B: 24.2 (#329), Step 5 Preview: 51.8 (#36)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Coding11661480
LMArena WebDev—1564
SciCode—58.9%
WeirdML3.2%—
BigCodeBench Instruct40.6%—
BigCodeBench Complete50.2%—
HumanEval+72%—
MBPP+64.3%—

Agentic & Tool Use Not comparable

Mixtral 8x22B: 23.1 (#127), Step 5 Preview: —

BenchmarkMixtral 8x22BStep 5 Preview
Cybench7.5%—

Reasoning Step 5 Preview leads

Mixtral 8x22B: 19.9 (#248), Step 5 Preview: 40.0 (#57)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Hard Prompts11501465
CritPt—20.9%
DTBench55.1%—
Epoch Capabilities Index122.03—
ForecastBench56.3—

Math Step 5 Preview leads

Mixtral 8x22B: 22.9 (#275), Step 5 Preview: 46.1 (#72)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Math11841470
ProofBench—42%
Omni-MATH16.3%—
MATH Level 524.2%—

Knowledge Step 5 Preview leads

Mixtral 8x22B: 15.1 (#293), Step 5 Preview: 41.2 (#112)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Expert11131470
GPQA Diamond34.1%—
MMLU-Pro46%—
GPQA (HELM)33.4%—
MMLU77.8%—

Multimodal Not comparable

Mixtral 8x22B: —, Step 5 Preview: 41.0 (#41)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Vision—1267

Multilingual Step 5 Preview leads

Mixtral 8x22B: 32.8 (#255), Step 5 Preview: 53.8 (#54)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Non-English11281432
LMArena Chinese11161519
LMArena Russian11581436
LMArena Spanish11511439
LMArena French1166—
LMArena German1141—
LMArena Japanese1037—
LMArena Korean1057—

Instruction Following Step 5 Preview leads

Mixtral 8x22B: 57.7 (#266), Step 5 Preview: 76.0 (#50)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Instruction Following11471444
IFEval72.4%—

Long Context Step 5 Preview leads

Mixtral 8x22B: 34.7 (#247), Step 5 Preview: 44.5 (#64)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Longer Query11441455

Writing & Preference Step 5 Preview leads

Mixtral 8x22B: 36.9 (#262), Step 5 Preview: 62.8 (#71)

BenchmarkMixtral 8x22BStep 5 Preview
LMArena Text11621442
LMArena Creative Writing11411410
LMArena Multi-Turn11301447
WildBench71.1%—

Frequently asked questions

Is Mixtral 8x22B better than Step 5 Preview?

Step 5 Preview is the stronger model overall, scoring 47.9 to 27.1 on the Noometry Index.

Which is cheaper, Mixtral 8x22B or Step 5 Preview?

Step 5 Preview is cheaper. It lists at $1 per million input tokens and $2.70 per million output tokens; Mixtral 8x22B lists at $2 and $6.

Is Mixtral 8x22B or Step 5 Preview better for coding?

Step 5 Preview scores higher on coding benchmarks: 51.8 versus 24.2 in the Noometry coding category.

Which has the bigger context window?

Step 5 Preview does, with 1.02M tokens against 64K.

How many benchmarks do Mixtral 8x22B and Step 5 Preview share?

13 benchmarks have published results for both models. Mixtral 8x22B has 34 scored results on Noometry and Step 5 Preview has 18.

Related comparisons

Go deeper