Noometry

Model comparison

Mixtral 8x22B vs Step 3.7 Flash

Step 3.7 Flash is the stronger model overall, scoring 37.3 to 27.1 on the Noometry Index.

Last verified . 0 shared benchmarks.

Mixtral 8x22B Mistral AI

27.1

Rank #333 Confirmed

Step 3.7 Flash StepFun

37.3

Rank #207 Reported

Summary

  • The widest gap is in math, where Step 3.7 Flash leads 42.9 to 22.9.
  • Step 3.7 Flash is cheaper at $0.18 / $1.11 per million input/output tokens, against $2 / $6 for Mixtral 8x22B.
  • Step 3.7 Flash accepts more context: 256K tokens versus 64K.

Side by side

Mixtral 8x22BStep 3.7 Flash
ProviderMistral AIStepFun
Noometry Index27.137.3
Released2024-04-172026-05-29
WeightsOpenOpen
Context window64K256K
Max output64K256K
Input $ / M tokens$2$0.18
Output $ / M tokens$6$1.11
Results tracked345

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Step 3.7 Flash leads

Mixtral 8x22B: 24.2 (#329), Step 3.7 Flash: 40.0 (#150)

BenchmarkMixtral 8x22BStep 3.7 Flash
SciCode—40%
WeirdML3.2%—
BigCodeBench Instruct40.6%—
LMArena Coding1166—
BigCodeBench Complete50.2%—
ALE-Bench—694.12
HumanEval+72%—
MBPP+64.3%—

Agentic & Tool Use Not comparable

Mixtral 8x22B: 23.1 (#127), Step 3.7 Flash: —

BenchmarkMixtral 8x22BStep 3.7 Flash
Cybench7.5%—

Reasoning Step 3.7 Flash leads

Mixtral 8x22B: 19.9 (#248), Step 3.7 Flash: 21.6 (#219)

BenchmarkMixtral 8x22BStep 3.7 Flash
NYT Connections (extended)—39.7%
CritPt—2.3%
LMArena Hard Prompts1150—
DTBench55.1%—
Epoch Capabilities Index122.03—
ForecastBench56.3—

Math Step 3.7 Flash leads

Mixtral 8x22B: 22.9 (#275), Step 3.7 Flash: 42.9 (#82)

BenchmarkMixtral 8x22BStep 3.7 Flash
MathArena Final-Answer Competitions—68.5%
Omni-MATH16.3%—
LMArena Math1184—
MATH Level 524.2%—

Knowledge Not comparable

Mixtral 8x22B: 15.1 (#293), Step 3.7 Flash: —

BenchmarkMixtral 8x22BStep 3.7 Flash
GPQA Diamond34.1%—
MMLU-Pro46%—
GPQA (HELM)33.4%—
LMArena Expert1113—
MMLU77.8%—

Multilingual Not comparable

Mixtral 8x22B: 32.8 (#255), Step 3.7 Flash: —

BenchmarkMixtral 8x22BStep 3.7 Flash
LMArena Non-English1128—
LMArena Chinese1116—
LMArena French1166—
LMArena German1141—
LMArena Japanese1037—
LMArena Korean1057—
LMArena Russian1158—
LMArena Spanish1151—

Instruction Following Not comparable

Mixtral 8x22B: 57.7 (#266), Step 3.7 Flash: —

BenchmarkMixtral 8x22BStep 3.7 Flash
IFEval72.4%—
LMArena Instruction Following1147—

Long Context Not comparable

Mixtral 8x22B: 34.7 (#247), Step 3.7 Flash: —

BenchmarkMixtral 8x22BStep 3.7 Flash
LMArena Longer Query1144—

Writing & Preference Not comparable

Mixtral 8x22B: 36.9 (#262), Step 3.7 Flash: —

BenchmarkMixtral 8x22BStep 3.7 Flash
LMArena Text1162—
LMArena Creative Writing1141—
WildBench71.1%—
LMArena Multi-Turn1130—

Frequently asked questions

Is Mixtral 8x22B better than Step 3.7 Flash?

Step 3.7 Flash is the stronger model overall, scoring 37.3 to 27.1 on the Noometry Index.

Which is cheaper, Mixtral 8x22B or Step 3.7 Flash?

Step 3.7 Flash is cheaper. It lists at $0.18 per million input tokens and $1.11 per million output tokens; Mixtral 8x22B lists at $2 and $6.

Is Mixtral 8x22B or Step 3.7 Flash better for coding?

Step 3.7 Flash scores higher on coding benchmarks: 40.0 versus 24.2 in the Noometry coding category.

Which has the bigger context window?

Step 3.7 Flash does, with 256K tokens against 64K.

How many benchmarks do Mixtral 8x22B and Step 3.7 Flash share?

0 benchmarks have published results for both models. Mixtral 8x22B has 34 scored results on Noometry and Step 3.7 Flash has 5.

Related comparisons

Go deeper