Noometry

Model comparison

Mixtral 8x7B vs Step 3.5 Flash

Step 3.5 Flash is the stronger model overall, scoring 42.3 to 27.1 on the Noometry Index.

Last verified . 17 shared benchmarks.

Mixtral 8x7B Mistral AI

27.1

Rank #334 Confirmed

Step 3.5 Flash StepFun

42.3

Rank #116 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Mixtral 8x7B scores higher in 0 categories and Step 3.5 Flash in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Step 3.5 Flash leads 39.6 to 11.0.
  • Step 3.5 Flash is cheaper at $0.10 / $0.30 per million input/output tokens, against $0.70 / $0.70 for Mixtral 8x7B.
  • Step 3.5 Flash accepts more context: 256K tokens versus 32K.

Side by side

Mixtral 8x7BStep 3.5 Flash
ProviderMistral AIStepFun
Noometry Index27.142.3
Released2023-12-112026-01-29
WeightsOpenOpen
Context window32K256K
Max output32K256K
Input $ / M tokens$0.70$0.10
Output $ / M tokens$0.70$0.30
Results tracked3819

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Step 3.5 Flash leads

Mixtral 8x7B: 32.8 (#269), Step 3.5 Flash: 42.4 (#105)

BenchmarkMixtral 8x7BStep 3.5 Flash
LMArena Coding11261436
HumanEval+39.6%—
MBPP+49.7%—

Reasoning Step 3.5 Flash leads

Mixtral 8x7B: 18.2 (#285), Step 3.5 Flash: 22.2 (#202)

BenchmarkMixtral 8x7BStep 3.5 Flash
LMArena Hard Prompts11151411
NYT Connections (extended)—28.4%
DTBench49.6%—
Adversarial NLI55.2%—
Epoch Capabilities Index118.47—
ForecastBench56.3—
HellaSwag86.7%—
PIQA83.6%—
WinoGrande77.2%—

Math Step 3.5 Flash leads

Mixtral 8x7B: 18.8 (#289), Step 3.5 Flash: 42.6 (#84)

BenchmarkMixtral 8x7BStep 3.5 Flash
LMArena Math11471408
MathArena Final-Answer Competitions—66.8%
Omni-MATH10.5%—
MATH Level 510%—
GSM8K74.4%—

Knowledge Step 3.5 Flash leads

Mixtral 8x7B: 11.0 (#301), Step 3.5 Flash: 39.6 (#132)

BenchmarkMixtral 8x7BStep 3.5 Flash
LMArena Expert10881421
GPQA Diamond30.6%—
MMLU-Pro33.5%—
GPQA (HELM)29.6%—
ARC (AI2) Challenge87.3%—
MMLU70.6%—
OpenBookQA85.8%—
TriviaQA82.2%—

Multilingual Step 3.5 Flash leads

Mixtral 8x7B: 29.6 (#266), Step 3.5 Flash: 50.5 (#119)

BenchmarkMixtral 8x7BStep 3.5 Flash
LMArena Non-English10771385
LMArena Chinese10551447
LMArena French11661421
LMArena German11141405
LMArena Japanese9311354
LMArena Korean9681352
LMArena Russian10901385
LMArena Spanish11111419

Instruction Following Step 3.5 Flash leads

Mixtral 8x7B: 51.0 (#297), Step 3.5 Flash: 73.1 (#124)

BenchmarkMixtral 8x7BStep 3.5 Flash
LMArena Instruction Following11091385
IFEval57.5%—

Long Context Step 3.5 Flash leads

Mixtral 8x7B: 33.4 (#260), Step 3.5 Flash: 42.8 (#117)

BenchmarkMixtral 8x7BStep 3.5 Flash
LMArena Longer Query11031402

Writing & Preference Step 3.5 Flash leads

Mixtral 8x7B: 34.2 (#270), Step 3.5 Flash: 58.8 (#113)

BenchmarkMixtral 8x7BStep 3.5 Flash
LMArena Text11321403
LMArena Creative Writing11091357
LMArena Multi-Turn11151405
WildBench67.3%—

Frequently asked questions

Is Mixtral 8x7B better than Step 3.5 Flash?

Step 3.5 Flash is the stronger model overall, scoring 42.3 to 27.1 on the Noometry Index.

Which is cheaper, Mixtral 8x7B or Step 3.5 Flash?

Step 3.5 Flash is cheaper. It lists at $0.10 per million input tokens and $0.30 per million output tokens; Mixtral 8x7B lists at $0.70 and $0.70.

Is Mixtral 8x7B or Step 3.5 Flash better for coding?

Step 3.5 Flash scores higher on coding benchmarks: 42.4 versus 32.8 in the Noometry coding category.

Which has the bigger context window?

Step 3.5 Flash does, with 256K tokens against 32K.

How many benchmarks do Mixtral 8x7B and Step 3.5 Flash share?

17 benchmarks have published results for both models. Mixtral 8x7B has 38 scored results on Noometry and Step 3.5 Flash has 19.

Related comparisons

Go deeper