Noometry

Model comparison

Mixtral 8x7B vs Solar Pro4

Solar Pro4 is the stronger model overall, scoring 42.1 to 27.1 on the Noometry Index.

Last verified . 17 shared benchmarks.

Mixtral 8x7B Mistral AI

27.1

Rank #334 Confirmed

Solar Pro4 Upstage

42.1

Rank #121 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Mixtral 8x7B scores higher in 0 categories and Solar Pro4 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Solar Pro4 leads 39.8 to 11.0.
  • Solar Pro4 is cheaper at $0.30 / $1.20 per million input/output tokens, against $0.70 / $0.70 for Mixtral 8x7B.
  • Solar Pro4 accepts more context: 524K tokens versus 32K.
  • Mixtral 8x7B has downloadable open weights; the other is API-only.

Side by side

Mixtral 8x7BSolar Pro4
ProviderMistral AIUpstage
Noometry Index27.142.1
Released2023-12-112026-08-06
WeightsOpenProprietary
Context window32K524K
Max output32K131K
Input $ / M tokens$0.70$0.30
Output $ / M tokens$0.70$1.20
Results tracked3818

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Solar Pro4 leads

Mixtral 8x7B: 32.8 (#269), Solar Pro4: 40.1 (#149)

BenchmarkMixtral 8x7BSolar Pro4
LMArena Coding11261437
LMArena WebDev—1371
HumanEval+39.6%—
MBPP+49.7%—

Reasoning Solar Pro4 leads

Mixtral 8x7B: 18.2 (#285), Solar Pro4: 28.5 (#104)

BenchmarkMixtral 8x7BSolar Pro4
LMArena Hard Prompts11151399
DTBench49.6%—
Adversarial NLI55.2%—
Epoch Capabilities Index118.47—
ForecastBench56.3—
HellaSwag86.7%—
PIQA83.6%—
WinoGrande77.2%—

Math Solar Pro4 leads

Mixtral 8x7B: 18.8 (#289), Solar Pro4: 38.8 (#128)

BenchmarkMixtral 8x7BSolar Pro4
LMArena Math11471416
Omni-MATH10.5%—
MATH Level 510%—
GSM8K74.4%—

Knowledge Solar Pro4 leads

Mixtral 8x7B: 11.0 (#301), Solar Pro4: 39.8 (#129)

BenchmarkMixtral 8x7BSolar Pro4
LMArena Expert10881427
GPQA Diamond30.6%—
MMLU-Pro33.5%—
GPQA (HELM)29.6%—
ARC (AI2) Challenge87.3%—
MMLU70.6%—
OpenBookQA85.8%—
TriviaQA82.2%—

Multilingual Solar Pro4 leads

Mixtral 8x7B: 29.6 (#266), Solar Pro4: 48.7 (#139)

BenchmarkMixtral 8x7BSolar Pro4
LMArena Non-English10771361
LMArena Chinese10551415
LMArena French11661397
LMArena German11141364
LMArena Japanese9311309
LMArena Korean9681382
LMArena Russian10901360
LMArena Spanish11111401

Instruction Following Solar Pro4 leads

Mixtral 8x7B: 51.0 (#297), Solar Pro4: 72.7 (#132)

BenchmarkMixtral 8x7BSolar Pro4
LMArena Instruction Following11091377
IFEval57.5%—

Long Context Solar Pro4 leads

Mixtral 8x7B: 33.4 (#260), Solar Pro4: 42.1 (#130)

BenchmarkMixtral 8x7BSolar Pro4
LMArena Longer Query11031381

Writing & Preference Solar Pro4 leads

Mixtral 8x7B: 34.2 (#270), Solar Pro4: 56.5 (#138)

BenchmarkMixtral 8x7BSolar Pro4
LMArena Text11321386
LMArena Creative Writing11091316
LMArena Multi-Turn11151385
WildBench67.3%—

Frequently asked questions

Is Mixtral 8x7B better than Solar Pro4?

Solar Pro4 is the stronger model overall, scoring 42.1 to 27.1 on the Noometry Index.

Which is cheaper, Mixtral 8x7B or Solar Pro4?

Solar Pro4 is cheaper. It lists at $0.30 per million input tokens and $1.20 per million output tokens; Mixtral 8x7B lists at $0.70 and $0.70.

Is Mixtral 8x7B or Solar Pro4 better for coding?

Solar Pro4 scores higher on coding benchmarks: 40.1 versus 32.8 in the Noometry coding category.

Which has the bigger context window?

Solar Pro4 does, with 524K tokens against 32K.

How many benchmarks do Mixtral 8x7B and Solar Pro4 share?

17 benchmarks have published results for both models. Mixtral 8x7B has 38 scored results on Noometry and Solar Pro4 has 18.

Related comparisons

Go deeper