Noometry

Model comparison

Mixtral 8x22B vs Solar Pro4

Solar Pro4 is the stronger model overall, scoring 42.1 to 27.1 on the Noometry Index.

Last verified . 17 shared benchmarks.

Mixtral 8x22B Mistral AI

27.1

Rank #333 Confirmed

Solar Pro4 Upstage

42.1

Rank #121 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Mixtral 8x22B scores higher in 0 categories and Solar Pro4 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Solar Pro4 leads 39.8 to 15.1.
  • Solar Pro4 is cheaper at $0.30 / $1.20 per million input/output tokens, against $2 / $6 for Mixtral 8x22B.
  • Solar Pro4 accepts more context: 524K tokens versus 64K.
  • Mixtral 8x22B has downloadable open weights; the other is API-only.

Side by side

Mixtral 8x22BSolar Pro4
ProviderMistral AIUpstage
Noometry Index27.142.1
Released2024-04-172026-08-06
WeightsOpenProprietary
Context window64K524K
Max output64K131K
Input $ / M tokens$2$0.30
Output $ / M tokens$6$1.20
Results tracked3418

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Solar Pro4 leads

Mixtral 8x22B: 24.2 (#329), Solar Pro4: 40.1 (#149)

BenchmarkMixtral 8x22BSolar Pro4
LMArena Coding11661437
LMArena WebDev—1371
WeirdML3.2%—
BigCodeBench Instruct40.6%—
BigCodeBench Complete50.2%—
HumanEval+72%—
MBPP+64.3%—

Agentic & Tool Use Not comparable

Mixtral 8x22B: 23.1 (#127), Solar Pro4: —

BenchmarkMixtral 8x22BSolar Pro4
Cybench7.5%—

Reasoning Solar Pro4 leads

Mixtral 8x22B: 19.9 (#248), Solar Pro4: 28.5 (#104)

BenchmarkMixtral 8x22BSolar Pro4
LMArena Hard Prompts11501399
DTBench55.1%—
Epoch Capabilities Index122.03—
ForecastBench56.3—

Math Solar Pro4 leads

Mixtral 8x22B: 22.9 (#275), Solar Pro4: 38.8 (#128)

BenchmarkMixtral 8x22BSolar Pro4
LMArena Math11841416
Omni-MATH16.3%—
MATH Level 524.2%—

Knowledge Solar Pro4 leads

Mixtral 8x22B: 15.1 (#293), Solar Pro4: 39.8 (#129)

BenchmarkMixtral 8x22BSolar Pro4
LMArena Expert11131427
GPQA Diamond34.1%—
MMLU-Pro46%—
GPQA (HELM)33.4%—
MMLU77.8%—

Multilingual Solar Pro4 leads

Mixtral 8x22B: 32.8 (#255), Solar Pro4: 48.7 (#139)

BenchmarkMixtral 8x22BSolar Pro4
LMArena Non-English11281361
LMArena Chinese11161415
LMArena French11661397
LMArena German11411364
LMArena Japanese10371309
LMArena Korean10571382
LMArena Russian11581360
LMArena Spanish11511401

Instruction Following Solar Pro4 leads

Mixtral 8x22B: 57.7 (#266), Solar Pro4: 72.7 (#132)

BenchmarkMixtral 8x22BSolar Pro4
LMArena Instruction Following11471377
IFEval72.4%—

Long Context Solar Pro4 leads

Mixtral 8x22B: 34.7 (#247), Solar Pro4: 42.1 (#130)

BenchmarkMixtral 8x22BSolar Pro4
LMArena Longer Query11441381

Writing & Preference Solar Pro4 leads

Mixtral 8x22B: 36.9 (#262), Solar Pro4: 56.5 (#138)

BenchmarkMixtral 8x22BSolar Pro4
LMArena Text11621386
LMArena Creative Writing11411316
LMArena Multi-Turn11301385
WildBench71.1%—

Frequently asked questions

Is Mixtral 8x22B better than Solar Pro4?

Solar Pro4 is the stronger model overall, scoring 42.1 to 27.1 on the Noometry Index.

Which is cheaper, Mixtral 8x22B or Solar Pro4?

Solar Pro4 is cheaper. It lists at $0.30 per million input tokens and $1.20 per million output tokens; Mixtral 8x22B lists at $2 and $6.

Is Mixtral 8x22B or Solar Pro4 better for coding?

Solar Pro4 scores higher on coding benchmarks: 40.1 versus 24.2 in the Noometry coding category.

Which has the bigger context window?

Solar Pro4 does, with 524K tokens against 64K.

How many benchmarks do Mixtral 8x22B and Solar Pro4 share?

17 benchmarks have published results for both models. Mixtral 8x22B has 34 scored results on Noometry and Solar Pro4 has 18.

Related comparisons

Go deeper