Noometry

Model comparison

Llama 3.1-405B vs Muse Spark

Muse Spark is the stronger model overall, scoring 50.6 to 30.7 on the Noometry Index.

Last verified . 19 shared benchmarks.

Llama 3.1-405B Meta

30.7

Rank #288 Confirmed

Muse Spark Meta

50.6

Rank #46 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Llama 3.1-405B scores higher in 0 categories and Muse Spark in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Muse Spark leads 65.7 to 30.4.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 9.7% for Llama 3.1-405B and 88.9% for Muse Spark.
  • Llama 3.1-405B has downloadable open weights; the other is API-only.

Side by side

Llama 3.1-405BMuse Spark
ProviderMetaMeta
Noometry Index30.750.6
Released2024-07-232026-04-08
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4227

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Muse Spark leads

Llama 3.1-405B: 33.1 (#262), Muse Spark: 46.2 (#69)

BenchmarkLlama 3.1-405BMuse Spark
LMArena Coding12911481
SciCode—51.5%
WeirdML21.4%—

Agentic & Tool Use Not comparable

Llama 3.1-405B: 21.0 (#140), Muse Spark: —

BenchmarkLlama 3.1-405BMuse Spark
TheAgentCompany7.4%—
Cybench7.5%—

Reasoning Muse Spark leads

Llama 3.1-405B: 16.8 (#300), Muse Spark: 35.9 (#67)

BenchmarkLlama 3.1-405BMuse Spark
LMArena Hard Prompts12691474
Epoch Capabilities Index128.75152.04
SimpleBench23%—
Kagi LLM Benchmark45%—
CritPt—11.3%
DTBench61.4%—
BIG-Bench Hard82.9%—
ForecastBench59.9—
HellaSwag89.2%—
PIQA85.9%—
WinoGrande89.2%—

Math Muse Spark leads

Llama 3.1-405B: 18.4 (#290), Muse Spark: 47.8 (#66)

BenchmarkLlama 3.1-405BMuse Spark
OTIS Mock AIME 2024-20259.7%88.9%
LMArena Math12811455
ProofBench—17%
Omni-MATH24.9%—
MATH Level 549.8%—
FrontierMath (Feb 2025 set)—39%
FrontierMath Tier 4 (v1)—14.6%

Knowledge Muse Spark leads

Llama 3.1-405B: 30.4 (#227), Muse Spark: 65.7 (#13)

BenchmarkLlama 3.1-405BMuse Spark
GPQA Diamond50.9%89.8%
LMArena Expert12431457
Humanity's Last Exam—40.6%
MMLU-Pro72.3%—
Confabulations17.6%—
GPQA (HELM)52.2%—
ARC (AI2) Challenge95.3%—
MMLU84.5%—
TriviaQA82.7%—

Multimodal Not comparable

Llama 3.1-405B: —, Muse Spark: 43.4 (#24)

BenchmarkLlama 3.1-405BMuse Spark
LMArena Vision—1306
LMArena Document—1444

Multilingual Muse Spark leads

Llama 3.1-405B: 40.7 (#214), Muse Spark: 56.1 (#24)

BenchmarkLlama 3.1-405BMuse Spark
LMArena Non-English12481464
LMArena Chinese12421509
LMArena French12791497
LMArena German12521497
LMArena Korean11841459
LMArena Russian12651466
LMArena Spanish12601472
LMArena Japanese1208—

Instruction Following Muse Spark leads

Llama 3.1-405B: 65.9 (#214), Muse Spark: 75.9 (#51)

BenchmarkLlama 3.1-405BMuse Spark
LMArena Instruction Following12591442
IFEval81.1%—

Long Context Muse Spark leads

Llama 3.1-405B: 38.4 (#197), Muse Spark: 44.4 (#69)

BenchmarkLlama 3.1-405BMuse Spark
LMArena Longer Query12661451

Writing & Preference Muse Spark leads

Llama 3.1-405B: 38.9 (#251), Muse Spark: 66.0 (#39)

BenchmarkLlama 3.1-405BMuse Spark
LMArena Text12841474
LMArena Creative Writing12621459
LMArena Multi-Turn12971477
EQ-Bench Creative Writing870—
WildBench78.3%—

Frequently asked questions

Is Llama 3.1-405B better than Muse Spark?

Muse Spark is the stronger model overall, scoring 50.6 to 30.7 on the Noometry Index.

Is Llama 3.1-405B or Muse Spark better for coding?

Muse Spark scores higher on coding benchmarks: 46.2 versus 33.1 in the Noometry coding category.

How many benchmarks do Llama 3.1-405B and Muse Spark share?

19 benchmarks have published results for both models. Llama 3.1-405B has 42 scored results on Noometry and Muse Spark has 27.

Related comparisons

Go deeper