Noometry

Model comparison

Gemma 2B vs Muse Spark 1.1

Muse Spark 1.1 is the stronger model overall, scoring 49.9 to 29.6 on the Noometry Index.

Last verified . 12 shared benchmarks.

Gemma 2B Google

29.6

Rank #307 Confirmed

Muse Spark 1.1 Meta

49.9

Rank #51 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Gemma 2B scores higher in 0 categories and Muse Spark 1.1 in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Muse Spark 1.1 leads 73.4 to 24.0.
  • Gemma 2B has downloadable open weights; the other is API-only.

Side by side

Gemma 2BMuse Spark 1.1
ProviderGoogleMeta
Noometry Index29.649.9
Released2024-02-212026-04-08
WeightsOpenProprietary
Context window—1.05M
Max output—131K
Input $ / M tokens—$1.25
Output $ / M tokens—$4.25
Results tracked2337

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Muse Spark 1.1 leads

Gemma 2B: 29.4 (#305), Muse Spark 1.1: 51.3 (#40)

BenchmarkGemma 2BMuse Spark 1.1
LMArena Coding10101498
DeepSWE—53.3%
LMArena WebDev—1542
SciCode—58.8%
HumanEval+20.7%—
MBPP+34.1%—

Agentic & Tool Use Not comparable

Gemma 2B: —, Muse Spark 1.1: 30.8 (#73)

BenchmarkGemma 2BMuse Spark 1.1
APEX-Agents—31.8%
τ²-bench Banking—40.5%
GBAEval—7.9%
GDP.pdf—15%
Vending-Bench 2—6,520

Reasoning Muse Spark 1.1 leads

Gemma 2B: 18.8 (#275), Muse Spark 1.1: 47.1 (#44)

BenchmarkGemma 2BMuse Spark 1.1
LMArena Hard Prompts9891486
Epoch Capabilities Index94.2154.21
NYT Connections (extended)—84.9%
CritPt—15.1%
DTBench—94.4%
LMCA—49.9%
Surface Evolver Bench—52.5%
BIG-Bench Hard35.2%—
HellaSwag71.4%—
PIQA77.3%—
WinoGrande65.4%—

Math Muse Spark 1.1 leads

Gemma 2B: 30.0 (#239), Muse Spark 1.1: 45.5 (#76)

BenchmarkGemma 2BMuse Spark 1.1
LMArena Math10091483
ProofBench—39%
GSM8K17.7%—

Knowledge Not comparable

Gemma 2B: —, Muse Spark 1.1: 53.1 (#59)

BenchmarkGemma 2BMuse Spark 1.1
SimpleQA Verified—57.8%
LMArena Expert—1478
ARC (AI2) Challenge42.1%—
BoolQ69.4%—
MMLU42.3%—
TriviaQA53.2%—

Multimodal Not comparable

Gemma 2B: —, Muse Spark 1.1: 42.6 (#29)

BenchmarkGemma 2BMuse Spark 1.1
LMArena Vision—1293
LMArena Document—1465

Multilingual Muse Spark 1.1 leads

Gemma 2B: 23.0 (#294), Muse Spark 1.1: 56.7 (#17)

BenchmarkGemma 2BMuse Spark 1.1
LMArena Non-English9581472
LMArena Chinese9861518
LMArena Russian9371483
LMArena French—1494
LMArena German—1466
LMArena Japanese—1451
LMArena Korean—1458
LMArena Spanish—1464

Instruction Following Muse Spark 1.1 leads

Gemma 2B: 48.5 (#302), Muse Spark 1.1: 76.5 (#39)

BenchmarkGemma 2BMuse Spark 1.1
LMArena Instruction Following9701457

Long Context Muse Spark 1.1 leads

Gemma 2B: 29.9 (#291), Muse Spark 1.1: 44.8 (#58)

BenchmarkGemma 2BMuse Spark 1.1
LMArena Longer Query9811462

Writing & Preference Muse Spark 1.1 leads

Gemma 2B: 24.0 (#308), Muse Spark 1.1: 73.4 (#11)

BenchmarkGemma 2BMuse Spark 1.1
LMArena Text10021479
LMArena Creative Writing9871437
LMArena Multi-Turn9451485
EQ-Bench Creative Writing—1927
EQ-Bench 4—1260

Frequently asked questions

Is Gemma 2B better than Muse Spark 1.1?

Muse Spark 1.1 is the stronger model overall, scoring 49.9 to 29.6 on the Noometry Index.

Is Gemma 2B or Muse Spark 1.1 better for coding?

Muse Spark 1.1 scores higher on coding benchmarks: 51.3 versus 29.4 in the Noometry coding category.

How many benchmarks do Gemma 2B and Muse Spark 1.1 share?

12 benchmarks have published results for both models. Gemma 2B has 23 scored results on Noometry and Muse Spark 1.1 has 37.

Related comparisons

Go deeper