Model comparison

Gemini 1.0 Pro vs Muse Glimmer

Muse Glimmer is the stronger model overall, scoring 41.7 to 27.3 on the Noometry Index.

Last verified . 12 shared benchmarks.

Gemini 1.0 Pro Google

27.3

Rank #332 Confirmed

Muse Glimmer Meta

41.7

Rank #131 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Gemini 1.0 Pro scores higher in 0 categories and Muse Glimmer in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Muse Glimmer leads 38.8 to 9.3.
  • Muse Glimmer has downloadable open weights; the other is API-only.

Side by side

Gemini 1.0 Pro and Muse Glimmer specifications
Gemini 1.0 ProMuse Glimmer
ProviderGoogleMeta
Noometry Index27.341.7
Released2023-12-132026-08-10
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2415

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Muse Glimmer leads

Gemini 1.0 Pro: 32.2 (#275), Muse Glimmer: 40.4 (#142)

Coding benchmarks
BenchmarkGemini 1.0 ProMuse Glimmer
LMArena Coding11081416
LMArena WebDev—1355
SciCode—44.9%
HumanEval+55.5%—
MBPP+61.4%—

Reasoning Muse Glimmer leads

Gemini 1.0 Pro: 17.1 (#296), Muse Glimmer: 25.7 (#144)

Reasoning benchmarks
BenchmarkGemini 1.0 ProMuse Glimmer
LMArena Hard Prompts11091396
CritPt—2.6%
DTBench45.9%—
Epoch Capabilities Index117.04—

Math Muse Glimmer leads

Gemini 1.0 Pro: 9.3 (#321), Muse Glimmer: 38.8 (#125)

Math benchmarks
BenchmarkGemini 1.0 ProMuse Glimmer
LMArena Math11321417
OTIS Mock AIME 2024-20251.1%—
MATH Level 511.2%—

Knowledge Muse Glimmer leads

Gemini 1.0 Pro: 15.6 (#291), Muse Glimmer: 38.5 (#143)

Knowledge benchmarks
BenchmarkGemini 1.0 ProMuse Glimmer
LMArena Expert10591386
GPQA Diamond34%—
MMLU70%—

Multilingual Muse Glimmer leads

Gemini 1.0 Pro: 33.4 (#252), Muse Glimmer: 50.5 (#122)

Multilingual benchmarks
BenchmarkGemini 1.0 ProMuse Glimmer
LMArena Non-English11381384
LMArena Chinese11241412
LMArena Russian11861390
LMArena French1145—
LMArena German1125—
LMArena Japanese1023—
LMArena Spanish1119—

Instruction Following Muse Glimmer leads

Gemini 1.0 Pro: 57.6 (#267), Muse Glimmer: 72.6 (#135)

Instruction Following benchmarks
BenchmarkGemini 1.0 ProMuse Glimmer
LMArena Instruction Following11141375

Long Context Muse Glimmer leads

Gemini 1.0 Pro: 34.3 (#249), Muse Glimmer: 42.1 (#129)

Long Context benchmarks
BenchmarkGemini 1.0 ProMuse Glimmer
LMArena Longer Query11321382

Writing & Preference Muse Glimmer leads

Gemini 1.0 Pro: 36.0 (#264), Muse Glimmer: 57.5 (#126)

Writing & Preference benchmarks
BenchmarkGemini 1.0 ProMuse Glimmer
LMArena Text11491389
LMArena Creative Writing11311339
LMArena Multi-Turn11391399

Frequently asked questions

Is Gemini 1.0 Pro better than Muse Glimmer?

Muse Glimmer is the stronger model overall, scoring 41.7 to 27.3 on the Noometry Index.

Is Gemini 1.0 Pro or Muse Glimmer better for coding?

Muse Glimmer scores higher on coding benchmarks: 40.4 versus 32.2 in the Noometry coding category.

How many benchmarks do Gemini 1.0 Pro and Muse Glimmer share?

12 benchmarks have published results for both models. Gemini 1.0 Pro has 24 scored results on Noometry and Muse Glimmer has 15.

Related comparisons

Go deeper