Noometry

Model comparison

Gemma 2 9B vs Step 1o Turbo 202506

Step 1o Turbo 202506 is the stronger model overall, scoring 39.7 to 25.9 on the Noometry Index.

Last verified . 13 shared benchmarks.

Gemma 2 9B Google

25.9

Rank #341 Confirmed

Step 1o Turbo 202506 StepFun

39.7

Rank #160 Confirmed

Summary

  • They share 13 benchmarks with published results for both. Gemma 2 9B scores higher in 0 categories and Step 1o Turbo 202506 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Step 1o Turbo 202506 leads 36.6 to 9.9.
  • Gemma 2 9B has downloadable open weights; the other is API-only.

Side by side

Gemma 2 9BStep 1o Turbo 202506
ProviderGoogleStepFun
Noometry Index25.939.7
Released2024-06-24—
WeightsOpenProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3514

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Step 1o Turbo 202506 leads

Gemma 2 9B: 29.4 (#304), Step 1o Turbo 202506: 39.2 (#160)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Coding11731339
BigCodeBench Instruct34.7%—
LiveBench Coding22.5%—
BigCodeBench Complete40.6%—

Reasoning Step 1o Turbo 202506 leads

Gemma 2 9B: 15.9 (#309), Step 1o Turbo 202506: 26.8 (#129)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Hard Prompts11711335
LiveBench Reasoning15.2%—
LiveBench Data Analysis36.4%—
Epoch Capabilities Index119.83—
LiveBench28.7%—
PIQA83.7%—

Math Step 1o Turbo 202506 leads

Gemma 2 9B: 9.9 (#318), Step 1o Turbo 202506: 36.6 (#164)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Math11831318
OTIS Mock AIME 2024-20250.6%—
LiveBench Math19.8%—
MATH Level 521%—
GSM8K84.9%—

Knowledge Step 1o Turbo 202506 leads

Gemma 2 9B: 9.7 (#305), Step 1o Turbo 202506: 36.1 (#176)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Expert11471308
GPQA Diamond27.5%—
BoolQ85.7%—
MMLU72.1%—

Multimodal Not comparable

Gemma 2 9B: —, Step 1o Turbo 202506: 36.1 (#80)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Vision—1186

Multilingual Step 1o Turbo 202506 leads

Gemma 2 9B: 36.6 (#238), Step 1o Turbo 202506: 45.3 (#173)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Non-English11881313
LMArena Chinese11851380
LMArena German11861314
LMArena Russian12001327
LMArena French1190—
LMArena Japanese1144—
LMArena Korean1137—
LMArena Spanish1200—

Instruction Following Step 1o Turbo 202506 leads

Gemma 2 9B: 57.6 (#269), Step 1o Turbo 202506: 69.2 (#176)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Instruction Following11781310
LiveBench Instruction Following52.6%—

Long Context Step 1o Turbo 202506 leads

Gemma 2 9B: 36.3 (#233), Step 1o Turbo 202506: 41.0 (#146)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Longer Query11971348

Writing & Preference Step 1o Turbo 202506 leads

Gemma 2 9B: 32.1 (#281), Step 1o Turbo 202506: 53.1 (#160)

BenchmarkGemma 2 9BStep 1o Turbo 202506
LMArena Text12071336
LMArena Creative Writing12061307
LMArena Multi-Turn11931340
EQ-Bench Creative Writing841—
LiveBench Language25.5%—

Frequently asked questions

Is Gemma 2 9B better than Step 1o Turbo 202506?

Step 1o Turbo 202506 is the stronger model overall, scoring 39.7 to 25.9 on the Noometry Index.

Is Gemma 2 9B or Step 1o Turbo 202506 better for coding?

Step 1o Turbo 202506 scores higher on coding benchmarks: 39.2 versus 29.4 in the Noometry coding category.

How many benchmarks do Gemma 2 9B and Step 1o Turbo 202506 share?

13 benchmarks have published results for both models. Gemma 2 9B has 35 scored results on Noometry and Step 1o Turbo 202506 has 14.

Related comparisons

Go deeper