Model comparison
Claude Opus 4 vs Gemma 4 31B IT
Claude Opus 4 and Gemma 4 31B IT score almost the same on the Noometry Index (43.1 vs 43.5), so choose on price, context window or the category you care about most.
Last verified . 25 shared benchmarks.
GoogleSummary
- They share 25 benchmarks with published results for both. Claude Opus 4 scores higher in 5 categories and Gemma 4 31B IT in 4 categories; 7 gaps are clear of the uncertainty.
- The widest gap is in multimodal, where Gemma 4 31B IT leads 41.6 to 31.5.
- The biggest single-benchmark swing is Kagi LLM Benchmark: 74.3% for Claude Opus 4 and 63.5% for Gemma 4 31B IT.
- Gemma 4 31B IT is cheaper at $0.09 / $0.34 per million input/output tokens, against $15 / $75 for Claude Opus 4.
- Gemma 4 31B IT accepts more context: 262K tokens versus 200K.
- Gemma 4 31B IT has downloadable open weights; the other is API-only.
Side by side
| Claude Opus 4 | Gemma 4 31B IT | |
|---|---|---|
| Provider | Anthropic | |
| Noometry Index | 43.1 | 43.5 |
| Released | 2025-05-22 | 2026-04-02 |
| Weights | Proprietary | Open |
| Context window | 200K | 262K |
| Max output | 32K | 33K |
| Input $ / M tokens | $15 | $0.09 |
| Output $ / M tokens | $75 | $0.34 |
| Results tracked | 56 | 35 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding Claude Opus 4 leads
Claude Opus 4: 47.2 (#62), Gemma 4 31B IT: 42.3 (#108)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| WeirdML | 43.7% | 52.3% |
| LMArena Coding | 1442 | 1459 |
| SWE-bench Verified | 70.7% | — |
| SWE-bench Verified (bash only) | 67.6% | — |
| Aider Polyglot | 72% | — |
| LMArena WebDev | — | 1366 |
| SciCode | — | 43.4% |
| GSO | 6.9% | — |
| ALE-Bench | — | 925.5 |
| AlgoTune | 1.33 | — |
Agentic & Tool Use Not comparable
Claude Opus 4: 34.8 (#42), Gemma 4 31B IT: —
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| Cybench | 38% | — |
| DeepResearch Bench | 46.8% | — |
| LMArena Search | 1127 | — |
| METR Time Horizons | 63.9% | — |
Reasoning Too close to call
Claude Opus 4: 27.3 (#121), Gemma 4 31B IT: 27.2 (#122)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| Kagi LLM Benchmark | 74.3% | 63.5% |
| CritPt | 0.3% | 1.4% |
| LMArena Hard Prompts | 1399 | 1448 |
| DTBench | 81.6% | 82.7% |
| LMCA | 37.4% | 39.3% |
| Epoch Capabilities Index | 142.67 | 142.74 |
| ARC-AGI-2 | 8.6% | — |
| SimpleBench | 58.8% | — |
| NYT Connections (extended) | — | 70.6% |
| ARC-AGI-1 | 35.7% | — |
| Chess Puzzles | — | 5% |
| EnigmaEval | 5.6% | — |
| Thematic Generalization | — | 53% |
| Surface Evolver Bench | — | 30.6% |
| ForecastBench | 61.1 | — |
Math Gemma 4 31B IT leads
Claude Opus 4: 42.0 (#86), Gemma 4 31B IT: 43.2 (#81)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| OTIS Mock AIME 2024-2025 | 64.4% | 73.3% |
| LMArena Math | 1390 | 1465 |
| Omni-MATH | 61.6% | — |
| MATH Level 5 | 85% | — |
| FrontierMath (Feb 2025 set) | 4.5% | — |
| FrontierMath Tier 4 (v1) | 4.2% | — |
Knowledge Claude Opus 4 leads
Claude Opus 4: 44.0 (#88), Gemma 4 31B IT: 37.9 (#151)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| GPQA Diamond | 76.3% | 75.8% |
| Vectara Hallucination Rate | 12% | 7.4% |
| LMArena Expert | 1386 | 1465 |
| Humanity's Last Exam | 10.7% | — |
| SimpleQA Verified | — | 10.4% |
| MMLU-Pro | 87.5% | — |
| Confabulations | 15.9% | — |
| GPQA (HELM) | 70.8% | — |
Multimodal Gemma 4 31B IT leads
Claude Opus 4: 31.5 (#106), Gemma 4 31B IT: 41.6 (#34)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| LMArena Vision | 1192 | 1277 |
| GeoBench | 49% | — |
| VPCT | 38% | — |
| LMArena Document | — | 1425 |
Multilingual Gemma 4 31B IT leads
Claude Opus 4: 48.8 (#138), Gemma 4 31B IT: 53.8 (#57)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| LMArena Non-English | 1362 | 1431 |
| LMArena Chinese | 1386 | 1476 |
| LMArena French | 1372 | 1435 |
| LMArena Russian | 1392 | 1460 |
| LMArena Spanish | 1389 | 1444 |
| LMArena German | 1391 | — |
| LMArena Japanese | 1331 | — |
| LMArena Korean | 1321 | — |
Instruction Following Claude Opus 4 leads
Claude Opus 4: 77.1 (#28), Gemma 4 31B IT: 75.5 (#61)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| LMArena Instruction Following | 1406 | 1433 |
| IFEval | 91.8% | — |
Long Context Gemma 4 31B IT leads
Claude Opus 4: 39.6 (#172), Gemma 4 31B IT: 44.2 (#71)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| LMArena Longer Query | 1422 | 1446 |
| Fiction.LiveBench | 61.1% | — |
Writing & Preference Too close to call
Claude Opus 4: 61.2 (#89), Gemma 4 31B IT: 60.5 (#96)
| Benchmark | Claude Opus 4 | Gemma 4 31B IT |
|---|---|---|
| LMArena Text | 1377 | 1443 |
| LMArena Creative Writing | 1387 | 1415 |
| EQ-Bench Creative Writing | 1580 | 1368 |
| LMArena Multi-Turn | 1396 | 1452 |
| Short-Story Creative Writing | 83.6% | — |
| WildBench | 85.2% | — |
| EQ-Bench 4 | — | 1120 |
Frequently asked questions
Is Claude Opus 4 better than Gemma 4 31B IT?
Claude Opus 4 and Gemma 4 31B IT score almost the same on the Noometry Index (43.1 vs 43.5), so choose on price, context window or the category you care about most.
Which is cheaper, Claude Opus 4 or Gemma 4 31B IT?
Gemma 4 31B IT is cheaper. It lists at $0.09 per million input tokens and $0.34 per million output tokens; Claude Opus 4 lists at $15 and $75.
Is Claude Opus 4 or Gemma 4 31B IT better for coding?
Claude Opus 4 scores higher on coding benchmarks: 47.2 versus 42.3 in the Noometry coding category.
Which has the bigger context window?
Gemma 4 31B IT does, with 262K tokens against 200K.
How many benchmarks do Claude Opus 4 and Gemma 4 31B IT share?
25 benchmarks have published results for both models. Claude Opus 4 has 56 scored results on Noometry and Gemma 4 31B IT has 35.