Noometry

Model comparison

gpt-oss-120b vs Grok-3 mini

Grok-3 mini is the stronger model overall, scoring 41.2 to 36.3 on the Noometry Index.

Last verified . 31 shared benchmarks.

gpt-oss-120b OpenAI

36.3

Rank #217 Confirmed

Grok-3 mini xAI

41.2

Rank #141 Confirmed

Summary

  • They share 31 benchmarks with published results for both. gpt-oss-120b scores higher in 2 categories and Grok-3 mini in 6 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where gpt-oss-120b leads 52.5 to 42.1.
  • The biggest single-benchmark swing is Omni-MATH: 68.8% for gpt-oss-120b and 31.8% for Grok-3 mini.
  • gpt-oss-120b has downloadable open weights; the other is API-only.

Side by side

gpt-oss-120bGrok-3 mini
ProviderOpenAIxAI
Noometry Index36.341.2
Released2025-08-052025-04-09
WeightsOpenProprietary
Context window131K—
Max output41K—
Input $ / M tokens$0.037—
Output $ / M tokens$0.17—
Results tracked4835

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-3 mini leads

gpt-oss-120b: 33.5 (#256), Grok-3 mini: 40.8 (#131)

Benchmarkgpt-oss-120bGrok-3 mini
Aider Polyglot41.8%49.3%
WeirdML48.2%42.6%
LMArena Coding13801379
SWE-bench Verified (bash only)26%—
SciCode36%—
ALE-Bench575.62—
AlgoTune1.41—

Agentic & Tool Use Not comparable

gpt-oss-120b: 12.2 (#153), Grok-3 mini: —

Benchmarkgpt-oss-120bGrok-3 mini
Terminal-Bench18.7%—
APEX-Agents4.4%—
METR Time Horizons56.6%—
Vending-Bench 2-21.53—

Reasoning gpt-oss-120b leads

gpt-oss-120b: 20.0 (#245), Grok-3 mini: 13.6 (#334)

Benchmarkgpt-oss-120bGrok-3 mini
Kagi LLM Benchmark58.6%61.3%
LMArena Hard Prompts13641375
Epoch Capabilities Index139.93140.35
ARC-AGI-2—0.4%
SimpleBench22.1%—
ARC-AGI-1—16.5%
CritPt1.1%—
Chess Puzzles20%—
Mystery Game Puzzles2%—
DTBench76.3%—
LMCA22.1%—
Surface Evolver Bench25%—

Math gpt-oss-120b leads

gpt-oss-120b: 52.5 (#50), Grok-3 mini: 42.1 (#85)

Benchmarkgpt-oss-120bGrok-3 mini
OTIS Mock AIME 2024-202588.9%77.8%
Omni-MATH68.8%31.8%
LMArena Math13891386
MATH Level 5—90.9%
FrontierMath (Feb 2025 set)—5.9%

Knowledge Grok-3 mini leads

gpt-oss-120b: 42.4 (#96), Grok-3 mini: 46.4 (#81)

Benchmarkgpt-oss-120bGrok-3 mini
GPQA Diamond75.8%76.3%
MMLU-Pro79.5%79.9%
Confabulations15.7%10.8%
GPQA (HELM)68.4%67.5%
LMArena Expert13561395
Vectara Hallucination Rate14.2%—

Multilingual Too close to call

gpt-oss-120b: 48.0 (#147), Grok-3 mini: 48.1 (#145)

Benchmarkgpt-oss-120bGrok-3 mini
LMArena Non-English13511352
LMArena Chinese13851387
LMArena French13691357
LMArena German13531349
LMArena Japanese13311342
LMArena Korean12821335
LMArena Russian13431353
LMArena Spanish13891381

Instruction Following Grok-3 mini leads

gpt-oss-120b: 69.3 (#173), Grok-3 mini: 78.5 (#9)

Benchmarkgpt-oss-120bGrok-3 mini
IFEval83.6%95.1%
LMArena Instruction Following13181357

Long Context Grok-3 mini leads

gpt-oss-120b: 31.4 (#278), Grok-3 mini: 41.0 (#147)

Benchmarkgpt-oss-120bGrok-3 mini
Fiction.LiveBench44.4%66.7%
LMArena Longer Query13191372

Writing & Preference Grok-3 mini leads

gpt-oss-120b: 46.5 (#217), Grok-3 mini: 52.5 (#169)

Benchmarkgpt-oss-120bGrok-3 mini
LMArena Text13651370
LMArena Creative Writing12751342
Short-Story Creative Writing77.1%73.5%
WildBench84.5%65.1%
LMArena Multi-Turn13401355
EQ-Bench Creative Writing961—

Frequently asked questions

Is gpt-oss-120b better than Grok-3 mini?

Grok-3 mini is the stronger model overall, scoring 41.2 to 36.3 on the Noometry Index.

Is gpt-oss-120b or Grok-3 mini better for coding?

Grok-3 mini scores higher on coding benchmarks: 40.8 versus 33.5 in the Noometry coding category.

How many benchmarks do gpt-oss-120b and Grok-3 mini share?

31 benchmarks have published results for both models. gpt-oss-120b has 48 scored results on Noometry and Grok-3 mini has 35.

Related comparisons

Go deeper