Model comparison

Granite 4.1 8b vs o3

o3 is the stronger model overall, scoring 47.5 to 37.4 on the Noometry Index.

Last verified October 10, 2026. 12 shared benchmarks.

Granite 4.1 8b IBM

37.4

Rank #205 Confirmed

o3 OpenAI

47.5

Rank #61 Confirmed

Summary

They share 12 benchmarks with published results for both. Granite 4.1 8b scores higher in 0 categories and o3 in 8 categories; 8 gaps are clear of the uncertainty.
The widest gap is in knowledge, where o3 leads 54.6 to 36.1.
Granite 4.1 8b has downloadable open weights; the other is API-only.

Side by side

Granite 4.1 8b and o3 specifications
	Granite 4.1 8b	o3
Provider	IBM	OpenAI
Noometry Index	37.4	47.5
Released	—	2025-04-16
Weights	Open	Proprietary
Context window	—	200K
Max output	—	100K
Input $ / M tokens	—	$2
Output $ / M tokens	—	$8
Results tracked	13	63

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

Granite 4.1 8b: 30.1 (#297), o3: 46.8 (#64)

Coding benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Coding	1312	1408
SWE-bench Verified	—	62.3%
SWE-bench Verified (bash only)	—	58.4%
Aider Polyglot	—	81.3%
LMArena WebDev	1192	—
GSO	—	8.8%
WeirdML	—	52.4%
CadEval	—	74%
ALE-Bench	—	933.55

Agentic & Tool Use Not comparable

Granite 4.1 8b: —, o3: 34.5 (#44)

Agentic & Tool Use benchmarks
Benchmark	Granite 4.1 8b	o3
Berkeley Function Calling Leaderboard	—	63%
GDPval	—	30.8%
DeepResearch Bench	—	45.2%
OSWorld	—	23%
LMArena Search	—	1144
METR Time Horizons	—	65.4%

Reasoning o3 leads

Granite 4.1 8b: 25.7 (#143), o3: 32.0 (#78)

Reasoning benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Hard Prompts	1293	1402
ARC-AGI-2	—	6.5%
SimpleBench	—	53.1%
Kagi LLM Benchmark	—	67.6%
ARC-AGI-1	—	60.8%
CritPt	—	1.4%
Chess Puzzles	—	38%
EnigmaEval	—	13.1%
Mystery Game Puzzles	—	29%
DTBench	—	84.8%
LMCA	—	39.7%
Epoch Capabilities Index	—	146.86
ForecastBench	—	62.5

Math o3 leads

Granite 4.1 8b: 36.4 (#166), o3: 50.2 (#58)

Math benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Math	1312	1426
FrontierMath (Tiers 1-3)	—	33.3%
OTIS Mock AIME 2024-2025	—	84.4%
Omni-MATH	—	71.4%
MATH Level 5	—	97.8%
FrontierMath (Feb 2025 set)	—	18.7%
FrontierMath Tier 4 (v1)	—	2.1%

Knowledge o3 leads

Granite 4.1 8b: 36.1 (#174), o3: 54.6 (#52)

Knowledge benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Expert	1309	1402
GPQA Diamond	—	81.8%
Humanity's Last Exam	—	20.3%
SimpleQA Verified	—	49.4%
MMLU-Pro	—	85.9%
Confabulations	—	14.4%
GPQA (HELM)	—	75.3%

Multimodal Not comparable

Granite 4.1 8b: —, o3: 41.4 (#36)

Multimodal benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Vision	—	1214
GeoBench	—	74%
VPCT	—	52%

Multilingual o3 leads

Granite 4.1 8b: 41.7 (#204), o3: 51.7 (#105)

Multilingual benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Non-English	1261	1401
LMArena Chinese	1337	1437
LMArena Russian	1240	1406
LMArena French	—	1430
LMArena German	—	1420
LMArena Japanese	—	1403
LMArena Korean	—	1370
LMArena Spanish	—	1395

Instruction Following o3 leads

Granite 4.1 8b: 66.9 (#203), o3: 72.8 (#127)

Instruction Following benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Instruction Following	1269	1368
IFEval	—	86.9%

Long Context o3 leads

Granite 4.1 8b: 38.7 (#193), o3: 53.3 (#6)

Long Context benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Longer Query	1275	1372
Fiction.LiveBench	—	88.9%
CL-bench	—	17.8%

Writing & Preference o3 leads

Granite 4.1 8b: 48.1 (#204), o3: 63.5 (#64)

Writing & Preference benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Text	1290	1410
LMArena Creative Writing	1251	1359
LMArena Multi-Turn	1268	1405
Short-Story Creative Writing	—	83.9%
EQ-Bench Creative Writing	—	1676
WildBench	—	86.1%

Frequently asked questions

Is Granite 4.1 8b better than o3?