Model comparison

o3 vs Qwen3-4B

o3 is the stronger model overall, scoring 47.5 to 31.9 on the Noometry Index.

Last verified October 11, 2026. 4 shared benchmarks.

o3 OpenAI

47.5

Rank #61 Confirmed

Qwen3-4B

Alibaba (Qwen)

31.9

Rank #264 Confirmed

Summary

They share 4 benchmarks with published results for both. o3 scores higher in 4 categories and Qwen3-4B in 0 categories; 4 gaps are clear of the uncertainty.
The widest gap is in knowledge, where o3 leads 54.6 to 33.0.
The biggest single-benchmark swing is Chess Puzzles: 38% for o3 and 4% for Qwen3-4B.
Qwen3-4B has downloadable open weights; the other is API-only.

Side by side

o3 and Qwen3-4B specifications
	o3	Qwen3-4B
Provider	OpenAI	Alibaba (Qwen)
Noometry Index	47.5	31.9
Released	2025-04-16	2025-04-29
Weights	Proprietary	Open
Context window	200K	—
Max output	100K	—
Input $ / M tokens	$2	—
Output $ / M tokens	$8	—
Results tracked	63	6

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

o3: 46.8 (#64), Qwen3-4B: —

Coding benchmarks
Benchmark	o3	Qwen3-4B
SWE-bench Verified	62.3%	—
SWE-bench Verified (bash only)	58.4%	—
Aider Polyglot	81.3%	—
GSO	8.8%	—
WeirdML	52.4%	—
LMArena Coding	1408	—
CadEval	74%	—
ALE-Bench	933.55	—

Agentic & Tool Use o3 leads

o3: 34.5 (#44), Qwen3-4B: 27.6 (#100)

Agentic & Tool Use benchmarks
Benchmark	o3	Qwen3-4B
Berkeley Function Calling Leaderboard	63%	35.7%
GDPval	30.8%	—
DeepResearch Bench	45.2%	—
OSWorld	23%	—
LMArena Search	1144	—
METR Time Horizons	65.4%	—

Reasoning o3 leads

o3: 32.0 (#78), Qwen3-4B: 19.2 (#268)

Reasoning benchmarks
Benchmark	o3	Qwen3-4B
Chess Puzzles	38%	4%
ARC-AGI-2	6.5%	—
SimpleBench	53.1%	—
Kagi LLM Benchmark	67.6%	—
ARC-AGI-1	60.8%	—
CritPt	1.4%	—
EnigmaEval	13.1%	—
LMArena Hard Prompts	1402	—
Mystery Game Puzzles	29%	—
DTBench	84.8%	—
LMCA	39.7%	—
Epoch Capabilities Index	146.86	—
ForecastBench	62.5	—

Math o3 leads

o3: 50.2 (#58), Qwen3-4B: 29.7 (#240)

Math benchmarks
Benchmark	o3	Qwen3-4B
OTIS Mock AIME 2024-2025	84.4%	52.2%
FrontierMath (Tiers 1-3)	33.3%	—
MathArena Final-Answer Competitions	—	38.5%
Omni-MATH	71.4%	—
LMArena Math	1426	—
MATH Level 5	97.8%	—
FrontierMath (Feb 2025 set)	18.7%	—
FrontierMath Tier 4 (v1)	2.1%	—

Knowledge o3 leads

o3: 54.6 (#52), Qwen3-4B: 33.0 (#208)

Knowledge benchmarks
Benchmark	o3	Qwen3-4B
GPQA Diamond	81.8%	52.3%
Humanity's Last Exam	20.3%	—
SimpleQA Verified	49.4%	—
MMLU-Pro	85.9%	—
Confabulations	14.4%	—
Vectara Hallucination Rate	—	5.7%
GPQA (HELM)	75.3%	—
LMArena Expert	1402	—

Multimodal Not comparable

o3: 41.4 (#36), Qwen3-4B: —

Multimodal benchmarks
Benchmark	o3	Qwen3-4B
LMArena Vision	1214	—
GeoBench	74%	—
VPCT	52%	—

Multilingual Not comparable

o3: 51.7 (#105), Qwen3-4B: —

Multilingual benchmarks
Benchmark	o3	Qwen3-4B
LMArena Non-English	1401	—
LMArena Chinese	1437	—
LMArena French	1430	—
LMArena German	1420	—
LMArena Japanese	1403	—
LMArena Korean	1370	—
LMArena Russian	1406	—
LMArena Spanish	1395	—

Instruction Following Not comparable

o3: 72.8 (#127), Qwen3-4B: —

Instruction Following benchmarks
Benchmark	o3	Qwen3-4B
IFEval	86.9%	—
LMArena Instruction Following	1368	—

Long Context Not comparable

o3: 53.3 (#6), Qwen3-4B: —

Long Context benchmarks
Benchmark	o3	Qwen3-4B
Fiction.LiveBench	88.9%	—
CL-bench	17.8%	—
LMArena Longer Query	1372	—

Writing & Preference Not comparable

o3: 63.5 (#64), Qwen3-4B: —

Writing & Preference benchmarks
Benchmark	o3	Qwen3-4B
LMArena Text	1410	—
LMArena Creative Writing	1359	—
Short-Story Creative Writing	83.9%	—
EQ-Bench Creative Writing	1676	—
WildBench	86.1%	—
LMArena Multi-Turn	1405	—

Frequently asked questions

Is o3 better than Qwen3-4B?