Model comparison

GPT-5.4 mini vs Qwen3 8B

GPT-5.4 mini is the stronger model overall, scoring 45.0 to 33.7 on the Noometry Index. Qwen3 8B costs 5.4× less per token, which makes it the better buy when GPT-5.4 mini's lead doesn't matter for your workload.

Last verified October 11, 2026. 9 shared benchmarks.

GPT-5.4 mini OpenAI

45.0

Rank #76 Confirmed

Qwen3 8B

Alibaba (Qwen)

33.7

Rank #238 Confirmed

Summary

They share 9 benchmarks with published results for both. GPT-5.4 mini scores higher in 5 categories and Qwen3 8B in 1 category; 5 gaps are clear of the uncertainty.
The widest gap is in knowledge, where GPT-5.4 mini leads 51.5 to 36.1.
The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 88.9% for GPT-5.4 mini and 56.1% for Qwen3 8B.
Qwen3 8B is cheaper at $0.18 / $0.70 per million input/output tokens, against $0.75 / $4.50 for GPT-5.4 mini.
GPT-5.4 mini accepts more context: 400K tokens versus 131K.
Qwen3 8B has downloadable open weights; the other is API-only.

Side by side

GPT-5.4 mini and Qwen3 8B specifications
	GPT-5.4 mini	Qwen3 8B
Provider	OpenAI	Alibaba (Qwen)
Noometry Index	45.0	33.7
Released	2026-03-17	2025-04
Weights	Proprietary	Open
Context window	400K	131K
Max output	128K	8K
Input $ / M tokens	$0.75	$0.18
Output $ / M tokens	$4.50	$0.70
Results tracked	46	11

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 mini leads

GPT-5.4 mini: 45.2 (#72), Qwen3 8B: 34.0 (#248)

Coding benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
SciCode	49.9%	22.6%
FrontierCode	27%	—
LMArena WebDev	1397	—
WeirdML	60.3%	—
LMArena Coding	1438	—
ALE-Bench	1,189	—

Agentic & Tool Use Too close to call

GPT-5.4 mini: 29.9 (#81), Qwen3 8B: 30.2 (#78)

Agentic & Tool Use benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
Berkeley Function Calling Leaderboard	—	42.6%
DeepResearch Bench	36.3%	—

Reasoning GPT-5.4 mini leads

GPT-5.4 mini: 30.4 (#85), Qwen3 8B: 16.6 (#303)

Reasoning benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
CritPt	10%	0%
Chess Puzzles	24%	5%
DTBench	80%	59.7%
LMCA	40.8%	8.8%
Epoch Capabilities Index	148.84	136.17
ARC-AGI-2	18.9%	—
Kagi LLM Benchmark	37.9%	—
NYT Connections (extended)	61.8%	—
ARC-AGI-1	63.7%	—
Thematic Generalization	61.7%	—
LMArena Hard Prompts	1424	—
Mystery Game Puzzles	11%	—
ForecastBench	57	—

Math GPT-5.4 mini leads

GPT-5.4 mini: 45.5 (#75), Qwen3 8B: 34.9 (#191)

Math benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
OTIS Mock AIME 2024-2025	88.9%	56.1%
FrontierMath (Tiers 1-3)	51.2%	—
FrontierMath Tier 4	9.8%	—
ProofBench	21%	—
LMArena Math	1419	—
FrontierMath (Feb 2025 set)	28.3%	—
FrontierMath Tier 4 (v1)	2.1%	—

Knowledge GPT-5.4 mini leads

GPT-5.4 mini: 51.5 (#67), Qwen3 8B: 36.1 (#173)

Knowledge benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
GPQA Diamond	86.9%	56.8%
Vectara Hallucination Rate	5.5%	4.8%
SimpleQA Verified	29.4%	—
LMArena Expert	1435	—

Multimodal Not comparable

GPT-5.4 mini: 39.7 (#56), Qwen3 8B: —

Multimodal benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
LMArena Vision	1245	—

Multilingual Not comparable

GPT-5.4 mini: 51.9 (#96), Qwen3 8B: —

Multilingual benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
LMArena Non-English	1405	—
LMArena Chinese	1446	—
LMArena French	1440	—
LMArena German	1409	—
LMArena Japanese	1374	—
LMArena Korean	1368	—
LMArena Russian	1417	—
LMArena Spanish	1405	—

Instruction Following Not comparable

GPT-5.4 mini: 74.1 (#102), Qwen3 8B: —

Instruction Following benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
LMArena Instruction Following	1405	—

Long Context GPT-5.4 mini leads

GPT-5.4 mini: 43.0 (#112), Qwen3 8B: 37.9 (#210)

Long Context benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
Fiction.LiveBench	—	62.1%
LMArena Longer Query	1407	—

Writing & Preference Not comparable

GPT-5.4 mini: 64.0 (#58), Qwen3 8B: —

Writing & Preference benchmarks
Benchmark	GPT-5.4 mini	Qwen3 8B
LMArena Text	1412	—
LMArena Creative Writing	1370	—
EQ-Bench Creative Writing	1665	—
LMArena Multi-Turn	1429	—

Frequently asked questions

Is GPT-5.4 mini better than Qwen3 8B?

Which is cheaper, GPT-5.4 mini or Qwen3 8B?

Qwen3 8B is cheaper. It lists at $0.18 per million input tokens and $0.70 per million output tokens; GPT-5.4 mini lists at $0.75 and $4.50.

Is GPT-5.4 mini or Qwen3 8B better for coding?

GPT-5.4 mini scores higher on coding benchmarks: 45.2 versus 34.0 in the Noometry coding category.

Which has the bigger context window?

GPT-5.4 mini does, with 400K tokens against 131K.

How many benchmarks do GPT-5.4 mini and Qwen3 8B share?

9 benchmarks have published results for both models. GPT-5.4 mini has 46 scored results on Noometry and Qwen3 8B has 11.