Noometry

Model comparison

GPT-5.4 mini vs Qwen3.8 27B

GPT-5.4 mini and Qwen3.8 27B score almost the same on the Noometry Index (44.8 vs 45.3), so choose on price, context window or the category you care about most.

Last verified . 27 shared benchmarks.

GPT-5.4 mini OpenAI

44.8

Rank #69 Confirmed

Qwen3.8 27B Alibaba (Qwen)

45.3

Rank #64 Confirmed

Summary

  • They share 27 benchmarks with published results for both. GPT-5.4 mini scores higher in 2 categories and Qwen3.8 27B in 7 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GPT-5.4 mini leads 51.4 to 40.9.
  • The biggest single-benchmark swing is DTBench: 80% for GPT-5.4 mini and 88% for Qwen3.8 27B.
  • Qwen3.8 27B is cheaper at $0.99 / $1.49 per million input/output tokens, against $0.75 / $4.50 for GPT-5.4 mini.
  • GPT-5.4 mini accepts more context: 400K tokens versus 262K.
  • Qwen3.8 27B has downloadable open weights; the other is API-only.

Side by side

GPT-5.4 miniQwen3.8 27B
ProviderOpenAIAlibaba (Qwen)
Noometry Index44.845.3
Released2026-03-172026-08-14
WeightsProprietaryOpen
Context window400K262K
Max output128K33K
Input $ / M tokens$0.75$0.99
Output $ / M tokens$4.50$1.49
Results tracked4628

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen3.8 27B leads

GPT-5.4 mini: 45.0 (#67), Qwen3.8 27B: 50.3 (#42)

BenchmarkGPT-5.4 miniQwen3.8 27B
LMArena WebDev13971592
SciCode49.9%46.6%
LMArena Coding14381487
FrontierCode27%—
WeirdML60.3%—
ALE-Bench1,189—

Agentic & Tool Use Not comparable

GPT-5.4 mini: 29.6 (#74), Qwen3.8 27B: —

BenchmarkGPT-5.4 miniQwen3.8 27B
DeepResearch Bench36.3%—

Reasoning Qwen3.8 27B leads

GPT-5.4 mini: 30.3 (#77), Qwen3.8 27B: 31.5 (#71)

BenchmarkGPT-5.4 miniQwen3.8 27B
NYT Connections (extended)61.8%54.5%
CritPt10%5.4%
LMArena Hard Prompts14241459
DTBench80%88%
LMCA40.8%41.4%
Epoch Capabilities Index148.83149.38
ARC-AGI-218.9%—
Kagi LLM Benchmark37.9%—
ARC-AGI-163.7%—
Chess Puzzles24%—
Thematic Generalization61.7%—
Mystery Game Puzzles11%—
Surface Evolver Bench—45%
ForecastBench57—

Math GPT-5.4 mini leads

GPT-5.4 mini: 45.4 (#63), Qwen3.8 27B: 35.7 (#139)

BenchmarkGPT-5.4 miniQwen3.8 27B
ProofBench21%16%
LMArena Math14181453
FrontierMath (Tiers 1-3)51.2%—
FrontierMath Tier 49.8%—
OTIS Mock AIME 2024-202588.9%—
FrontierMath (Feb 2025 set)28.3%—
FrontierMath Tier 4 (v1)2.1%—

Knowledge GPT-5.4 mini leads

GPT-5.4 mini: 51.4 (#62), Qwen3.8 27B: 40.9 (#105)

BenchmarkGPT-5.4 miniQwen3.8 27B
LMArena Expert14341482
GPQA Diamond86.9%—
SimpleQA Verified29.4%—
Vectara Hallucination Rate5.5%—

Multimodal Qwen3.8 27B leads

GPT-5.4 mini: 39.0 (#51), Qwen3.8 27B: 40.5 (#36)

BenchmarkGPT-5.4 miniQwen3.8 27B
LMArena Vision12451270

Multilingual Qwen3.8 27B leads

GPT-5.4 mini: 52.1 (#94), Qwen3.8 27B: 54.0 (#54)

BenchmarkGPT-5.4 miniQwen3.8 27B
LMArena Non-English14051431
LMArena Chinese14461504
LMArena French14401468
LMArena German14071435
LMArena Japanese13751392
LMArena Korean13691389
LMArena Russian14171421
LMArena Spanish14031444

Instruction Following Qwen3.8 27B leads

GPT-5.4 mini: 74.3 (#99), Qwen3.8 27B: 75.8 (#54)

BenchmarkGPT-5.4 miniQwen3.8 27B
LMArena Instruction Following14051436

Long Context Qwen3.8 27B leads

GPT-5.4 mini: 42.9 (#107), Qwen3.8 27B: 44.3 (#66)

BenchmarkGPT-5.4 miniQwen3.8 27B
LMArena Longer Query14071448

Writing & Preference Qwen3.8 27B leads

GPT-5.4 mini: 64.2 (#56), Qwen3.8 27B: 65.9 (#41)

BenchmarkGPT-5.4 miniQwen3.8 27B
LMArena Text14111441
LMArena Creative Writing13701382
EQ-Bench Creative Writing16651671
LMArena Multi-Turn14291443

Frequently asked questions

Is GPT-5.4 mini better than Qwen3.8 27B?

GPT-5.4 mini and Qwen3.8 27B score almost the same on the Noometry Index (44.8 vs 45.3), so choose on price, context window or the category you care about most.

Which is cheaper, GPT-5.4 mini or Qwen3.8 27B?

Qwen3.8 27B is cheaper. It lists at $0.99 per million input tokens and $1.49 per million output tokens; GPT-5.4 mini lists at $0.75 and $4.50.

Is GPT-5.4 mini or Qwen3.8 27B better for coding?

Qwen3.8 27B scores higher on coding benchmarks: 50.3 versus 45.0 in the Noometry coding category.

Which has the bigger context window?

GPT-5.4 mini does, with 400K tokens against 262K.

How many benchmarks do GPT-5.4 mini and Qwen3.8 27B share?

27 benchmarks have published results for both models. GPT-5.4 mini has 46 scored results on Noometry and Qwen3.8 27B has 28.

Related comparisons

Go deeper