Noometry

Model comparison

o3-pro vs Qwen3.8 27B

Qwen3.8 27B is the stronger model overall, scoring 46.0 to 42.9 on the Noometry Index.

Last verified . 5 shared benchmarks.

o3-pro OpenAI

42.9

Rank #105 Confirmed

Qwen3.8 27B Alibaba (Qwen)

46.0

Rank #68 Confirmed

Summary

  • They share 5 benchmarks with published results for both. o3-pro scores higher in 2 categories and Qwen3.8 27B in 3 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in long context, where o3-pro leads 72.2 to 44.3.
  • The biggest single-benchmark swing is ARC-AGI-2: 4.9% for o3-pro and 42.4% for Qwen3.8 27B.
  • Qwen3.8 27B is cheaper at $0.99 / $1.49 per million input/output tokens, against $20 / $80 for o3-pro.
  • Qwen3.8 27B accepts more context: 262K tokens versus 200K.
  • Qwen3.8 27B has downloadable open weights; the other is API-only.

Side by side

o3-proQwen3.8 27B
ProviderOpenAIAlibaba (Qwen)
Noometry Index42.946.0
Released2025-06-102026-08-14
WeightsProprietaryOpen
Context window200K262K
Max output100K33K
Input $ / M tokens$20$0.99
Output $ / M tokens$80$1.49
Results tracked1231

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3-pro leads

o3-pro: 55.5 (#24), Qwen3.8 27B: 50.5 (#44)

Benchmarko3-proQwen3.8 27B
Aider Polyglot84.9%—
LMArena WebDev—1593
SciCode—46.6%
WeirdML58.2%—
LMArena Coding—1482

Agentic & Tool Use Not comparable

o3-pro: —, Qwen3.8 27B: 32.9 (#57)

Benchmarko3-proQwen3.8 27B
APEX-Agents—47.5%

Reasoning Qwen3.8 27B leads

o3-pro: 23.8 (#171), Qwen3.8 27B: 41.0 (#54)

Benchmarko3-proQwen3.8 27B
ARC-AGI-24.9%42.4%
ARC-AGI-159.3%87.5%
DTBench86.9%88%
LMCA38.5%41.4%
Epoch Capabilities Index147.42149.38
Kagi LLM Benchmark72.1%—
NYT Connections (extended)—54.5%
CritPt—5.4%
LMArena Hard Prompts—1460
Surface Evolver Bench—45%

Math Not comparable

o3-pro: —, Qwen3.8 27B: 37.1 (#161)

Benchmarko3-proQwen3.8 27B
ProofBench—16%
LMArena Math—1456

Knowledge Qwen3.8 27B leads

o3-pro: 29.5 (#238), Qwen3.8 27B: 41.6 (#109)

Benchmarko3-proQwen3.8 27B
Confabulations14.2%—
Vectara Hallucination Rate23.3%—
LMArena Expert—1482

Multimodal Not comparable

o3-pro: —, Qwen3.8 27B: 41.3 (#37)

Benchmarko3-proQwen3.8 27B
LMArena Vision—1271

Multilingual Not comparable

o3-pro: —, Qwen3.8 27B: 53.7 (#60)

Benchmarko3-proQwen3.8 27B
LMArena Non-English—1430
LMArena Chinese—1504
LMArena French—1465
LMArena German—1438
LMArena Japanese—1384
LMArena Korean—1393
LMArena Russian—1415
LMArena Spanish—1448

Instruction Following Not comparable

o3-pro: —, Qwen3.8 27B: 75.8 (#53)

Benchmarko3-proQwen3.8 27B
LMArena Instruction Following—1439

Long Context o3-pro leads

o3-pro: 72.2 (#1), Qwen3.8 27B: 44.3 (#70)

Benchmarko3-proQwen3.8 27B
Fiction.LiveBench97.2%—
LMArena Longer Query—1450

Writing & Preference Qwen3.8 27B leads

o3-pro: 57.1 (#133), Qwen3.8 27B: 65.8 (#43)

Benchmarko3-proQwen3.8 27B
LMArena Text—1441
LMArena Creative Writing—1384
Short-Story Creative Writing84.4%—
EQ-Bench Creative Writing—1671
LMArena Multi-Turn—1441

Frequently asked questions

Is o3-pro better than Qwen3.8 27B?

Qwen3.8 27B is the stronger model overall, scoring 46.0 to 42.9 on the Noometry Index.

Which is cheaper, o3-pro or Qwen3.8 27B?

Qwen3.8 27B is cheaper. It lists at $0.99 per million input tokens and $1.49 per million output tokens; o3-pro lists at $20 and $80.

Is o3-pro or Qwen3.8 27B better for coding?

o3-pro scores higher on coding benchmarks: 55.5 versus 50.5 in the Noometry coding category.

Which has the bigger context window?

Qwen3.8 27B does, with 262K tokens against 200K.

How many benchmarks do o3-pro and Qwen3.8 27B share?

5 benchmarks have published results for both models. o3-pro has 12 scored results on Noometry and Qwen3.8 27B has 31.

Related comparisons

Go deeper