Noometry

Model comparison

Grok 3 vs Qwen3.8 27B

Qwen3.8 27B is the stronger model overall, scoring 45.3 to 39.6 on the Noometry Index.

Last verified . 19 shared benchmarks.

Grok 3 xAI

39.6

Rank #150 Confirmed

Qwen3.8 27B Alibaba (Qwen)

45.3

Rank #64 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Grok 3 scores higher in 2 categories and Qwen3.8 27B in 6 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Qwen3.8 27B leads 31.5 to 13.8.
  • Qwen3.8 27B has downloadable open weights; the other is API-only.

Side by side

Grok 3Qwen3.8 27B
ProviderxAIAlibaba (Qwen)
Noometry Index39.645.3
Released2025-04-092026-08-14
WeightsProprietaryOpen
Context window—262K
Max output—33K
Input $ / M tokens—$0.99
Output $ / M tokens—$1.49
Results tracked4028

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen3.8 27B leads

Grok 3: 41.5 (#113), Qwen3.8 27B: 50.3 (#42)

BenchmarkGrok 3Qwen3.8 27B
LMArena Coding14321487
Aider Polyglot53.3%—
LMArena WebDev—1592
SciCode—46.6%
WeirdML37.2%—

Agentic & Tool Use Not comparable

Grok 3: 30.2 (#69), Qwen3.8 27B: —

BenchmarkGrok 3Qwen3.8 27B
BALROG29.5%—

Reasoning Qwen3.8 27B leads

Grok 3: 13.8 (#322), Qwen3.8 27B: 31.5 (#71)

BenchmarkGrok 3Qwen3.8 27B
LMArena Hard Prompts14331459
Epoch Capabilities Index138.34149.38
ARC-AGI-20%—
SimpleBench36.1%—
Kagi LLM Benchmark61.3%—
NYT Connections (extended)—54.5%
ARC-AGI-15.5%—
CritPt—5.4%
DTBench—88%
LMCA—41.4%
Surface Evolver Bench—45%

Math Too close to call

Grok 3: 36.2 (#122), Qwen3.8 27B: 35.7 (#139)

BenchmarkGrok 3Qwen3.8 27B
LMArena Math13901453
OTIS Mock AIME 2024-202555.6%—
ProofBench—16%
Omni-MATH46.4%—
MATH Level 588.7%—
FrontierMath (Feb 2025 set)3.8%—
FrontierMath Tier 4 (v1)0%—

Knowledge Grok 3 leads

Grok 3: 46.0 (#76), Qwen3.8 27B: 40.9 (#105)

BenchmarkGrok 3Qwen3.8 27B
LMArena Expert14201482
GPQA Diamond75.8%—
MMLU-Pro78.8%—
Confabulations14.2%—
Vectara Hallucination Rate5.8%—
GPQA (HELM)65%—

Multimodal Not comparable

Grok 3: —, Qwen3.8 27B: 40.5 (#36)

BenchmarkGrok 3Qwen3.8 27B
LMArena Vision—1270

Multilingual Qwen3.8 27B leads

Grok 3: 52.5 (#81), Qwen3.8 27B: 54.0 (#54)

BenchmarkGrok 3Qwen3.8 27B
LMArena Non-English14111431
LMArena Chinese14481504
LMArena French14601468
LMArena German14311435
LMArena Japanese13891392
LMArena Korean13741389
LMArena Russian14161421
LMArena Spanish14171444

Instruction Following Too close to call

Grok 3: 75.2 (#69), Qwen3.8 27B: 75.8 (#54)

BenchmarkGrok 3Qwen3.8 27B
LMArena Instruction Following14091436
IFEval88.4%—

Long Context Qwen3.8 27B leads

Grok 3: 38.7 (#187), Qwen3.8 27B: 44.3 (#66)

BenchmarkGrok 3Qwen3.8 27B
LMArena Longer Query14391448
Fiction.LiveBench58.3%—

Writing & Preference Qwen3.8 27B leads

Grok 3: 56.0 (#136), Qwen3.8 27B: 65.9 (#41)

BenchmarkGrok 3Qwen3.8 27B
LMArena Text14261441
LMArena Creative Writing14141382
EQ-Bench Creative Writing11861671
LMArena Multi-Turn14241443
Short-Story Creative Writing76.4%—
WildBench84.9%—

Frequently asked questions

Is Grok 3 better than Qwen3.8 27B?

Qwen3.8 27B is the stronger model overall, scoring 45.3 to 39.6 on the Noometry Index.

Is Grok 3 or Qwen3.8 27B better for coding?

Qwen3.8 27B scores higher on coding benchmarks: 50.3 versus 41.5 in the Noometry coding category.

How many benchmarks do Grok 3 and Qwen3.8 27B share?

19 benchmarks have published results for both models. Grok 3 has 40 scored results on Noometry and Qwen3.8 27B has 28.

Related comparisons

Go deeper