Noometry

Model comparison

Mercury vs Qwen3.5 397B-A17B

Qwen3.5 397B-A17B is the stronger model overall, scoring 46.0 to 37.6 on the Noometry Index.

Last verified . 9 shared benchmarks.

Mercury Inception

37.6

Rank #199 Confirmed

Qwen3.5 397B-A17B Alibaba (Qwen)

46.0

Rank #67 Confirmed

Summary

  • They share 9 benchmarks with published results for both. Mercury scores higher in 0 categories and Qwen3.5 397B-A17B in 6 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Qwen3.5 397B-A17B leads 34.5 to 17.5.
  • The biggest single-benchmark swing is Kagi LLM Benchmark: 21.6% for Mercury and 73.7% for Qwen3.5 397B-A17B.
  • Qwen3.5 397B-A17B has downloadable open weights; the other is API-only.

Side by side

MercuryQwen3.5 397B-A17B
ProviderInceptionAlibaba (Qwen)
Noometry Index37.646.0
Released—2026-02-01
WeightsProprietaryOpen
Context window—262K
Max output—66K
Input $ / M tokens—$0.60
Output $ / M tokens—$3.60
Results tracked936

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen3.5 397B-A17B leads

Mercury: 38.7 (#170), Qwen3.5 397B-A17B: 42.0 (#114)

BenchmarkMercuryQwen3.5 397B-A17B
LMArena Coding13221465
LMArena WebDev—1400

Agentic & Tool Use Not comparable

Mercury: —, Qwen3.5 397B-A17B: 33.3 (#53)

BenchmarkMercuryQwen3.5 397B-A17B
APEX-Agents—24.9%
τ²-bench Airline—81.5%
τ²-bench Banking—9.8%
τ²-bench Retail—84.4%
τ²-bench Telecom—97.8%

Reasoning Qwen3.5 397B-A17B leads

Mercury: 17.5 (#293), Qwen3.5 397B-A17B: 34.5 (#70)

BenchmarkMercuryQwen3.5 397B-A17B
Kagi LLM Benchmark21.6%73.7%
LMArena Hard Prompts12851448
NYT Connections (extended)—58.9%
Chess Puzzles—13%
Thematic Generalization—65.1%
Mystery Game Puzzles—18%
DTBench—87.5%
LMCA—37.9%
Epoch Capabilities Index—146.65

Math Not comparable

Mercury: —, Qwen3.5 397B-A17B: 46.1 (#73)

BenchmarkMercuryQwen3.5 397B-A17B
FrontierMath (Tiers 1-3)—31.2%
OTIS Mock AIME 2024-2025—88.9%
LMArena Math—1454

Knowledge Not comparable

Mercury: —, Qwen3.5 397B-A17B: 53.3 (#58)

BenchmarkMercuryQwen3.5 397B-A17B
GPQA Diamond—86.4%
LMArena Expert—1462

Multimodal Not comparable

Mercury: —, Qwen3.5 397B-A17B: 40.7 (#44)

BenchmarkMercuryQwen3.5 397B-A17B
LMArena Vision—1263

Multilingual Qwen3.5 397B-A17B leads

Mercury: 41.6 (#206), Qwen3.5 397B-A17B: 53.7 (#59)

BenchmarkMercuryQwen3.5 397B-A17B
LMArena Non-English12601430
LMArena Chinese—1500
LMArena French—1461
LMArena German—1447
LMArena Japanese—1426
LMArena Korean—1384
LMArena Russian—1429
LMArena Spanish—1441

Instruction Following Qwen3.5 397B-A17B leads

Mercury: 65.2 (#224), Qwen3.5 397B-A17B: 75.0 (#77)

BenchmarkMercuryQwen3.5 397B-A17B
LMArena Instruction Following12391424

Long Context Qwen3.5 397B-A17B leads

Mercury: 38.4 (#198), Qwen3.5 397B-A17B: 44.1 (#74)

BenchmarkMercuryQwen3.5 397B-A17B
LMArena Longer Query12661442

Writing & Preference Qwen3.5 397B-A17B leads

Mercury: 46.2 (#221), Qwen3.5 397B-A17B: 62.3 (#79)

BenchmarkMercuryQwen3.5 397B-A17B
LMArena Text12821438
LMArena Creative Writing11911401
LMArena Multi-Turn12821446
EQ-Bench Creative Writing—1478

Frequently asked questions

Is Mercury better than Qwen3.5 397B-A17B?

Qwen3.5 397B-A17B is the stronger model overall, scoring 46.0 to 37.6 on the Noometry Index.

Is Mercury or Qwen3.5 397B-A17B better for coding?

Qwen3.5 397B-A17B scores higher on coding benchmarks: 42.0 versus 38.7 in the Noometry coding category.

How many benchmarks do Mercury and Qwen3.5 397B-A17B share?

9 benchmarks have published results for both models. Mercury has 9 scored results on Noometry and Qwen3.5 397B-A17B has 36.

Related comparisons

Go deeper