Noometry

Model comparison

Qwen3.5 397B-A17B vs Yi-Lightning

Qwen3.5 397B-A17B is the stronger model overall, scoring 45.7 to 36.5 on the Noometry Index.

Last verified . 17 shared benchmarks.

Qwen3.5 397B-A17B Alibaba (Qwen)

45.7

Rank #60 Confirmed

Yi-Lightning 01.AI

36.5

Rank #206 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Qwen3.5 397B-A17B scores higher in 8 categories and Yi-Lightning in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Qwen3.5 397B-A17B leads 53.3 to 34.8.
  • Qwen3.5 397B-A17B has downloadable open weights; the other is API-only.

Side by side

Qwen3.5 397B-A17BYi-Lightning
ProviderAlibaba (Qwen)01.AI
Noometry Index45.736.5
Released2026-02-012024-12-02
WeightsOpenProprietary
Context window262K—
Max output66K—
Input $ / M tokens$0.60—
Output $ / M tokens$3.60—
Results tracked3618

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen3.5 397B-A17B leads

Qwen3.5 397B-A17B: 41.7 (#108), Yi-Lightning: 27.0 (#315)

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Coding14641313
Aider Polyglot—12.9%
LMArena WebDev1400—

Agentic & Tool Use Not comparable

Qwen3.5 397B-A17B: 33.2 (#51), Yi-Lightning: —

BenchmarkQwen3.5 397B-A17BYi-Lightning
APEX-Agents24.9%—
τ²-bench Airline81.5%—
τ²-bench Banking9.8%—
τ²-bench Retail84.4%—
τ²-bench Telecom97.8%—

Reasoning Qwen3.5 397B-A17B leads

Qwen3.5 397B-A17B: 33.6 (#62), Yi-Lightning: 24.9 (#137)

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Hard Prompts14481302
Kagi LLM Benchmark73.7%—
NYT Connections (extended)58.9%—
Chess Puzzles13%—
Thematic Generalization65.1%—
Mystery Game Puzzles18%—
DTBench87.5%—
LMCA37.9%—
Epoch Capabilities Index146.64—

Math Qwen3.5 397B-A17B leads

Qwen3.5 397B-A17B: 44.9 (#64), Yi-Lightning: 33.5 (#173)

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Math14551300
FrontierMath (Tiers 1-3)31.2%—
OTIS Mock AIME 2024-202588.9%—

Knowledge Qwen3.5 397B-A17B leads

Qwen3.5 397B-A17B: 53.3 (#53), Yi-Lightning: 34.8 (#182)

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Expert14611286
GPQA Diamond86.4%—

Multimodal Not comparable

Qwen3.5 397B-A17B: 40.0 (#42), Yi-Lightning: —

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Vision1262—

Multilingual Qwen3.5 397B-A17B leads

Qwen3.5 397B-A17B: 54.0 (#56), Yi-Lightning: 42.4 (#191)

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Non-English14311269
LMArena Chinese15011320
LMArena French14611304
LMArena German14471267
LMArena Japanese14271228
LMArena Korean13821192
LMArena Russian14291255
LMArena Spanish14431315

Instruction Following Qwen3.5 397B-A17B leads

Qwen3.5 397B-A17B: 75.2 (#73), Yi-Lightning: 67.6 (#191)

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Instruction Following14241278

Long Context Qwen3.5 397B-A17B leads

Qwen3.5 397B-A17B: 44.1 (#71), Yi-Lightning: 39.4 (#176)

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Longer Query14421297

Writing & Preference Qwen3.5 397B-A17B leads

Qwen3.5 397B-A17B: 62.5 (#74), Yi-Lightning: 50.4 (#178)

BenchmarkQwen3.5 397B-A17BYi-Lightning
LMArena Text14381302
LMArena Creative Writing14011280
LMArena Multi-Turn14471311
EQ-Bench Creative Writing1478—

Frequently asked questions

Is Qwen3.5 397B-A17B better than Yi-Lightning?

Qwen3.5 397B-A17B is the stronger model overall, scoring 45.7 to 36.5 on the Noometry Index.

Is Qwen3.5 397B-A17B or Yi-Lightning better for coding?

Qwen3.5 397B-A17B scores higher on coding benchmarks: 41.7 versus 27.0 in the Noometry coding category.

How many benchmarks do Qwen3.5 397B-A17B and Yi-Lightning share?

17 benchmarks have published results for both models. Qwen3.5 397B-A17B has 36 scored results on Noometry and Yi-Lightning has 18.

Related comparisons

Go deeper