Noometry

Model comparison

Olmo 2 0325 32b Instruct vs Qwen2.5 7B Instruct

Olmo 2 0325 32b Instruct is the stronger model overall, scoring 32.7 to 29.0 on the Noometry Index.

Last verified . 5 shared benchmarks.

Olmo 2 0325 32b Instruct Allen Institute for AI (Ai2)

32.7

Rank #254 Confirmed

Qwen2.5 7B Instruct Alibaba (Qwen)

29.0

Rank #320 Confirmed

Summary

  • They share 5 benchmarks with published results for both. Olmo 2 0325 32b Instruct scores higher in 3 categories and Qwen2.5 7B Instruct in 3 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in math, where Olmo 2 0325 32b Instruct leads 26.8 to 12.6.
  • The biggest single-benchmark swing is Omni-MATH: 16.1% for Olmo 2 0325 32b Instruct and 29.4% for Qwen2.5 7B Instruct.

Side by side

Olmo 2 0325 32b InstructQwen2.5 7B Instruct
ProviderAllen Institute for AI (Ai2)Alibaba (Qwen)
Noometry Index32.729.0
Released—2024-09
WeightsOpenOpen
Context window—131K
Max output—8K
Input $ / M tokens—$0.17
Output $ / M tokens—$0.70
Results tracked1615

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 7B Instruct leads

Olmo 2 0325 32b Instruct: 35.2 (#227), Qwen2.5 7B Instruct: 36.5 (#208)

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
BigCodeBench Instruct—37.6%
LMArena Coding1210—
BigCodeBench Complete—46.1%

Agentic & Tool Use Not comparable

Olmo 2 0325 32b Instruct: —, Qwen2.5 7B Instruct: 23.8 (#124)

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
BALROG—7.8%

Reasoning Olmo 2 0325 32b Instruct leads

Olmo 2 0325 32b Instruct: 23.6 (#175), Qwen2.5 7B Instruct: 14.8 (#322)

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
Chess Puzzles—0%
LMArena Hard Prompts1208—
DTBench—47.7%
LMCA—6.4%
Epoch Capabilities Index—118.51

Math Olmo 2 0325 32b Instruct leads

Olmo 2 0325 32b Instruct: 26.8 (#255), Qwen2.5 7B Instruct: 12.6 (#306)

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
Omni-MATH16.1%29.4%
OTIS Mock AIME 2024-2025—2.5%
LMArena Math1208—

Knowledge Olmo 2 0325 32b Instruct leads

Olmo 2 0325 32b Instruct: 19.5 (#279), Qwen2.5 7B Instruct: 17.0 (#286)

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
MMLU-Pro41.4%53.9%
GPQA (HELM)28.7%34.1%
GPQA Diamond—35.5%
MMLU—72.9%

Multilingual Not comparable

Olmo 2 0325 32b Instruct: 34.8 (#248), Qwen2.5 7B Instruct: —

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
LMArena Non-English1160—
LMArena Chinese1192—
LMArena Russian1187—

Instruction Following Qwen2.5 7B Instruct leads

Olmo 2 0325 32b Instruct: 61.5 (#244), Qwen2.5 7B Instruct: 63.2 (#231)

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
IFEval78%74.1%
LMArena Instruction Following1186—

Long Context Not comparable

Olmo 2 0325 32b Instruct: 36.2 (#234), Qwen2.5 7B Instruct: —

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
LMArena Longer Query1194—

Writing & Preference Qwen2.5 7B Instruct leads

Olmo 2 0325 32b Instruct: 42.1 (#236), Qwen2.5 7B Instruct: 48.8 (#195)

BenchmarkOlmo 2 0325 32b InstructQwen2.5 7B Instruct
WildBench73.4%73.1%
LMArena Text1218—
LMArena Creative Writing1199—
LMArena Multi-Turn1221—

Frequently asked questions

Is Olmo 2 0325 32b Instruct better than Qwen2.5 7B Instruct?

Olmo 2 0325 32b Instruct is the stronger model overall, scoring 32.7 to 29.0 on the Noometry Index.

Is Olmo 2 0325 32b Instruct or Qwen2.5 7B Instruct better for coding?

Qwen2.5 7B Instruct scores higher on coding benchmarks: 36.5 versus 35.2 in the Noometry coding category.

How many benchmarks do Olmo 2 0325 32b Instruct and Qwen2.5 7B Instruct share?

5 benchmarks have published results for both models. Olmo 2 0325 32b Instruct has 16 scored results on Noometry and Qwen2.5 7B Instruct has 15.

Related comparisons

Go deeper