Noometry

Model comparison

Llama 3.1-70B vs MiMo-V2.5

MiMo-V2.5 is the stronger model overall, scoring 43.4 to 29.6 on the Noometry Index.

Last verified . 17 shared benchmarks.

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

MiMo-V2.5 Xiaomi

43.4

Rank #93 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Llama 3.1-70B scores higher in 0 categories and MiMo-V2.5 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where MiMo-V2.5 leads 61.6 to 35.4.
  • MiMo-V2.5 is cheaper at $0.14 / $0.28 per million input/output tokens, against $0.40 / $0.40 for Llama 3.1-70B.
  • MiMo-V2.5 accepts more context: 1.05M tokens versus 128K.

Side by side

Llama 3.1-70BMiMo-V2.5
ProviderMetaXiaomi
Noometry Index29.643.4
Released2024-07-232026-04-22
WeightsOpenOpen
Context window128K1.05M
Max output4K131K
Input $ / M tokens$0.40$0.14
Output $ / M tokens$0.40$0.28
Results tracked3523

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding MiMo-V2.5 leads

Llama 3.1-70B: 30.3 (#296), MiMo-V2.5: 43.9 (#81)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Coding12601469
LMArena WebDev—1438
SciCode—43.1%
WeirdML9%—
BigCodeBench Instruct46.1%—
BigCodeBench Complete54.8%—
ALE-Bench—513.95

Agentic & Tool Use Not comparable

Llama 3.1-70B: 25.1 (#112), MiMo-V2.5: —

BenchmarkLlama 3.1-70BMiMo-V2.5
TheAgentCompany6.9%—
BALROG27.9%—

Reasoning MiMo-V2.5 leads

Llama 3.1-70B: 21.6 (#220), MiMo-V2.5: 28.6 (#101)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Hard Prompts12411450
CritPt—3.7%
DTBench60%—
LMCA14.8%—
Epoch Capabilities Index125.92—

Math MiMo-V2.5 leads

Llama 3.1-70B: 13.5 (#304), MiMo-V2.5: 36.8 (#163)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Math12521436
OTIS Mock AIME 2024-20253.6%—
ProofBench—16%
Omni-MATH21%—
MATH Level 536.7%—

Knowledge MiMo-V2.5 leads

Llama 3.1-70B: 24.2 (#269), MiMo-V2.5: 40.8 (#115)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Expert12091460
GPQA Diamond44.2%—
MMLU-Pro65.3%—
GPQA (HELM)42.6%—
MMLU80.1%—

Multimodal Not comparable

Llama 3.1-70B: —, MiMo-V2.5: 39.8 (#54)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Vision—1247

Multilingual MiMo-V2.5 leads

Llama 3.1-70B: 38.8 (#225), MiMo-V2.5: 51.9 (#99)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Non-English12191404
LMArena Chinese12151468
LMArena French12611447
LMArena German12221421
LMArena Japanese11321306
LMArena Korean11401363
LMArena Russian12341395
LMArena Spanish12531416

Instruction Following MiMo-V2.5 leads

Llama 3.1-70B: 65.3 (#223), MiMo-V2.5: 75.5 (#60)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Instruction Following12311434
IFEval82.1%—

Long Context MiMo-V2.5 leads

Llama 3.1-70B: 37.6 (#214), MiMo-V2.5: 44.2 (#73)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Longer Query12411445

Writing & Preference MiMo-V2.5 leads

Llama 3.1-70B: 35.4 (#267), MiMo-V2.5: 61.6 (#86)

BenchmarkLlama 3.1-70BMiMo-V2.5
LMArena Text12611428
LMArena Creative Writing12321393
LMArena Multi-Turn12561445
EQ-Bench Creative Writing784—
WildBench75.8%—

Frequently asked questions

Is Llama 3.1-70B better than MiMo-V2.5?

MiMo-V2.5 is the stronger model overall, scoring 43.4 to 29.6 on the Noometry Index.

Which is cheaper, Llama 3.1-70B or MiMo-V2.5?

MiMo-V2.5 is cheaper. It lists at $0.14 per million input tokens and $0.28 per million output tokens; Llama 3.1-70B lists at $0.40 and $0.40.

Is Llama 3.1-70B or MiMo-V2.5 better for coding?

MiMo-V2.5 scores higher on coding benchmarks: 43.9 versus 30.3 in the Noometry coding category.

Which has the bigger context window?

MiMo-V2.5 does, with 1.05M tokens against 128K.

How many benchmarks do Llama 3.1-70B and MiMo-V2.5 share?

17 benchmarks have published results for both models. Llama 3.1-70B has 35 scored results on Noometry and MiMo-V2.5 has 23.

Related comparisons

Go deeper