Model comparison

Llama 3.2 1B vs Mistral Nemo

Mistral Nemo is the stronger model overall, scoring 26.4 to 20.1 on the Noometry Index. Llama 3.2 1B costs 2.1× less per token, which makes it the better buy when Mistral Nemo's lead doesn't matter for your workload.

Last verified . 5 shared benchmarks.

Llama 3.2 1B Meta

20.1

Rank #354 Confirmed

Mistral Nemo Mistral AI

26.4

Rank #337 Confirmed

Summary

  • They share 5 benchmarks with published results for both. Llama 3.2 1B scores higher in 0 categories and Mistral Nemo in 5 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in math, where Mistral Nemo leads 25.5 to 10.4.
  • The biggest single-benchmark swing is Berkeley Function Calling Leaderboard: 10.8% for Llama 3.2 1B and 27.6% for Mistral Nemo.
  • Llama 3.2 1B is cheaper at $0.027 / $0.20 per million input/output tokens, against $0.15 / $0.15 for Mistral Nemo.
  • Mistral Nemo accepts more context: 128K tokens versus 60K.

Side by side

Llama 3.2 1B and Mistral Nemo specifications
Llama 3.2 1BMistral Nemo
ProviderMetaMistral AI
Noometry Index20.126.4
Released2024-09-242024-07-01
WeightsOpenOpen
Context window60K128K
Max output54K128K
Input $ / M tokens$0.027$0.15
Output $ / M tokens$0.20$0.15
Results tracked2210

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Llama 3.2 1B: 21.1 (#338), Mistral Nemo: —

Coding benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
BigCodeBench Instruct8.2%—
LMArena Coding1070—
BigCodeBench Complete11.3%—

Agentic & Tool Use Mistral Nemo leads

Llama 3.2 1B: 14.6 (#150), Mistral Nemo: 23.5 (#125)

Agentic & Tool Use benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
Berkeley Function Calling Leaderboard10.8%27.6%
BALROG6.6%17.6%

Reasoning Mistral Nemo leads

Llama 3.2 1B: 16.2 (#308), Mistral Nemo: 20.7 (#232)

Reasoning benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
Epoch Capabilities Index101.99118.68
Chess Puzzles0%—
LMArena Hard Prompts1044—
DTBench—48.6%
PIQA—83.5%

Math Mistral Nemo leads

Llama 3.2 1B: 10.4 (#313), Mistral Nemo: 25.5 (#268)

Math benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
OTIS Mock AIME 2024-20250.6%—
LMArena Math1086—
MATH Level 5—10.8%
GSM8K—84.2%

Knowledge Mistral Nemo leads

Llama 3.2 1B: 7.2 (#312), Mistral Nemo: 12.3 (#298)

Knowledge benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
GPQA Diamond23.9%29.9%
LMArena Expert1007—
BoolQ—82.5%

Multilingual Not comparable

Llama 3.2 1B: 23.8 (#292), Mistral Nemo: —

Multilingual benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
LMArena Non-English973—
LMArena Chinese959—
LMArena German1014—
LMArena Russian941—

Instruction Following Not comparable

Llama 3.2 1B: 52.4 (#290), Mistral Nemo: —

Instruction Following benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
LMArena Instruction Following1031—

Long Context Not comparable

Llama 3.2 1B: 31.9 (#274), Mistral Nemo: —

Long Context benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
LMArena Longer Query1050—

Writing & Preference Mistral Nemo leads

Llama 3.2 1B: 21.3 (#310), Mistral Nemo: 28.5 (#296)

Writing & Preference benchmarks
BenchmarkLlama 3.2 1BMistral Nemo
EQ-Bench Creative Writing200881
LMArena Text1055—
LMArena Creative Writing1033—
LMArena Multi-Turn1030—

Frequently asked questions

Is Llama 3.2 1B better than Mistral Nemo?

Mistral Nemo is the stronger model overall, scoring 26.4 to 20.1 on the Noometry Index. Llama 3.2 1B costs 2.1× less per token, which makes it the better buy when Mistral Nemo's lead doesn't matter for your workload.

Which is cheaper, Llama 3.2 1B or Mistral Nemo?

Llama 3.2 1B is cheaper. It lists at $0.027 per million input tokens and $0.20 per million output tokens; Mistral Nemo lists at $0.15 and $0.15.

Which has the bigger context window?

Mistral Nemo does, with 128K tokens against 60K.

How many benchmarks do Llama 3.2 1B and Mistral Nemo share?

5 benchmarks have published results for both models. Llama 3.2 1B has 22 scored results on Noometry and Mistral Nemo has 10.

Related comparisons

Go deeper