Noometry

Model comparison

Llama 3.2 1B vs StarCoder 2 3B

Llama 3.2 1B has enough public results to be ranked (#354); StarCoder 2 3B does not yet, so treat this comparison as directional.

Last verified . 2 shared benchmarks.

Llama 3.2 1B Meta

20.1

Rank #354 Confirmed

StarCoder 2 3B NVIDIA

36.3

Unranked Sparse

Summary

  • They share 2 benchmarks with published results for both. Llama 3.2 1B scores higher in 0 categories and StarCoder 2 3B in 1 category; one gap is clear of the uncertainty.
  • The widest gap is in coding, where StarCoder 2 3B leads 33.5 to 21.1.
  • The biggest single-benchmark swing is BigCodeBench Complete: 11.3% for Llama 3.2 1B and 21.4% for StarCoder 2 3B.

Side by side

Llama 3.2 1BStarCoder 2 3B
ProviderMetaNVIDIA
Noometry Index20.136.3
Released2024-09-242024-02-22
WeightsOpenOpen
Context window60K—
Max output54K—
Input $ / M tokens$0.027—
Output $ / M tokens$0.20—
Results tracked227

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding StarCoder 2 3B leads

Llama 3.2 1B: 21.1 (#338), StarCoder 2 3B: 33.5

BenchmarkLlama 3.2 1BStarCoder 2 3B
BigCodeBench Complete11.3%21.4%
BigCodeBench Instruct8.2%—
LMArena Coding1070—
HumanEval+—27.4%

Agentic & Tool Use Not comparable

Llama 3.2 1B: 14.6 (#150), StarCoder 2 3B: —

BenchmarkLlama 3.2 1BStarCoder 2 3B
Berkeley Function Calling Leaderboard10.8%—
BALROG6.6%—

Reasoning Not comparable

Llama 3.2 1B: 16.2 (#308), StarCoder 2 3B: —

BenchmarkLlama 3.2 1BStarCoder 2 3B
Epoch Capabilities Index101.9988.76
Chess Puzzles0%—
LMArena Hard Prompts1044—
WinoGrande—57.1%

Math Not comparable

Llama 3.2 1B: 10.4 (#313), StarCoder 2 3B: —

BenchmarkLlama 3.2 1BStarCoder 2 3B
OTIS Mock AIME 2024-20250.6%—
LMArena Math1086—
GSM8K—21.6%

Knowledge Not comparable

Llama 3.2 1B: 7.2 (#312), StarCoder 2 3B: —

BenchmarkLlama 3.2 1BStarCoder 2 3B
GPQA Diamond23.9%—
LMArena Expert1007—
ARC (AI2) Challenge—34.2%
MMLU—36.6%

Multilingual Not comparable

Llama 3.2 1B: 23.8 (#292), StarCoder 2 3B: —

BenchmarkLlama 3.2 1BStarCoder 2 3B
LMArena Non-English973—
LMArena Chinese959—
LMArena German1014—
LMArena Russian941—

Instruction Following Not comparable

Llama 3.2 1B: 52.4 (#290), StarCoder 2 3B: —

BenchmarkLlama 3.2 1BStarCoder 2 3B
LMArena Instruction Following1031—

Long Context Not comparable

Llama 3.2 1B: 31.9 (#274), StarCoder 2 3B: —

BenchmarkLlama 3.2 1BStarCoder 2 3B
LMArena Longer Query1050—

Writing & Preference Not comparable

Llama 3.2 1B: 21.3 (#310), StarCoder 2 3B: —

BenchmarkLlama 3.2 1BStarCoder 2 3B
LMArena Text1055—
LMArena Creative Writing1033—
EQ-Bench Creative Writing200—
LMArena Multi-Turn1030—

Frequently asked questions

Is Llama 3.2 1B better than StarCoder 2 3B?

Llama 3.2 1B has enough public results to be ranked (#354); StarCoder 2 3B does not yet, so treat this comparison as directional.

Is Llama 3.2 1B or StarCoder 2 3B better for coding?

StarCoder 2 3B scores higher on coding benchmarks: 33.5 versus 21.1 in the Noometry coding category.

How many benchmarks do Llama 3.2 1B and StarCoder 2 3B share?

2 benchmarks have published results for both models. Llama 3.2 1B has 22 scored results on Noometry and StarCoder 2 3B has 7.

Related comparisons

Go deeper