Noometry

Model comparison

Gemini 1.0 Pro vs Llama 3.2 11B

Gemini 1.0 Pro has enough public results to be ranked (#327); Llama 3.2 11B does not yet, so treat this comparison as directional.

Last verified . 1 shared benchmarks.

Gemini 1.0 Pro Google

27.1

Rank #327 Confirmed

Llama 3.2 11B Meta

31.6

Unranked Sparse

Summary

  • They share 1 benchmark with published results for both.
  • Llama 3.2 11B has downloadable open weights; the other is API-only.

Side by side

Gemini 1.0 ProLlama 3.2 11B
ProviderGoogleMeta
Noometry Index27.131.6
Released2023-12-132024-09-24
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked243

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemini 1.0 Pro: 31.6 (#270), Llama 3.2 11B: —

BenchmarkGemini 1.0 ProLlama 3.2 11B
LMArena Coding1108—
HumanEval+55.5%—
MBPP+61.4%—

Agentic & Tool Use Not comparable

Gemini 1.0 Pro: —, Llama 3.2 11B: 27.0

BenchmarkGemini 1.0 ProLlama 3.2 11B
BALROG—16.8%

Reasoning Not comparable

Gemini 1.0 Pro: 16.4 (#292), Llama 3.2 11B: —

BenchmarkGemini 1.0 ProLlama 3.2 11B
LMArena Hard Prompts1109—
DTBench45.9%—
Epoch Capabilities Index116.97—

Math Not comparable

Gemini 1.0 Pro: 8.7 (#315), Llama 3.2 11B: —

BenchmarkGemini 1.0 ProLlama 3.2 11B
OTIS Mock AIME 2024-20251.1%—
LMArena Math1132—
MATH Level 511.2%—

Knowledge Not comparable

Gemini 1.0 Pro: 15.5 (#286), Llama 3.2 11B: —

BenchmarkGemini 1.0 ProLlama 3.2 11B
MMLU70%56.5%
GPQA Diamond34%—
LMArena Expert1059—

Multimodal Not comparable

Gemini 1.0 Pro: —, Llama 3.2 11B: 23.8

BenchmarkGemini 1.0 ProLlama 3.2 11B
LMArena Vision—966

Multilingual Not comparable

Gemini 1.0 Pro: 33.5 (#247), Llama 3.2 11B: —

BenchmarkGemini 1.0 ProLlama 3.2 11B
LMArena Non-English1138—
LMArena Chinese1124—
LMArena French1144—
LMArena German1125—
LMArena Japanese1023—
LMArena Russian1186—
LMArena Spanish1118—

Instruction Following Not comparable

Gemini 1.0 Pro: 57.9 (#262), Llama 3.2 11B: —

BenchmarkGemini 1.0 ProLlama 3.2 11B
LMArena Instruction Following1114—

Long Context Not comparable

Gemini 1.0 Pro: 34.3 (#244), Llama 3.2 11B: —

BenchmarkGemini 1.0 ProLlama 3.2 11B
LMArena Longer Query1132—

Writing & Preference Not comparable

Gemini 1.0 Pro: 36.2 (#259), Llama 3.2 11B: —

BenchmarkGemini 1.0 ProLlama 3.2 11B
LMArena Text1149—
LMArena Creative Writing1130—
LMArena Multi-Turn1139—

Frequently asked questions

Is Gemini 1.0 Pro better than Llama 3.2 11B?

Gemini 1.0 Pro has enough public results to be ranked (#327); Llama 3.2 11B does not yet, so treat this comparison as directional.

How many benchmarks do Gemini 1.0 Pro and Llama 3.2 11B share?

1 benchmark has published results for both models. Gemini 1.0 Pro has 24 scored results on Noometry and Llama 3.2 11B has 3.

Related comparisons

Go deeper