Noometry

Model comparison

Llama-3.3-70B-Instruct vs Molmo 2 8b

Molmo 2 8b is the stronger model overall, scoring 39.1 to 30.6 on the Noometry Index.

Last verified . 4 shared benchmarks.

Llama-3.3-70B-Instruct Meta

30.6

Rank #291 Confirmed

Molmo 2 8b Allen Institute for AI (Ai2)

39.1

Rank #174 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Llama-3.3-70B-Instruct scores higher in 1 category and Molmo 2 8b in 3 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Molmo 2 8b leads 25.6 to 14.1.

Side by side

Llama-3.3-70B-InstructMolmo 2 8b
ProviderMetaAllen Institute for AI (Ai2)
Noometry Index30.639.1
Released2024-12-06—
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$0.10—
Output $ / M tokens$0.32—
Results tracked435

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Llama-3.3-70B-Instruct: 31.0 (#290), Molmo 2 8b: —

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
SciCode26%—
WeirdML14.4%—
BigCodeBench Instruct46.9%—
LiveBench Coding36.6%—
LMArena Coding1268—
BigCodeBench Complete57.5%—

Agentic & Tool Use Not comparable

Llama-3.3-70B-Instruct: 25.8 (#105), Molmo 2 8b: —

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
Berkeley Function Calling Leaderboard31.9%—
BALROG23%—

Reasoning Molmo 2 8b leads

Llama-3.3-70B-Instruct: 14.1 (#327), Molmo 2 8b: 25.6 (#146)

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
LMArena Hard Prompts12571287
SimpleBench19.9%—
CritPt0%—
LiveBench Reasoning50.8%—
DTBench59.5%—
LiveBench Data Analysis49.5%—
LMCA17.5%—
Epoch Capabilities Index127.33—
ForecastBench58.6—
LiveBench50.2%—

Math Not comparable

Llama-3.3-70B-Instruct: 15.3 (#298), Molmo 2 8b: —

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
OTIS Mock AIME 2024-20255.1%—
LiveBench Math42.2%—
LMArena Math1267—
MATH Level 541.6%—

Knowledge Not comparable

Llama-3.3-70B-Instruct: 30.6 (#226), Molmo 2 8b: —

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
GPQA Diamond47.4%—
Confabulations22.8%—
Vectara Hallucination Rate4.1%—
LMArena Expert1225—
MMLU86.3%—

Multimodal Not comparable

Llama-3.3-70B-Instruct: —, Molmo 2 8b: 30.2 (#112)

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
LMArena Vision—1081

Multilingual Molmo 2 8b leads

Llama-3.3-70B-Instruct: 39.9 (#220), Molmo 2 8b: 42.7 (#190)

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
LMArena Non-English12361276
LMArena Chinese1217—
LMArena French1281—
LMArena German1251—
LMArena Japanese1150—
LMArena Korean1143—
LMArena Russian1252—
LMArena Spanish1270—

Instruction Following Llama-3.3-70B-Instruct leads

Llama-3.3-70B-Instruct: 71.1 (#157), Molmo 2 8b: 67.0 (#201)

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
LMArena Instruction Following12421270
LiveBench Instruction Following82.7%—

Long Context Not comparable

Llama-3.3-70B-Instruct: 26.4 (#295), Molmo 2 8b: —

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
Fiction.LiveBench33.3%—
LMArena Longer Query1256—

Writing & Preference Molmo 2 8b leads

Llama-3.3-70B-Instruct: 47.6 (#207), Molmo 2 8b: 49.4 (#191)

BenchmarkLlama-3.3-70B-InstructMolmo 2 8b
LMArena Text12741288
LMArena Creative Writing1250—
LMArena Multi-Turn1280—
LiveBench Language39.2%—

Frequently asked questions

Is Llama-3.3-70B-Instruct better than Molmo 2 8b?

Molmo 2 8b is the stronger model overall, scoring 39.1 to 30.6 on the Noometry Index.

How many benchmarks do Llama-3.3-70B-Instruct and Molmo 2 8b share?

4 benchmarks have published results for both models. Llama-3.3-70B-Instruct has 43 scored results on Noometry and Molmo 2 8b has 5.

Related comparisons

Go deeper