Noometry

Model comparison

Llama 4 Scout vs Tulu 3 (Tülu 3) 70B

Tulu 3 (Tülu 3) 70B is the stronger model overall, scoring 33.0 to 27.7 on the Noometry Index.

Last verified . 14 shared benchmarks.

Llama 4 Scout Meta

27.7

Rank #330 Confirmed

Tulu 3 (Tülu 3) 70B Allen Institute for AI (Ai2)

33.0

Rank #251 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Llama 4 Scout scores higher in 4 categories and Tulu 3 (Tülu 3) 70B in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Tulu 3 (Tülu 3) 70B leads 36.0 to 20.2.
  • The biggest single-benchmark swing is MATH Level 5: 62.3% for Llama 4 Scout and 42.7% for Tulu 3 (Tülu 3) 70B.

Side by side

Llama 4 ScoutTulu 3 (Tülu 3) 70B
ProviderMetaAllen Institute for AI (Ai2)
Noometry Index27.733.0
Released2025-04-052024-11-21
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$0.10—
Output $ / M tokens$0.30—
Results tracked4314

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Tulu 3 (Tülu 3) 70B leads

Llama 4 Scout: 20.2 (#339), Tulu 3 (Tülu 3) 70B: 36.0 (#214)

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
LMArena Coding12861235
SWE-bench Verified (bash only)9.1%—
SciCode17%—
BigCodeBench Complete43.1%—

Agentic & Tool Use Not comparable

Llama 4 Scout: 24.6 (#119), Tulu 3 (Tülu 3) 70B: —

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
Berkeley Function Calling Leaderboard28.1%—

Reasoning Tulu 3 (Tülu 3) 70B leads

Llama 4 Scout: 9.1 (#345), Tulu 3 (Tülu 3) 70B: 23.9 (#169)

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
LMArena Hard Prompts12661220
ARC-AGI-20%—
Kagi LLM Benchmark36.9%—
ARC-AGI-10.5%—
CritPt0%—
DTBench57.9%—
LMCA12%—
Epoch Capabilities Index129.64—
ForecastBench57.5—

Math Llama 4 Scout leads

Llama 4 Scout: 19.6 (#286), Tulu 3 (Tülu 3) 70B: 14.2 (#303)

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
OTIS Mock AIME 2024-20257.8%4.4%
LMArena Math12871242
MATH Level 562.3%42.7%
Omni-MATH37.3%—
FrontierMath (Feb 2025 set)0%—

Knowledge Llama 4 Scout leads

Llama 4 Scout: 31.9 (#217), Tulu 3 (Tülu 3) 70B: 25.0 (#264)

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
GPQA Diamond51.8%46.3%
MMLU-Pro74.2%—
Vectara Hallucination Rate7.7%—
GPQA (HELM)50.7%—
LMArena Expert1235—

Multimodal Not comparable

Llama 4 Scout: 32.2 (#102), Tulu 3 (Tülu 3) 70B: —

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
LMArena Vision1118—
SpatialViz-Bench34.2%—

Multilingual Llama 4 Scout leads

Llama 4 Scout: 41.0 (#212), Tulu 3 (Tülu 3) 70B: 39.9 (#222)

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
LMArena Non-English12521236
LMArena Chinese12551249
LMArena Russian12631246
LMArena French1282—
LMArena German1272—
LMArena Japanese1206—
LMArena Korean1207—
LMArena Spanish1278—

Instruction Following Too close to call

Llama 4 Scout: 65.8 (#217), Tulu 3 (Tülu 3) 70B: 64.8 (#227)

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
LMArena Instruction Following12481233
IFEval81.8%—

Long Context Tulu 3 (Tülu 3) 70B leads

Llama 4 Scout: 27.5 (#294), Tulu 3 (Tülu 3) 70B: 37.1 (#222)

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
LMArena Longer Query12651224
Fiction.LiveBench36%—

Writing & Preference Tulu 3 (Tülu 3) 70B leads

Llama 4 Scout: 37.0 (#261), Tulu 3 (Tülu 3) 70B: 45.6 (#223)

BenchmarkLlama 4 ScoutTulu 3 (Tülu 3) 70B
LMArena Text12791256
LMArena Creative Writing12491231
LMArena Multi-Turn12801252
EQ-Bench Creative Writing783—
WildBench78%—

Frequently asked questions

Is Llama 4 Scout better than Tulu 3 (Tülu 3) 70B?

Tulu 3 (Tülu 3) 70B is the stronger model overall, scoring 33.0 to 27.7 on the Noometry Index.

Is Llama 4 Scout or Tulu 3 (Tülu 3) 70B better for coding?

Tulu 3 (Tülu 3) 70B scores higher on coding benchmarks: 36.0 versus 20.2 in the Noometry coding category.

How many benchmarks do Llama 4 Scout and Tulu 3 (Tülu 3) 70B share?

14 benchmarks have published results for both models. Llama 4 Scout has 43 scored results on Noometry and Tulu 3 (Tülu 3) 70B has 14.

Related comparisons

Go deeper