Noometry

Model comparison

Inkling-Small vs Llama 3-70B

Inkling-Small is the stronger model overall, scoring 46.5 to 28.5 on the Noometry Index.

Last verified . 20 shared benchmarks.

Inkling-Small Thinking Machines Lab

46.5

Rank #57 Confirmed

Llama 3-70B Meta

28.5

Rank #317 Confirmed

Summary

  • They share 20 benchmarks with published results for both. Inkling-Small scores higher in 8 categories and Llama 3-70B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Inkling-Small leads 44.9 to 12.0.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 90% for Inkling-Small and 4.3% for Llama 3-70B.

Side by side

Inkling-SmallLlama 3-70B
ProviderThinking Machines LabMeta
Noometry Index46.528.5
Released2026-07-152024-04-18
WeightsOpenOpen
Context window524K—
Max output1.05M—
Input $ / M tokens$0.45—
Output $ / M tokens$1.20—
Results tracked3331

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Inkling-Small leads

Inkling-Small: 43.3 (#80), Llama 3-70B: 35.3 (#215)

BenchmarkInkling-SmallLlama 3-70B
LMArena Coding14511206
LMArena WebDev1408—
SciCode48.7%—
BigCodeBench Instruct—43.6%
BigCodeBench Complete—54.5%
HumanEval+—72%
MBPP+—69%

Agentic & Tool Use Not comparable

Inkling-Small: —, Llama 3-70B: 20.9 (#132)

BenchmarkInkling-SmallLlama 3-70B
Cybench—5%

Reasoning Inkling-Small leads

Inkling-Small: 39.0 (#53), Llama 3-70B: 17.4 (#284)

BenchmarkInkling-SmallLlama 3-70B
LMArena Hard Prompts14241195
Epoch Capabilities Index150.13122.91
ARC-AGI-240.1%—
Kagi LLM Benchmark—35.1%
ARC-AGI-184%—
CritPt8.3%—
Chess Puzzles18%—
Mystery Game Puzzles6%—
DTBench—54.2%
ForecastBench—57.1
WinoGrande—83.5%

Math Inkling-Small leads

Inkling-Small: 44.9 (#65), Llama 3-70B: 12.0 (#299)

BenchmarkInkling-SmallLlama 3-70B
OTIS Mock AIME 2024-202590%4.3%
LMArena Math14531218
FrontierMath (Tiers 1-3)46.3%—
FrontierMath Tier 417.1%—
ProofBench6%—
MATH Level 5—22.6%

Knowledge Inkling-Small leads

Inkling-Small: 48.3 (#71), Llama 3-70B: 20.8 (#272)

BenchmarkInkling-SmallLlama 3-70B
GPQA Diamond88.5%40.6%
LMArena Expert14421149
SimpleQA Verified19.1%—
MMLU—79.3%

Multimodal Not comparable

Inkling-Small: 38.4 (#59), Llama 3-70B: —

BenchmarkInkling-SmallLlama 3-70B
LMArena Vision1235—

Multilingual Inkling-Small leads

Inkling-Small: 51.7 (#102), Llama 3-70B: 33.8 (#246)

BenchmarkInkling-SmallLlama 3-70B
LMArena Non-English14001142
LMArena Chinese14581114
LMArena French14381231
LMArena German14121169
LMArena Japanese13991017
LMArena Korean13551017
LMArena Russian13921159
LMArena Spanish14261241

Instruction Following Inkling-Small leads

Inkling-Small: 73.9 (#112), Llama 3-70B: 62.8 (#233)

BenchmarkInkling-SmallLlama 3-70B
LMArena Instruction Following13971194

Long Context Inkling-Small leads

Inkling-Small: 42.7 (#113), Llama 3-70B: 35.6 (#235)

BenchmarkInkling-SmallLlama 3-70B
LMArena Longer Query14011174

Writing & Preference Inkling-Small leads

Inkling-Small: 59.8 (#102), Llama 3-70B: 43.0 (#225)

BenchmarkInkling-SmallLlama 3-70B
LMArena Text14131221
LMArena Creative Writing13301210
LMArena Multi-Turn14201223
EQ-Bench Creative Writing1491—

Frequently asked questions

Is Inkling-Small better than Llama 3-70B?

Inkling-Small is the stronger model overall, scoring 46.5 to 28.5 on the Noometry Index.

Is Inkling-Small or Llama 3-70B better for coding?

Inkling-Small scores higher on coding benchmarks: 43.3 versus 35.3 in the Noometry coding category.

How many benchmarks do Inkling-Small and Llama 3-70B share?

20 benchmarks have published results for both models. Inkling-Small has 33 scored results on Noometry and Llama 3-70B has 31.

Related comparisons

Go deeper