Noometry

Model comparison

Inkling vs Llama 3.1-405B

Inkling is the stronger model overall, scoring 44.1 to 30.7 on the Noometry Index.

Last verified . 24 shared benchmarks.

Inkling Thinking Machines Lab

44.1

Rank #80 Confirmed

Llama 3.1-405B Meta

30.7

Rank #288 Confirmed

Summary

  • They share 24 benchmarks with published results for both. Inkling scores higher in 9 categories and Llama 3.1-405B in 0 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Inkling leads 65.2 to 38.9.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 88.9% for Inkling and 9.7% for Llama 3.1-405B.

Side by side

InklingLlama 3.1-405B
ProviderThinking Machines LabMeta
Noometry Index44.130.7
Released2026-07-152024-07-23
WeightsOpenOpen
Context window66K—
Max output66K—
Input $ / M tokens$1.87—
Output $ / M tokens$4.68—
Results tracked4142

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Inkling leads

Inkling: 34.5 (#234), Llama 3.1-405B: 33.1 (#262)

BenchmarkInklingLlama 3.1-405B
WeirdML32.3%21.4%
LMArena Coding14641291
FrontierCode14%—
LMArena WebDev1413—
FrontierSWE4.1%—
SciCode47%—
ALE-Bench946—

Agentic & Tool Use Inkling leads

Inkling: 29.6 (#85), Llama 3.1-405B: 21.0 (#140)

BenchmarkInklingLlama 3.1-405B
APEX-Agents33.8%—
TheAgentCompany—7.4%
τ²-bench Banking25%—
Cybench—7.5%

Reasoning Inkling leads

Inkling: 40.4 (#56), Llama 3.1-405B: 16.8 (#300)

BenchmarkInklingLlama 3.1-405B
SimpleBench50%23%
LMArena Hard Prompts14511269
DTBench87.5%61.4%
Epoch Capabilities Index148.54128.75
ARC-AGI-236.5%—
Kagi LLM Benchmark—45%
ARC-AGI-179.5%—
CritPt5.4%—
Chess Puzzles21%—
LMCA37.6%—
BIG-Bench Hard—82.9%
ForecastBench—59.9
HellaSwag—89.2%
PIQA—85.9%
WinoGrande—89.2%

Math Inkling leads

Inkling: 31.3 (#225), Llama 3.1-405B: 18.4 (#290)

BenchmarkInklingLlama 3.1-405B
OTIS Mock AIME 2024-202588.9%9.7%
LMArena Math14791281
FrontierMath (Tiers 1-3)33.3%—
FrontierMath Tier 44.9%—
ProofBench0%—
Omni-MATH—24.9%
MATH Level 5—49.8%

Knowledge Inkling leads

Inkling: 55.1 (#49), Llama 3.1-405B: 30.4 (#227)

BenchmarkInklingLlama 3.1-405B
GPQA Diamond88.3%50.9%
LMArena Expert14651243
SimpleQA Verified40.3%—
MMLU-Pro—72.3%
Confabulations—17.6%
GPQA (HELM)—52.2%
ARC (AI2) Challenge—95.3%
MMLU—84.5%
TriviaQA—82.7%

Multilingual Inkling leads

Inkling: 54.0 (#52), Llama 3.1-405B: 40.7 (#214)

BenchmarkInklingLlama 3.1-405B
LMArena Non-English14341248
LMArena Chinese14901242
LMArena French14581279
LMArena German14461252
LMArena Japanese14291208
LMArena Korean14041184
LMArena Russian14291265
LMArena Spanish14481260

Instruction Following Inkling leads

Inkling: 75.1 (#71), Llama 3.1-405B: 65.9 (#214)

BenchmarkInklingLlama 3.1-405B
LMArena Instruction Following14261259
IFEval—81.1%

Long Context Inkling leads

Inkling: 43.8 (#86), Llama 3.1-405B: 38.4 (#197)

BenchmarkInklingLlama 3.1-405B
LMArena Longer Query14341266

Writing & Preference Inkling leads

Inkling: 65.2 (#51), Llama 3.1-405B: 38.9 (#251)

BenchmarkInklingLlama 3.1-405B
LMArena Text14411284
LMArena Creative Writing13871262
EQ-Bench Creative Writing1611870
LMArena Multi-Turn14361297
WildBench—78.3%
EQ-Bench 41226—

Frequently asked questions

Is Inkling better than Llama 3.1-405B?

Inkling is the stronger model overall, scoring 44.1 to 30.7 on the Noometry Index.

Is Inkling or Llama 3.1-405B better for coding?

Inkling scores higher on coding benchmarks: 34.5 versus 33.1 in the Noometry coding category.

How many benchmarks do Inkling and Llama 3.1-405B share?

24 benchmarks have published results for both models. Inkling has 41 scored results on Noometry and Llama 3.1-405B has 42.

Related comparisons

Go deeper