Noometry

Model comparison

GPT-4 Turbo vs Inkling-Small

Inkling-Small is the stronger model overall, scoring 46.5 to 30.3 on the Noometry Index.

Last verified . 23 shared benchmarks.

GPT-4 Turbo OpenAI

30.3

Rank #281 Confirmed

Inkling-Small Thinking Machines Lab

46.5

Rank #57 Confirmed

Summary

  • They share 23 benchmarks with published results for both. GPT-4 Turbo scores higher in 0 categories and Inkling-Small in 9 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in math, where Inkling-Small leads 44.9 to 8.6.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 6.7% for GPT-4 Turbo and 90% for Inkling-Small.
  • Inkling-Small is cheaper at $0.45 / $1.20 per million input/output tokens, against $10 / $30 for GPT-4 Turbo.
  • Inkling-Small accepts more context: 524K tokens versus 128K.
  • Inkling-Small has downloadable open weights; the other is API-only.

Side by side

GPT-4 TurboInkling-Small
ProviderOpenAIThinking Machines Lab
Noometry Index30.346.5
Released2023-11-062026-07-15
WeightsProprietaryOpen
Context window128K524K
Max output4K1.05M
Input $ / M tokens$10$0.45
Output $ / M tokens$30$1.20
Results tracked3633

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Inkling-Small leads

GPT-4 Turbo: 33.4 (#247), Inkling-Small: 43.3 (#80)

BenchmarkGPT-4 TurboInkling-Small
LMArena Coding12681451
LMArena WebDev—1408
SciCode—48.7%
WeirdML18%—
BigCodeBench Instruct48.2%—
BigCodeBench Complete58.2%—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Inkling-Small: —

BenchmarkGPT-4 TurboInkling-Small
METR Time Horizons36.7%—

Reasoning Inkling-Small leads

GPT-4 Turbo: 14.9 (#311), Inkling-Small: 39.0 (#53)

BenchmarkGPT-4 TurboInkling-Small
Chess Puzzles6%18%
LMArena Hard Prompts12511424
Epoch Capabilities Index127.25150.13
ARC-AGI-2—40.1%
SimpleBench25.1%—
ARC-AGI-1—84%
CritPt—8.3%
Mystery Game Puzzles—6%
DTBench61.6%—
LMCA9.8%—
ForecastBench59.4—

Math Inkling-Small leads

GPT-4 Turbo: 8.6 (#316), Inkling-Small: 44.9 (#65)

BenchmarkGPT-4 TurboInkling-Small
FrontierMath (Tiers 1-3)0.7%46.3%
OTIS Mock AIME 2024-20256.7%90%
LMArena Math12721453
FrontierMath Tier 4—17.1%
ProofBench—6%
MATH Level 546.7%—

Knowledge Inkling-Small leads

GPT-4 Turbo: 24.3 (#263), Inkling-Small: 48.3 (#71)

BenchmarkGPT-4 TurboInkling-Small
GPQA Diamond46.6%88.5%
LMArena Expert12231442
SimpleQA Verified—19.1%
Confabulations28.4%—
MMLU81.3%—

Multimodal Inkling-Small leads

GPT-4 Turbo: 30.0 (#104), Inkling-Small: 38.4 (#59)

BenchmarkGPT-4 TurboInkling-Small
LMArena Vision10901235

Multilingual Inkling-Small leads

GPT-4 Turbo: 40.7 (#211), Inkling-Small: 51.7 (#102)

BenchmarkGPT-4 TurboInkling-Small
LMArena Non-English12451400
LMArena Chinese12421458
LMArena French12751438
LMArena German12591412
LMArena Japanese11941399
LMArena Korean11871355
LMArena Russian12591392
LMArena Spanish12601426

Instruction Following Inkling-Small leads

GPT-4 Turbo: 66.0 (#211), Inkling-Small: 73.9 (#112)

BenchmarkGPT-4 TurboInkling-Small
LMArena Instruction Following12501397

Long Context Inkling-Small leads

GPT-4 Turbo: 38.0 (#200), Inkling-Small: 42.7 (#113)

BenchmarkGPT-4 TurboInkling-Small
LMArena Longer Query12541401

Writing & Preference Inkling-Small leads

GPT-4 Turbo: 47.9 (#201), Inkling-Small: 59.8 (#102)

BenchmarkGPT-4 TurboInkling-Small
LMArena Text12721413
LMArena Creative Writing12691330
LMArena Multi-Turn12671420
EQ-Bench Creative Writing—1491

Frequently asked questions

Is GPT-4 Turbo better than Inkling-Small?

Inkling-Small is the stronger model overall, scoring 46.5 to 30.3 on the Noometry Index.

Which is cheaper, GPT-4 Turbo or Inkling-Small?

Inkling-Small is cheaper. It lists at $0.45 per million input tokens and $1.20 per million output tokens; GPT-4 Turbo lists at $10 and $30.

Is GPT-4 Turbo or Inkling-Small better for coding?

Inkling-Small scores higher on coding benchmarks: 43.3 versus 33.4 in the Noometry coding category.

Which has the bigger context window?

Inkling-Small does, with 524K tokens against 128K.

How many benchmarks do GPT-4 Turbo and Inkling-Small share?

23 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Inkling-Small has 33.

Related comparisons

Go deeper