Model comparison
Inkling vs o3
o3 is the stronger model overall, scoring 47.4 to 44.1 on the Noometry Index.
Last verified . 32 shared benchmarks.
Summary
- They share 32 benchmarks with published results for both. Inkling scores higher in 5 categories and o3 in 4 categories; 8 gaps are clear of the uncertainty.
- The widest gap is in math, where o3 leads 48.9 to 31.2.
- The biggest single-benchmark swing is ARC-AGI-2: 36.5% for Inkling and 6.5% for o3.
- Inkling is cheaper at $1.87 / $4.68 per million input/output tokens, against $2 / $8 for o3.
- o3 accepts more context: 200K tokens versus 66K.
- Inkling has downloadable open weights; the other is API-only.
Side by side
| Inkling | o3 | |
|---|---|---|
| Provider | Thinking Machines Lab | OpenAI |
| Noometry Index | 44.1 | 47.4 |
| Released | 2026-07-15 | 2025-04-16 |
| Weights | Open | Proprietary |
| Context window | 66K | 200K |
| Max output | 66K | 100K |
| Input $ / M tokens | $1.87 | $2 |
| Output $ / M tokens | $4.68 | $8 |
| Results tracked | 41 | 63 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding o3 leads
Inkling: 34.1 (#229), o3: 47.3 (#57)
| Benchmark | Inkling | o3 |
|---|---|---|
| WeirdML | 32.3% | 52.4% |
| LMArena Coding | 1463 | 1408 |
| ALE-Bench | 946 | 933.55 |
| SWE-bench Verified | — | 62.3% |
| FrontierCode | 14% | — |
| SWE-bench Verified (bash only) | — | 58.4% |
| Aider Polyglot | — | 81.3% |
| LMArena WebDev | 1412 | — |
| FrontierSWE | 4.1% | — |
| SciCode | 47% | — |
| GSO | — | 8.8% |
| CadEval | — | 74% |
Agentic & Tool Use o3 leads
Inkling: 29.5 (#76), o3: 34.3 (#42)
| Benchmark | Inkling | o3 |
|---|---|---|
| APEX-Agents | 33.8% | — |
| Berkeley Function Calling Leaderboard | — | 63% |
| GDPval | — | 30.8% |
| τ²-bench Banking | 25% | — |
| DeepResearch Bench | — | 45.2% |
| OSWorld | — | 23% |
| LMArena Search | — | 1144 |
| METR Time Horizons | — | 65.4% |
Reasoning Inkling leads
Inkling: 40.6 (#48), o3: 32.1 (#66)
| Benchmark | Inkling | o3 |
|---|---|---|
| ARC-AGI-2 | 36.5% | 6.5% |
| SimpleBench | 50% | 53.1% |
| ARC-AGI-1 | 79.5% | 60.8% |
| CritPt | 5.4% | 1.4% |
| Chess Puzzles | 21% | 38% |
| LMArena Hard Prompts | 1452 | 1402 |
| DTBench | 87.5% | 84.8% |
| LMCA | 37.6% | 39.7% |
| Epoch Capabilities Index | 148.6 | 146.88 |
| Kagi LLM Benchmark | — | 67.6% |
| EnigmaEval | — | 13.1% |
| Mystery Game Puzzles | — | 29% |
| ForecastBench | — | 62.5 |
Math o3 leads
Inkling: 31.2 (#198), o3: 48.9 (#49)
| Benchmark | Inkling | o3 |
|---|---|---|
| FrontierMath (Tiers 1-3) | 33.3% | 33.3% |
| OTIS Mock AIME 2024-2025 | 88.9% | 84.4% |
| LMArena Math | 1482 | 1426 |
| FrontierMath Tier 4 | 4.9% | — |
| ProofBench | 0% | — |
| Omni-MATH | — | 71.4% |
| MATH Level 5 | — | 97.8% |
| FrontierMath (Feb 2025 set) | — | 18.7% |
| FrontierMath Tier 4 (v1) | — | 2.1% |
Knowledge Too close to call
Inkling: 55.2 (#44), o3: 54.5 (#47)
| Benchmark | Inkling | o3 |
|---|---|---|
| GPQA Diamond | 88.3% | 81.8% |
| SimpleQA Verified | 40.3% | 49.4% |
| LMArena Expert | 1468 | 1403 |
| Humanity's Last Exam | — | 20.3% |
| MMLU-Pro | — | 85.9% |
| Confabulations | — | 14.4% |
| GPQA (HELM) | — | 75.3% |
Multimodal Not comparable
Inkling: —, o3: 40.7 (#35)
| Benchmark | Inkling | o3 |
|---|---|---|
| LMArena Vision | — | 1214 |
| GeoBench | — | 74% |
| VPCT | — | 52% |
Multilingual Inkling leads
Inkling: 54.2 (#49), o3: 51.9 (#99)
| Benchmark | Inkling | o3 |
|---|---|---|
| LMArena Non-English | 1434 | 1402 |
| LMArena Chinese | 1495 | 1438 |
| LMArena French | 1453 | 1430 |
| LMArena German | 1456 | 1420 |
| LMArena Japanese | 1428 | 1404 |
| LMArena Korean | 1401 | 1371 |
| LMArena Russian | 1430 | 1407 |
| LMArena Spanish | 1442 | 1395 |
Instruction Following Inkling leads
Inkling: 75.3 (#65), o3: 73.0 (#122)
| Benchmark | Inkling | o3 |
|---|---|---|
| LMArena Instruction Following | 1427 | 1368 |
| IFEval | — | 86.9% |
Long Context o3 leads
Inkling: 43.9 (#79), o3: 53.2 (#6)
| Benchmark | Inkling | o3 |
|---|---|---|
| LMArena Longer Query | 1436 | 1372 |
| Fiction.LiveBench | — | 88.9% |
| CL-bench | — | 17.8% |
Writing & Preference Inkling leads
Inkling: 65.5 (#48), o3: 63.7 (#62)
| Benchmark | Inkling | o3 |
|---|---|---|
| LMArena Text | 1443 | 1410 |
| LMArena Creative Writing | 1390 | 1359 |
| EQ-Bench Creative Writing | 1611 | 1676 |
| LMArena Multi-Turn | 1436 | 1405 |
| Short-Story Creative Writing | — | 83.9% |
| WildBench | — | 86.1% |
| EQ-Bench 4 | 1226 | — |
Frequently asked questions
Is Inkling better than o3?
o3 is the stronger model overall, scoring 47.4 to 44.1 on the Noometry Index.
Which is cheaper, Inkling or o3?
Inkling is cheaper. It lists at $1.87 per million input tokens and $4.68 per million output tokens; o3 lists at $2 and $8.
Is Inkling or o3 better for coding?
o3 scores higher on coding benchmarks: 47.3 versus 34.1 in the Noometry coding category.
Which has the bigger context window?
o3 does, with 200K tokens against 66K.
How many benchmarks do Inkling and o3 share?
32 benchmarks have published results for both models. Inkling has 41 scored results on Noometry and o3 has 63.