Noometry

Model comparison

Chatgpt 4o Latest 20250326 vs Tulu 3 (Tülu 3) 70B

Chatgpt 4o Latest 20250326 is the stronger model overall, scoring 43.8 to 33.0 on the Noometry Index.

Last verified . 11 shared benchmarks.

Chatgpt 4o Latest 20250326 OpenAI

43.8

Rank #82 Confirmed

Tulu 3 (Tülu 3) 70B Allen Institute for AI (Ai2)

33.0

Rank #251 Confirmed

Summary

  • They share 11 benchmarks with published results for both. Chatgpt 4o Latest 20250326 scores higher in 8 categories and Tulu 3 (Tülu 3) 70B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Chatgpt 4o Latest 20250326 leads 38.6 to 14.2.
  • Tulu 3 (Tülu 3) 70B has downloadable open weights; the other is API-only.

Side by side

Chatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
ProviderOpenAIAllen Institute for AI (Ai2)
Noometry Index43.833.0
Released—2024-11-21
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2114

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Chatgpt 4o Latest 20250326 leads

Chatgpt 4o Latest 20250326: 41.6 (#122), Tulu 3 (Tülu 3) 70B: 36.0 (#214)

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
LMArena Coding14131235

Reasoning Chatgpt 4o Latest 20250326 leads

Chatgpt 4o Latest 20250326: 33.8 (#71), Tulu 3 (Tülu 3) 70B: 23.9 (#169)

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
LMArena Hard Prompts14241220
Kagi LLM Benchmark75%—

Math Chatgpt 4o Latest 20250326 leads

Chatgpt 4o Latest 20250326: 38.6 (#134), Tulu 3 (Tülu 3) 70B: 14.2 (#303)

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
LMArena Math14071242
OTIS Mock AIME 2024-2025—4.4%
MATH Level 5—42.7%

Knowledge Chatgpt 4o Latest 20250326 leads

Chatgpt 4o Latest 20250326: 39.2 (#137), Tulu 3 (Tülu 3) 70B: 25.0 (#264)

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
GPQA Diamond—46.3%
Confabulations16.6%—
LMArena Expert1401—

Multimodal Not comparable

Chatgpt 4o Latest 20250326: 39.6 (#58), Tulu 3 (Tülu 3) 70B: —

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
LMArena Vision1243—

Multilingual Chatgpt 4o Latest 20250326 leads

Chatgpt 4o Latest 20250326: 52.9 (#76), Tulu 3 (Tülu 3) 70B: 39.9 (#222)

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
LMArena Non-English14191236
LMArena Chinese14571249
LMArena Russian14291246
LMArena French1446—
LMArena German1423—
LMArena Japanese1405—
LMArena Korean1396—
LMArena Spanish1434—

Instruction Following Chatgpt 4o Latest 20250326 leads

Chatgpt 4o Latest 20250326: 74.0 (#107), Tulu 3 (Tülu 3) 70B: 64.8 (#227)

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
LMArena Instruction Following14031233

Long Context Chatgpt 4o Latest 20250326 leads

Chatgpt 4o Latest 20250326: 43.1 (#107), Tulu 3 (Tülu 3) 70B: 37.1 (#222)

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
LMArena Longer Query14131224

Writing & Preference Chatgpt 4o Latest 20250326 leads

Chatgpt 4o Latest 20250326: 62.6 (#72), Tulu 3 (Tülu 3) 70B: 45.6 (#223)

BenchmarkChatgpt 4o Latest 20250326Tulu 3 (Tülu 3) 70B
LMArena Text14291256
LMArena Creative Writing14051231
LMArena Multi-Turn14541252
EQ-Bench Creative Writing1501—

Frequently asked questions

Is Chatgpt 4o Latest 20250326 better than Tulu 3 (Tülu 3) 70B?

Chatgpt 4o Latest 20250326 is the stronger model overall, scoring 43.8 to 33.0 on the Noometry Index.

Is Chatgpt 4o Latest 20250326 or Tulu 3 (Tülu 3) 70B better for coding?

Chatgpt 4o Latest 20250326 scores higher on coding benchmarks: 41.6 versus 36.0 in the Noometry coding category.

How many benchmarks do Chatgpt 4o Latest 20250326 and Tulu 3 (Tülu 3) 70B share?

11 benchmarks have published results for both models. Chatgpt 4o Latest 20250326 has 21 scored results on Noometry and Tulu 3 (Tülu 3) 70B has 14.

Related comparisons

Go deeper