Noometry

Model comparison

Gemini 2.0 Flash-Lite vs Tulu 3 (Tülu 3) 70B

Gemini 2.0 Flash-Lite is the stronger model overall, scoring 37.3 to 32.6 on the Noometry Index.

Last verified . 11 shared benchmarks.

Gemini 2.0 Flash-Lite Google

37.3

Rank #189 Confirmed

Tulu 3 (Tülu 3) 70B Allen Institute for AI (Ai2)

32.6

Rank #242 Confirmed

Summary

  • They share 11 benchmarks with published results for both. Gemini 2.0 Flash-Lite scores higher in 7 categories and Tulu 3 (Tülu 3) 70B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Gemini 2.0 Flash-Lite leads 31.8 to 13.3.
  • Tulu 3 (Tülu 3) 70B has downloadable open weights; the other is API-only.

Side by side

Gemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
ProviderGoogleAllen Institute for AI (Ai2)
Noometry Index37.332.6
Released2025-02-052024-11-21
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3214

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 2.0 Flash-Lite leads

Gemini 2.0 Flash-Lite: 37.4 (#181), Tulu 3 (Tülu 3) 70B: 35.4 (#211)

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
LMArena Coding13221236
LiveBench Coding47.1%—

Reasoning Tulu 3 (Tülu 3) 70B leads

Gemini 2.0 Flash-Lite: 21.2 (#203), Tulu 3 (Tülu 3) 70B: 22.9 (#172)

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
LMArena Hard Prompts13241220
LiveBench Reasoning50.1%—
DTBench52.5%—
LiveBench Data Analysis65.5%—
ForecastBench57.1—
LiveBench54.3%—

Math Gemini 2.0 Flash-Lite leads

Gemini 2.0 Flash-Lite: 31.8 (#192), Tulu 3 (Tülu 3) 70B: 13.3 (#297)

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
LMArena Math13091242
OTIS Mock AIME 2024-2025—4.4%
Omni-MATH37.4%—
LiveBench Math58.1%—
MATH Level 5—42.7%

Knowledge Gemini 2.0 Flash-Lite leads

Gemini 2.0 Flash-Lite: 34.6 (#185), Tulu 3 (Tülu 3) 70B: 25.1 (#257)

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
GPQA Diamond—46.3%
MMLU-Pro72%—
GPQA (HELM)50%—
LMArena Expert1305—

Multimodal Not comparable

Gemini 2.0 Flash-Lite: 30.6 (#103), Tulu 3 (Tülu 3) 70B: —

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
LMArena Vision1100—

Multilingual Gemini 2.0 Flash-Lite leads

Gemini 2.0 Flash-Lite: 46.2 (#156), Tulu 3 (Tülu 3) 70B: 40.1 (#217)

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
LMArena Non-English13231236
LMArena Chinese13391249
LMArena Russian13281246
LMArena French1346—
LMArena German1306—
LMArena Japanese1302—
LMArena Korean1325—
LMArena Spanish1313—

Instruction Following Gemini 2.0 Flash-Lite leads

Gemini 2.0 Flash-Lite: 70.6 (#158), Tulu 3 (Tülu 3) 70B: 65.0 (#222)

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
LMArena Instruction Following13051233
LiveBench Instruction Following78.3%—
IFEval82.4%—

Long Context Gemini 2.0 Flash-Lite leads

Gemini 2.0 Flash-Lite: 40.1 (#155), Tulu 3 (Tülu 3) 70B: 37.1 (#217)

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
LMArena Longer Query13211224

Writing & Preference Gemini 2.0 Flash-Lite leads

Gemini 2.0 Flash-Lite: 51.9 (#172), Tulu 3 (Tülu 3) 70B: 45.8 (#218)

BenchmarkGemini 2.0 Flash-LiteTulu 3 (Tülu 3) 70B
LMArena Text13301256
LMArena Creative Writing13191231
LMArena Multi-Turn13071252
WildBench79%—
LiveBench Language34.3%—

Frequently asked questions

Is Gemini 2.0 Flash-Lite better than Tulu 3 (Tülu 3) 70B?

Gemini 2.0 Flash-Lite is the stronger model overall, scoring 37.3 to 32.6 on the Noometry Index.

Is Gemini 2.0 Flash-Lite or Tulu 3 (Tülu 3) 70B better for coding?

Gemini 2.0 Flash-Lite scores higher on coding benchmarks: 37.4 versus 35.4 in the Noometry coding category.

How many benchmarks do Gemini 2.0 Flash-Lite and Tulu 3 (Tülu 3) 70B share?

11 benchmarks have published results for both models. Gemini 2.0 Flash-Lite has 32 scored results on Noometry and Tulu 3 (Tülu 3) 70B has 14.

Related comparisons

Go deeper