Noometry

Model comparison

Gemini 1.5 Flash 8B vs GPT-4.1 nano

Gemini 1.5 Flash 8B is the stronger model overall, scoring 29.9 to 27.9 on the Noometry Index.

Last verified . 18 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

GPT-4.1 nano OpenAI

27.9

Rank #327 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 4 categories and GPT-4.1 nano in 5 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Gemini 1.5 Flash 8B leads 37.0 to 23.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.6% for Gemini 1.5 Flash 8B and 28.9% for GPT-4.1 nano.

Side by side

Gemini 1.5 Flash 8BGPT-4.1 nano
ProviderGoogleOpenAI
Noometry Index29.927.9
Released2024-10-032025-04-14
WeightsProprietaryProprietary
Context window—1.05M
Max output—33K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.40
Results tracked2138

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 35.5 (#225), GPT-4.1 nano: 24.1 (#330)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
LMArena Coding12181306
Aider Polyglot—8.9%
SciCode—25.9%
WeirdML—19%

Agentic & Tool Use Not comparable

Gemini 1.5 Flash 8B: —, GPT-4.1 nano: 26.5 (#104)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
Berkeley Function Calling Leaderboard—33%

Reasoning Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 20.0 (#244), GPT-4.1 nano: 8.5 (#349)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
LMArena Hard Prompts12091286
DTBench50%52.5%
ARC-AGI-2—0%
Kagi LLM Benchmark—33.3%
ARC-AGI-1—0%
CritPt—0%
LMCA—5.5%
Epoch Capabilities Index—129.62

Math GPT-4.1 nano leads

Gemini 1.5 Flash 8B: 14.2 (#302), GPT-4.1 nano: 26.9 (#252)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
OTIS Mock AIME 2024-20254.6%28.9%
LMArena Math12071274
Omni-MATH—36.7%
MATH Level 5—70%
FrontierMath (Feb 2025 set)—1%

Knowledge GPT-4.1 nano leads

Gemini 1.5 Flash 8B: 16.0 (#289), GPT-4.1 nano: 21.8 (#273)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
GPQA Diamond33%48.9%
LMArena Expert11851272
SimpleQA Verified—6%
MMLU-Pro—55%
GPQA (HELM)—50.7%

Multimodal GPT-4.1 nano leads

Gemini 1.5 Flash 8B: 28.2 (#115), GPT-4.1 nano: 29.2 (#113)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
LMArena Vision10441063

Multilingual GPT-4.1 nano leads

Gemini 1.5 Flash 8B: 38.5 (#229), GPT-4.1 nano: 41.6 (#205)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
LMArena Non-English12151260
LMArena Chinese12311270
LMArena German12061288
LMArena Japanese11501198
LMArena Russian12361261
LMArena French1234—
LMArena Korean1140—
LMArena Spanish1212—

Instruction Following GPT-4.1 nano leads

Gemini 1.5 Flash 8B: 62.8 (#236), GPT-4.1 nano: 67.8 (#193)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
LMArena Instruction Following11991267
IFEval—84.3%

Long Context Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 37.0 (#225), GPT-4.1 nano: 23.7 (#296)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
LMArena Longer Query12191283
Fiction.LiveBench—25%

Writing & Preference Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 42.8 (#232), GPT-4.1 nano: 40.5 (#243)

BenchmarkGemini 1.5 Flash 8BGPT-4.1 nano
LMArena Text12261285
LMArena Creative Writing12181260
LMArena Multi-Turn11851277
EQ-Bench Creative Writing—946
WildBench—81.2%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than GPT-4.1 nano?

Gemini 1.5 Flash 8B is the stronger model overall, scoring 29.9 to 27.9 on the Noometry Index.

Is Gemini 1.5 Flash 8B or GPT-4.1 nano better for coding?

Gemini 1.5 Flash 8B scores higher on coding benchmarks: 35.5 versus 24.1 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and GPT-4.1 nano share?

18 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and GPT-4.1 nano has 38.

Related comparisons

Go deeper