Noometry

Model comparison

Gemini 1.5 Pro (May 2024) vs video-SALMONN 2+

Gemini 1.5 Pro (May 2024) has enough public results to be ranked (#261); video-SALMONN 2+ does not yet, so treat this comparison as directional.

Last verified . 1 shared benchmarks.

Gemini 1.5 Pro (May 2024) Google

32.1

Rank #261 Confirmed

video-SALMONN 2+ ByteDance Seed

41.4

Unranked Sparse

Summary

  • They share 1 benchmark with published results for both. Gemini 1.5 Pro (May 2024) scores higher in 0 categories and video-SALMONN 2+ in 1 category; one gap is clear of the uncertainty.
  • The widest gap is in multimodal, where video-SALMONN 2+ leads 42.5 to 36.8.

Side by side

Gemini 1.5 Pro (May 2024)video-SALMONN 2+
ProviderGoogleByteDance Seed
Noometry Index32.141.4
Released2024-02-152025-06-18
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked451

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemini 1.5 Pro (May 2024): 34.2 (#241), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
WeirdML22.2%—
BigCodeBench Instruct43.8%—
LMArena Coding1294—
BigCodeBench Complete57.5%—
CadEval34%—
HumanEval+79.3%—
MBPP+74.6%—

Agentic & Tool Use Not comparable

Gemini 1.5 Pro (May 2024): 17.9 (#145), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
TheAgentCompany3.4%—
Cybench7.5%—
BALROG21%—

Reasoning Not comparable

Gemini 1.5 Pro (May 2024): 12.3 (#338), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
ARC-AGI-20.8%—
SimpleBench27.1%—
LMArena Hard Prompts1296—
DTBench59%—
BIG-Bench Hard89.2%—
Epoch Capabilities Index131.73—
ForecastBench58.4—

Math Not comparable

Gemini 1.5 Pro (May 2024): 25.8 (#266), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
OTIS Mock AIME 2024-202523.1%—
Omni-MATH36.4%—
LMArena Math1315—
MATH Level 570.4%—

Knowledge Not comparable

Gemini 1.5 Pro (May 2024): 29.4 (#239), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
GPQA Diamond57.2%—
Humanity's Last Exam4.6%—
MMLU-Pro73.7%—
Confabulations13.5%—
GPQA (HELM)53.4%—
LMArena Expert1279—
MMLU86.9%—

Multimodal video-SALMONN 2+ leads

Gemini 1.5 Pro (May 2024): 36.8 (#77), video-SALMONN 2+: 42.5

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
Video-MME75%79.7%
LMArena Vision1161—

Multilingual Not comparable

Gemini 1.5 Pro (May 2024): 45.3 (#174), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
LMArena Non-English1312—
LMArena Chinese1331—
LMArena French1302—
LMArena German1286—
LMArena Japanese1292—
LMArena Korean1298—
LMArena Russian1320—
LMArena Spanish1311—

Instruction Following Not comparable

Gemini 1.5 Pro (May 2024): 68.6 (#185), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
IFEval83.7%—
LMArena Instruction Following1297—

Long Context Not comparable

Gemini 1.5 Pro (May 2024): 39.8 (#169), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
LMArena Longer Query1308—

Writing & Preference Not comparable

Gemini 1.5 Pro (May 2024): 52.4 (#172), video-SALMONN 2+: —

BenchmarkGemini 1.5 Pro (May 2024)video-SALMONN 2+
LMArena Text1319—
LMArena Creative Writing1333—
WildBench81.3%—
LMArena Multi-Turn1296—

Frequently asked questions

Is Gemini 1.5 Pro (May 2024) better than video-SALMONN 2+?

Gemini 1.5 Pro (May 2024) has enough public results to be ranked (#261); video-SALMONN 2+ does not yet, so treat this comparison as directional.

How many benchmarks do Gemini 1.5 Pro (May 2024) and video-SALMONN 2+ share?

1 benchmark has published results for both models. Gemini 1.5 Pro (May 2024) has 45 scored results on Noometry and video-SALMONN 2+ has 1.

Related comparisons

Go deeper