Noometry

Model comparison

gpt-oss-20b vs Llama 3.1-405B

gpt-oss-20b is the stronger model overall, scoring 32.1 to 30.4 on the Noometry Index.

Last verified . 28 shared benchmarks.

gpt-oss-20b OpenAI

32.1

Rank #249 Confirmed

Llama 3.1-405B Meta

30.4

Rank #279 Confirmed

Summary

  • They share 28 benchmarks with published results for both. gpt-oss-20b scores higher in 5 categories and Llama 3.1-405B in 4 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where gpt-oss-20b leads 37.6 to 17.3.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 65.3% for gpt-oss-20b and 9.7% for Llama 3.1-405B.

Side by side

gpt-oss-20bLlama 3.1-405B
ProviderOpenAIMeta
Noometry Index32.130.4
Released2025-08-052024-07-23
WeightsOpenOpen
Context window131K—
Max output16K—
Input $ / M tokens$0.018—
Output $ / M tokens$0.09—
Results tracked3442

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding gpt-oss-20b leads

gpt-oss-20b: 37.1 (#185), Llama 3.1-405B: 32.7 (#257)

Benchmarkgpt-oss-20bLlama 3.1-405B
WeirdML40.9%21.4%
LMArena Coding13071291
SciCode34.4%—
ALE-Bench566.05—

Agentic & Tool Use Llama 3.1-405B leads

gpt-oss-20b: 9.2 (#146), Llama 3.1-405B: 20.9 (#131)

Benchmarkgpt-oss-20bLlama 3.1-405B
Terminal-Bench3.4%—
TheAgentCompany—7.4%
Cybench—7.5%

Reasoning gpt-oss-20b leads

gpt-oss-20b: 18.8 (#251), Llama 3.1-405B: 16.4 (#293)

Benchmarkgpt-oss-20bLlama 3.1-405B
Kagi LLM Benchmark53.2%45%
LMArena Hard Prompts12741270
DTBench68%61.4%
Epoch Capabilities Index137.81128.76
SimpleBench—23%
CritPt1.4%—
Chess Puzzles4%—
LMCA14.5%—
BIG-Bench Hard—82.9%
ForecastBench—59.9
HellaSwag—89.2%
PIQA—85.9%
WinoGrande—89.2%

Math gpt-oss-20b leads

gpt-oss-20b: 37.6 (#93), Llama 3.1-405B: 17.3 (#283)

Benchmarkgpt-oss-20bLlama 3.1-405B
OTIS Mock AIME 2024-202565.3%9.7%
Omni-MATH56.5%24.9%
LMArena Math13171281
MATH Level 5—49.8%

Knowledge gpt-oss-20b leads

gpt-oss-20b: 34.6 (#186), Llama 3.1-405B: 30.4 (#221)

Benchmarkgpt-oss-20bLlama 3.1-405B
GPQA Diamond60.8%50.9%
MMLU-Pro74%72.3%
GPQA (HELM)59.4%52.2%
LMArena Expert12581243
Confabulations—17.6%
ARC (AI2) Challenge—95.3%
MMLU—84.5%
TriviaQA—82.7%

Multilingual gpt-oss-20b leads

gpt-oss-20b: 42.4 (#192), Llama 3.1-405B: 40.9 (#209)

Benchmarkgpt-oss-20bLlama 3.1-405B
LMArena Non-English12691248
LMArena Chinese13111242
LMArena German12491252
LMArena Japanese12451208
LMArena Korean12391184
LMArena Russian12791265
LMArena Spanish12671259
LMArena French—1278

Instruction Following Llama 3.1-405B leads

gpt-oss-20b: 62.0 (#235), Llama 3.1-405B: 66.1 (#209)

Benchmarkgpt-oss-20bLlama 3.1-405B
IFEval73.2%81.1%
LMArena Instruction Following12361259

Long Context Too close to call

gpt-oss-20b: 37.9 (#203), Llama 3.1-405B: 38.4 (#192)

Benchmarkgpt-oss-20bLlama 3.1-405B
LMArena Longer Query12511266

Writing & Preference Llama 3.1-405B leads

gpt-oss-20b: 35.8 (#260), Llama 3.1-405B: 39.1 (#246)

Benchmarkgpt-oss-20bLlama 3.1-405B
LMArena Text12881284
LMArena Creative Writing12011262
EQ-Bench Creative Writing666870
WildBench73.7%78.3%
LMArena Multi-Turn12681297

Frequently asked questions

Is gpt-oss-20b better than Llama 3.1-405B?

gpt-oss-20b is the stronger model overall, scoring 32.1 to 30.4 on the Noometry Index.

Is gpt-oss-20b or Llama 3.1-405B better for coding?

gpt-oss-20b scores higher on coding benchmarks: 37.1 versus 32.7 in the Noometry coding category.

How many benchmarks do gpt-oss-20b and Llama 3.1-405B share?

28 benchmarks have published results for both models. gpt-oss-20b has 34 scored results on Noometry and Llama 3.1-405B has 42.

Related comparisons

Go deeper