Noometry

Model comparison

gpt-oss-20b vs Mistral 7B

gpt-oss-20b is the stronger model overall, scoring 32.5 to 23.0 on the Noometry Index.

Last verified . 20 shared benchmarks.

gpt-oss-20b OpenAI

32.5

Rank #255 Confirmed

Mistral 7B Mistral AI

23.0

Rank #351 Confirmed

Summary

  • They share 20 benchmarks with published results for both. gpt-oss-20b scores higher in 8 categories and Mistral 7B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where gpt-oss-20b leads 39.4 to 8.1.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 65.3% for gpt-oss-20b and 0.3% for Mistral 7B.
  • gpt-oss-20b is cheaper at $0.018 / $0.09 per million input/output tokens, against $0.25 / $0.25 for Mistral 7B.
  • gpt-oss-20b accepts more context: 131K tokens versus 8K.

Side by side

gpt-oss-20bMistral 7B
ProviderOpenAIMistral AI
Noometry Index32.523.0
Released2025-08-052023-09-27
WeightsOpenOpen
Context window131K8K
Max output16K8K
Input $ / M tokens$0.018$0.25
Output $ / M tokens$0.09$0.25
Results tracked3437

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding gpt-oss-20b leads

gpt-oss-20b: 37.6 (#192), Mistral 7B: 26.4 (#326)

Benchmarkgpt-oss-20bMistral 7B
LMArena Coding13061082
SciCode34.4%—
WeirdML40.9%—
BigCodeBench Instruct—19.5%
BigCodeBench Complete—27.3%
ALE-Bench566.05—
HumanEval+—36%
MBPP+—42.1%

Agentic & Tool Use Not comparable

gpt-oss-20b: 9.3 (#154), Mistral 7B: —

Benchmarkgpt-oss-20bMistral 7B
Terminal-Bench3.4%—

Reasoning gpt-oss-20b leads

gpt-oss-20b: 19.3 (#261), Mistral 7B: 13.1 (#336)

Benchmarkgpt-oss-20bMistral 7B
Chess Puzzles4%0%
LMArena Hard Prompts12741067
DTBench68%42.5%
Epoch Capabilities Index137.82112.21
Kagi LLM Benchmark53.2%—
CritPt1.4%—
LMCA14.5%—
Adversarial NLI—47.1%
BIG-Bench Hard—56.1%
HellaSwag—81%
PIQA—83%
WinoGrande—75.3%

Math gpt-oss-20b leads

gpt-oss-20b: 39.4 (#103), Mistral 7B: 8.1 (#325)

Benchmarkgpt-oss-20bMistral 7B
OTIS Mock AIME 2024-202565.3%0.3%
LMArena Math13171085
Omni-MATH56.5%—
MATH Level 5—3.7%
GSM8K—54.4%

Knowledge gpt-oss-20b leads

gpt-oss-20b: 34.6 (#195), Mistral 7B: 7.4 (#311)

Benchmarkgpt-oss-20bMistral 7B
GPQA Diamond60.8%15.2%
LMArena Expert12581036
MMLU-Pro74%—
GPQA (HELM)59.4%—
ARC (AI2) Challenge—78.6%
BoolQ—87.4%
MMLU—62.5%
OpenBookQA—79.8%
TriviaQA—75.2%

Multilingual gpt-oss-20b leads

gpt-oss-20b: 42.2 (#197), Mistral 7B: 25.8 (#283)

Benchmarkgpt-oss-20bMistral 7B
LMArena Non-English12681012
LMArena Chinese13141009
LMArena German1255987
LMArena Japanese1244878
LMArena Russian12781018
LMArena Spanish12671026
LMArena French—1037
LMArena Korean1236—

Instruction Following gpt-oss-20b leads

gpt-oss-20b: 61.8 (#240), Mistral 7B: 54.2 (#280)

Benchmarkgpt-oss-20bMistral 7B
LMArena Instruction Following12361060
IFEval73.2%—

Long Context gpt-oss-20b leads

gpt-oss-20b: 37.9 (#209), Mistral 7B: 32.2 (#271)

Benchmarkgpt-oss-20bMistral 7B
LMArena Longer Query12501060

Writing & Preference gpt-oss-20b leads

gpt-oss-20b: 35.5 (#265), Mistral 7B: 30.7 (#286)

Benchmarkgpt-oss-20bMistral 7B
LMArena Text12871090
LMArena Creative Writing12011068
LMArena Multi-Turn12681062
EQ-Bench Creative Writing666—
WildBench73.7%—

Frequently asked questions

Is gpt-oss-20b better than Mistral 7B?

gpt-oss-20b is the stronger model overall, scoring 32.5 to 23.0 on the Noometry Index.

Which is cheaper, gpt-oss-20b or Mistral 7B?

gpt-oss-20b is cheaper. It lists at $0.018 per million input tokens and $0.09 per million output tokens; Mistral 7B lists at $0.25 and $0.25.

Is gpt-oss-20b or Mistral 7B better for coding?

gpt-oss-20b scores higher on coding benchmarks: 37.6 versus 26.4 in the Noometry coding category.

Which has the bigger context window?

gpt-oss-20b does, with 131K tokens against 8K.

How many benchmarks do gpt-oss-20b and Mistral 7B share?

20 benchmarks have published results for both models. gpt-oss-20b has 34 scored results on Noometry and Mistral 7B has 37.

Related comparisons

Go deeper