Noometry

Model comparison

GPT-4.1 nano vs Mistral Small

Mistral Small is the stronger model overall, scoring 33.4 to 27.9 on the Noometry Index.

Last verified . 24 shared benchmarks.

GPT-4.1 nano OpenAI

27.9

Rank #327 Confirmed

Mistral Small Mistral AI

33.4

Rank #243 Confirmed

Summary

  • They share 24 benchmarks with published results for both. GPT-4.1 nano scores higher in 2 categories and Mistral Small in 8 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Mistral Small leads 40.4 to 23.7.
  • The biggest single-benchmark swing is MATH Level 5: 70% for GPT-4.1 nano and 46.8% for Mistral Small.
  • GPT-4.1 nano is cheaper at $0.10 / $0.40 per million input/output tokens, against $0.15 / $0.60 for Mistral Small.
  • GPT-4.1 nano accepts more context: 1.05M tokens versus 262K.
  • Mistral Small has downloadable open weights; the other is API-only.

Side by side

GPT-4.1 nanoMistral Small
ProviderOpenAIMistral AI
Noometry Index27.933.4
Released2025-04-142024-02-26
WeightsProprietaryOpen
Context window1.05M262K
Max output33K256K
Input $ / M tokens$0.10$0.15
Output $ / M tokens$0.40$0.60
Results tracked3839

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small leads

GPT-4.1 nano: 24.1 (#330), Mistral Small: 34.0 (#247)

BenchmarkGPT-4.1 nanoMistral Small
SciCode25.9%26.5%
LMArena Coding13061362
Aider Polyglot8.9%—
WeirdML19%—
BigCodeBench Instruct—36.1%
LiveBench Coding—36.2%
BigCodeBench Complete—46.6%
ALE-Bench—497.62

Agentic & Tool Use Mistral Small leads

GPT-4.1 nano: 26.5 (#104), Mistral Small: 28.1 (#93)

BenchmarkGPT-4.1 nanoMistral Small
Berkeley Function Calling Leaderboard33%37.1%

Reasoning Mistral Small leads

GPT-4.1 nano: 8.5 (#349), Mistral Small: 19.8 (#250)

BenchmarkGPT-4.1 nanoMistral Small
Kagi LLM Benchmark33.3%37.8%
CritPt0%0%
LMArena Hard Prompts12861335
DTBench52.5%70.9%
LMCA5.5%20.6%
ARC-AGI-20%—
ARC-AGI-10%—
LiveBench Reasoning—44.8%
LiveBench Data Analysis—53.7%
Epoch Capabilities Index129.62—
LiveBench—44%

Math GPT-4.1 nano leads

GPT-4.1 nano: 26.9 (#252), Mistral Small: 16.4 (#293)

BenchmarkGPT-4.1 nanoMistral Small
OTIS Mock AIME 2024-202528.9%5.8%
LMArena Math12741341
MATH Level 570%46.8%
Omni-MATH36.7%—
LiveBench Math—39.9%
FrontierMath (Feb 2025 set)1%—

Knowledge Mistral Small leads

GPT-4.1 nano: 21.8 (#273), Mistral Small: 31.0 (#222)

BenchmarkGPT-4.1 nanoMistral Small
GPQA Diamond48.9%47.5%
LMArena Expert12721291
SimpleQA Verified6%—
MMLU-Pro55%—
Vectara Hallucination Rate—5.1%
GPQA (HELM)50.7%—
MMLU—68.7%

Multimodal Mistral Small leads

GPT-4.1 nano: 29.2 (#113), Mistral Small: 33.5 (#96)

BenchmarkGPT-4.1 nanoMistral Small
LMArena Vision10631142

Multilingual Mistral Small leads

GPT-4.1 nano: 41.6 (#205), Mistral Small: 45.5 (#169)

BenchmarkGPT-4.1 nanoMistral Small
LMArena Non-English12601315
LMArena Chinese12701340
LMArena German12881340
LMArena Japanese11981275
LMArena Russian12611324
LMArena French—1337
LMArena Korean—1259
LMArena Spanish—1346

Instruction Following GPT-4.1 nano leads

GPT-4.1 nano: 67.8 (#193), Mistral Small: 66.4 (#209)

BenchmarkGPT-4.1 nanoMistral Small
LMArena Instruction Following12671310
LiveBench Instruction Following—63.7%
IFEval84.3%—

Long Context Mistral Small leads

GPT-4.1 nano: 23.7 (#296), Mistral Small: 40.4 (#156)

BenchmarkGPT-4.1 nanoMistral Small
LMArena Longer Query12831327
Fiction.LiveBench25%—

Writing & Preference Mistral Small leads

GPT-4.1 nano: 40.5 (#243), Mistral Small: 52.5 (#171)

BenchmarkGPT-4.1 nanoMistral Small
LMArena Text12851338
LMArena Creative Writing12601305
LMArena Multi-Turn12771344
EQ-Bench Creative Writing946—
WildBench81.2%—
LiveBench Language—30.5%

Frequently asked questions

Is GPT-4.1 nano better than Mistral Small?

Mistral Small is the stronger model overall, scoring 33.4 to 27.9 on the Noometry Index.

Which is cheaper, GPT-4.1 nano or Mistral Small?

GPT-4.1 nano is cheaper. It lists at $0.10 per million input tokens and $0.40 per million output tokens; Mistral Small lists at $0.15 and $0.60.

Is GPT-4.1 nano or Mistral Small better for coding?

Mistral Small scores higher on coding benchmarks: 34.0 versus 24.1 in the Noometry coding category.

Which has the bigger context window?

GPT-4.1 nano does, with 1.05M tokens against 262K.

How many benchmarks do GPT-4.1 nano and Mistral Small share?

24 benchmarks have published results for both models. GPT-4.1 nano has 38 scored results on Noometry and Mistral Small has 39.

Related comparisons

Go deeper