Model comparison

Falcon-40B vs GPT-4o

GPT-4o has enough public results to be ranked (#324); Falcon-40B does not yet, so treat this comparison as directional.

Last verified October 10, 2026. 2 shared benchmarks.

Falcon-40B

Technology Innovation Institute

—

Unranked

GPT-4o OpenAI

28.6

Rank #324 Confirmed

Summary

They share 2 benchmarks with published results for both.
Falcon-40B has downloadable open weights; the other is API-only.

Side by side

Falcon-40B and GPT-4o specifications
	Falcon-40B	GPT-4o
Provider	Technology Innovation Institute	OpenAI
Noometry Index	—	28.6
Released	2023-03-15	2024-05-13
Weights	Open	Proprietary
Context window	—	128K
Max output	—	16K
Input $ / M tokens	—	$2.50
Output $ / M tokens	—	$10
Results tracked	12	72

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Falcon-40B: —, GPT-4o: 24.8 (#328)

Coding benchmarks
Benchmark	Falcon-40B	GPT-4o
SWE-bench Verified	—	31%
SWE-bench Verified (bash only)	—	21.6%
Aider Polyglot	—	45.3%
GSO	—	0%
WeirdML	—	25.1%
BigCodeBench Instruct	—	51.1%
LiveBench Coding	—	51.4%
LMArena Coding	—	1297
BigCodeBench Complete	—	61.1%
CadEval	—	26%
HumanEval+	—	87.2%
MBPP+	—	72.2%

Agentic & Tool Use Not comparable

Falcon-40B: —, GPT-4o: 21.0 (#141)

Agentic & Tool Use benchmarks
Benchmark	Falcon-40B	GPT-4o
GDPval	—	9.9%
TheAgentCompany	—	8.6%
Cybench	—	12.5%
BALROG	—	32.3%
LMArena Search	—	1006
METR Time Horizons	—	40.8%

Reasoning Not comparable

Falcon-40B: —, GPT-4o: 9.4 (#343)

Reasoning benchmarks
Benchmark	Falcon-40B	GPT-4o
Epoch Capabilities Index	104.45	128.97
ARC-AGI-2	—	0%
SimpleBench	—	17.8%
ARC-AGI-1	—	4.5%
CritPt	—	0%
Chess Puzzles	—	13%
EnigmaEval	—	0.8%
LiveBench Reasoning	—	55.8%
LMArena Hard Prompts	—	1281
DTBench	—	64.5%
LiveBench Data Analysis	—	60.9%
LMCA	—	16.6%
BIG-Bench Hard	37.1%	—
ForecastBench	—	57.7
HellaSwag	85.3%	—
LAMBADA	77.3%	—
LiveBench	—	55.3%
PIQA	83%	—
WinoGrande	76.9%	—

Math Not comparable

Falcon-40B: —, GPT-4o: 10.6 (#312)

Math benchmarks
Benchmark	Falcon-40B	GPT-4o
FrontierMath (Tiers 1-3)	—	0.4%
OTIS Mock AIME 2024-2025	—	6.4%
Omni-MATH	—	29.3%
LiveBench Math	—	49.5%
LMArena Math	—	1285
MATH Level 5	—	53.3%
FrontierMath (Feb 2025 set)	—	0.3%
GSM8K	33.8%	—

Knowledge Not comparable

Falcon-40B: —, GPT-4o: 28.8 (#242)

Knowledge benchmarks
Benchmark	Falcon-40B	GPT-4o
MMLU	56.9%	88.1%
GPQA Diamond	—	49.2%
Humanity's Last Exam	—	2.7%
SimpleQA Verified	—	26%
MMLU-Pro	—	71.3%
Confabulations	—	15.3%
Vectara Hallucination Rate	—	9.6%
GPQA (HELM)	—	52%
LMArena Expert	—	1250
ARC (AI2) Challenge	61.9%	—
BoolQ	83.1%	—
OpenBookQA	56.6%	—
TriviaQA	79.9%	—

Multimodal Not comparable

Falcon-40B: —, GPT-4o: 34.5 (#91)

Multimodal benchmarks
Benchmark	Falcon-40B	GPT-4o
LMArena Vision	—	1137
Video-MME	—	71.9%
GeoBench	—	71%
VPCT	—	40%
ScienceQA	—	88.5%

Multilingual Not comparable

Falcon-40B: —, GPT-4o: 43.2 (#186)

Multilingual benchmarks
Benchmark	Falcon-40B	GPT-4o
LMArena Non-English	—	1283
LMArena Chinese	—	1277
LMArena French	—	1304
LMArena German	—	1282
LMArena Japanese	—	1257
LMArena Korean	—	1234
LMArena Russian	—	1286
LMArena Spanish	—	1292

Instruction Following Not comparable

Falcon-40B: —, GPT-4o: 66.6 (#207)

Instruction Following benchmarks
Benchmark	Falcon-40B	GPT-4o
LiveBench Instruction Following	—	68.6%
IFEval	—	81.7%
LMArena Instruction Following	—	1278

Long Context Not comparable

Falcon-40B: —, GPT-4o: 39.4 (#179)

Long Context benchmarks
Benchmark	Falcon-40B	GPT-4o
Fiction.LiveBench	—	66.7%
LMArena Longer Query	—	1289

Writing & Preference Not comparable

Falcon-40B: —, GPT-4o: 52.6 (#166)

Writing & Preference benchmarks
Benchmark	Falcon-40B	GPT-4o
LMArena Text	—	1300
LMArena Creative Writing	—	1292
Short-Story Creative Writing	—	81.8%
WildBench	—	82.8%
LMArena Multi-Turn	—	1302
LiveBench Language	—	47.6%

Frequently asked questions

Is Falcon-40B better than GPT-4o?