Model comparison

GPT-6.1 Sol vs Llama 3.1-8B

Llama 3.1-8B has enough public results to be ranked (#347); GPT-6.1 Sol does not yet, so treat this comparison as directional.

Last verified October 10, 2026. 0 shared benchmarks.

GPT-6.1 Sol OpenAI

42.1

Unranked Sparse

Llama 3.1-8B

Summary

The widest gap is in coding, where GPT-6.1 Sol leads 51.4 to 19.9.
Llama 3.1-8B is cheaper at $0.05 / $0.08 per million input/output tokens, against $2 / $10 for GPT-6.1 Sol.
GPT-6.1 Sol accepts more context: 1.05M tokens versus 128K.
Llama 3.1-8B has downloadable open weights; the other is API-only.

Side by side

GPT-6.1 Sol and Llama 3.1-8B specifications
	GPT-6.1 Sol	Llama 3.1-8B
Provider	OpenAI	Meta
Noometry Index	42.1	22.8
Released	2026-09-29	2024-07-23
Weights	Proprietary	Open
Context window	1.05M	128K
Max output	128K	4K
Input $ / M tokens	$2	$0.05
Output $ / M tokens	$10	$0.08
Results tracked	2	43

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-6.1 Sol leads

GPT-6.1 Sol: 51.4, Llama 3.1-8B: 19.9 (#335)

Coding benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
DeepSWE	75.2%	—
SciCode	—	13.2%
WeirdML	—	1.7%
BigCodeBench Instruct	—	32.8%
LMArena Coding	—	1195
BigCodeBench Complete	—	40.5%
HumanEval+	—	62.8%
MBPP+	—	55.6%

Agentic & Tool Use GPT-6.1 Sol leads

GPT-6.1 Sol: 34.3, Llama 3.1-8B: 22.3 (#123)

Agentic & Tool Use benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
Berkeley Function Calling Leaderboard	—	25.8%
BALROG	—	15.1%
GDP.pdf	32%	—

Reasoning Not comparable

GPT-6.1 Sol: —, Llama 3.1-8B: 14.5 (#316)

Reasoning benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
CritPt	—	0%
Chess Puzzles	—	0%
LMArena Hard Prompts	—	1175
DTBench	—	50.9%
LMCA	—	5.4%
Epoch Capabilities Index	—	116.48
PIQA	—	81.2%

Math Not comparable

GPT-6.1 Sol: —, Llama 3.1-8B: 9.5 (#311)

Math benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
OTIS Mock AIME 2024-2025	—	1.7%
Omni-MATH	—	13.7%
LMArena Math	—	1179
MATH Level 5	—	22.9%
GSM8K	—	82.4%

Knowledge Not comparable

GPT-6.1 Sol: —, Llama 3.1-8B: 8.0 (#302)

Knowledge benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
GPQA Diamond	—	27%
MMLU-Pro	—	40.6%
GPQA (HELM)	—	24.7%
LMArena Expert	—	1144
BoolQ	—	82.8%
MMLU	—	56.1%

Multilingual Not comparable

GPT-6.1 Sol: —, Llama 3.1-8B: 34.2 (#244)

Multilingual benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
LMArena Non-English	—	1148
LMArena Chinese	—	1151
LMArena French	—	1176
LMArena German	—	1144
LMArena Japanese	—	1061
LMArena Korean	—	1053
LMArena Russian	—	1158
LMArena Spanish	—	1169

Instruction Following Not comparable

GPT-6.1 Sol: —, Llama 3.1-8B: 59.1 (#253)

Instruction Following benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
IFEval	—	74.3%
LMArena Instruction Following	—	1159

Long Context Not comparable

GPT-6.1 Sol: —, Llama 3.1-8B: 35.8 (#233)

Long Context benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
LMArena Longer Query	—	1182

Writing & Preference Not comparable

GPT-6.1 Sol: —, Llama 3.1-8B: 29.9 (#285)

Writing & Preference benchmarks
Benchmark	GPT-6.1 Sol	Llama 3.1-8B
LMArena Text	—	1187
LMArena Creative Writing	—	1154
EQ-Bench Creative Writing	—	713
WildBench	—	68.7%
LMArena Multi-Turn	—	1172

Frequently asked questions

Is GPT-6.1 Sol better than Llama 3.1-8B?

Llama 3.1-8B has enough public results to be ranked (#347); GPT-6.1 Sol does not yet, so treat this comparison as directional.

Which is cheaper, GPT-6.1 Sol or Llama 3.1-8B?

Llama 3.1-8B is cheaper. It lists at $0.05 per million input tokens and $0.08 per million output tokens; GPT-6.1 Sol lists at $2 and $10.

Is GPT-6.1 Sol or Llama 3.1-8B better for coding?

GPT-6.1 Sol scores higher on coding benchmarks: 51.4 versus 19.9 in the Noometry coding category.

Which has the bigger context window?

GPT-6.1 Sol does, with 1.05M tokens against 128K.

How many benchmarks do GPT-6.1 Sol and Llama 3.1-8B share?

0 benchmarks have published results for both models. GPT-6.1 Sol has 2 scored results on Noometry and Llama 3.1-8B has 43.