Model comparison

Grok 4.20 (Non-Reasoning) vs Mistral Medium 3.5

Grok 4.20 (Non-Reasoning) is the stronger model overall, scoring 48.6 to 40.2 on the Noometry Index.

Last verified October 10, 2026. 21 shared benchmarks.

Grok 4.20 (Non-Reasoning) xAI

48.6

Rank #54 Confirmed

Mistral Medium 3.5

Mistral AI

40.2

Rank #152 Confirmed

Summary

They share 21 benchmarks with published results for both. Grok 4.20 (Non-Reasoning) scores higher in 8 categories and Mistral Medium 3.5 in 1 category; 8 gaps are clear of the uncertainty.
The widest gap is in reasoning, where Grok 4.20 (Non-Reasoning) leads 52.3 to 17.3.
The biggest single-benchmark swing is NYT Connections (extended): 85.4% for Grok 4.20 (Non-Reasoning) and 12.9% for Mistral Medium 3.5.
Grok 4.20 (Non-Reasoning) is cheaper at $1.25 / $2.50 per million input/output tokens, against $1.50 / $7.50 for Mistral Medium 3.5.
Grok 4.20 (Non-Reasoning) accepts more context: 1M tokens versus 262K.
Mistral Medium 3.5 has downloadable open weights; the other is API-only.

Side by side

Grok 4.20 (Non-Reasoning) and Mistral Medium 3.5 specifications
	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
Provider	xAI	Mistral AI
Noometry Index	48.6	40.2
Released	2026-02-17	—
Weights	Proprietary	Open
Context window	1M	262K
Max output	30K	210K
Input $ / M tokens	$1.25	$1.50
Output $ / M tokens	$2.50	$7.50
Results tracked	46	22

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 4.20 (Non-Reasoning) leads

Grok 4.20 (Non-Reasoning): 42.1 (#112), Mistral Medium 3.5: 36.0 (#213)

Coding benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
LMArena WebDev	1375	1264
LMArena Coding	1459	1461
WeirdML	52.3%	—
ALE-Bench	1,150	—

Agentic & Tool Use Not comparable

Grok 4.20 (Non-Reasoning): 34.4 (#46), Mistral Medium 3.5: —

Agentic & Tool Use benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
Terminal-Bench	57.3%	—
τ²-bench Banking	18%	—
LMArena Search	1189	—
Vending-Bench 2	4,663	—

Reasoning Grok 4.20 (Non-Reasoning) leads

Grok 4.20 (Non-Reasoning): 52.3 (#32), Mistral Medium 3.5: 17.3 (#295)

Reasoning benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
Kagi LLM Benchmark	75%	41.4%
NYT Connections (extended)	85.4%	12.9%
LMArena Hard Prompts	1451	1436
Epoch Capabilities Index	151.98	141.35
ARC-AGI-2	65.1%	—
ARC-AGI-1	89.5%	—
Chess Puzzles	24%	—
Thematic Generalization	63.8%	—
DTBench	90.1%	—
LMCA	38.7%	—
ForecastBench	61.4	—

Math Grok 4.20 (Non-Reasoning) leads

Grok 4.20 (Non-Reasoning): 48.2 (#65), Mistral Medium 3.5: 39.1 (#113)

Math benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
LMArena Math	1455	1431
FrontierMath (Tiers 1-3)	44.9%	—
FrontierMath Tier 4	17.1%	—
OTIS Mock AIME 2024-2025	92.2%	—
ProofBench	14%	—

Knowledge Grok 4.20 (Non-Reasoning) leads

Grok 4.20 (Non-Reasoning): 52.8 (#60), Mistral Medium 3.5: 40.0 (#126)

Knowledge benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
LMArena Expert	1439	1432
GPQA Diamond	89.3%	—
SimpleQA Verified	30.2%	—

Multimodal Mistral Medium 3.5 leads

Grok 4.20 (Non-Reasoning): 33.3 (#98), Mistral Medium 3.5: 38.3 (#65)

Multimodal benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
LMArena Vision	1263	1223
Blueprint-Bench 2	0%	—
LMArena Document	1416	—

Multilingual Grok 4.20 (Non-Reasoning) leads

Grok 4.20 (Non-Reasoning): 54.5 (#40), Mistral Medium 3.5: 51.9 (#100)

Multilingual benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
LMArena Non-English	1441	1404
LMArena Chinese	1481	1442
LMArena French	1476	1448
LMArena German	1465	1451
LMArena Korean	1417	1385
LMArena Russian	1458	1395
LMArena Spanish	1443	1409
LMArena Japanese	1449	—

Instruction Following Too close to call

Grok 4.20 (Non-Reasoning): 74.8 (#83), Mistral Medium 3.5: 74.6 (#90)

Instruction Following benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
LMArena Instruction Following	1420	1415

Long Context Grok 4.20 (Non-Reasoning) leads

Grok 4.20 (Non-Reasoning): 45.5 (#34), Mistral Medium 3.5: 43.2 (#103)

Long Context benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
LMArena Longer Query	1437	1415
CL-bench	22.2%	—
CL-bench Life	11.9%	—

Writing & Preference Grok 4.20 (Non-Reasoning) leads

Grok 4.20 (Non-Reasoning): 65.7 (#44), Mistral Medium 3.5: 58.5 (#117)

Writing & Preference benchmarks
Benchmark	Grok 4.20 (Non-Reasoning)	Mistral Medium 3.5
LMArena Text	1451	1421
LMArena Creative Writing	1438	1374
LMArena Multi-Turn	1456	1423
EQ-Bench Creative Writing	1574	—
EQ-Bench 4	—	993

Frequently asked questions

Is Grok 4.20 (Non-Reasoning) better than Mistral Medium 3.5?

Grok 4.20 (Non-Reasoning) is the stronger model overall, scoring 48.6 to 40.2 on the Noometry Index.

Which is cheaper, Grok 4.20 (Non-Reasoning) or Mistral Medium 3.5?

Grok 4.20 (Non-Reasoning) is cheaper. It lists at $1.25 per million input tokens and $2.50 per million output tokens; Mistral Medium 3.5 lists at $1.50 and $7.50.

Is Grok 4.20 (Non-Reasoning) or Mistral Medium 3.5 better for coding?

Grok 4.20 (Non-Reasoning) scores higher on coding benchmarks: 42.1 versus 36.0 in the Noometry coding category.

Which has the bigger context window?

Grok 4.20 (Non-Reasoning) does, with 1M tokens against 262K.

How many benchmarks do Grok 4.20 (Non-Reasoning) and Mistral Medium 3.5 share?

21 benchmarks have published results for both models. Grok 4.20 (Non-Reasoning) has 46 scored results on Noometry and Mistral Medium 3.5 has 22.