Model comparison
Claude 3.5 Sonnet vs Kimi K2.7 Code
Kimi K2.7 Code is the stronger model overall, scoring 43.0 to 34.4 on the Noometry Index.
Last verified . 5 shared benchmarks.
Summary
- They share 5 benchmarks with published results for both. Claude 3.5 Sonnet scores higher in 1 category and Kimi K2.7 Code in 4 categories; 5 gaps are clear of the uncertainty.
- The widest gap is in math, where Kimi K2.7 Code leads 52.6 to 18.0.
- The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 8.5% for Claude 3.5 Sonnet and 95.6% for Kimi K2.7 Code.
- Kimi K2.7 Code has downloadable open weights; the other is API-only.
Side by side
| Claude 3.5 Sonnet | Kimi K2.7 Code | |
|---|---|---|
| Provider | Anthropic | Moonshot AI |
| Noometry Index | 34.4 | 43.0 |
| Released | 2024-06-20 | 2026-06-12 |
| Weights | Proprietary | Open |
| Context window | — | 262K |
| Max output | — | 262K |
| Input $ / M tokens | — | $0.95 |
| Output $ / M tokens | — | $4 |
| Results tracked | 60 | 19 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding Kimi K2.7 Code leads
Claude 3.5 Sonnet: 39.5 (#145), Kimi K2.7 Code: 42.4 (#92)
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| WeirdML | 40% | 54.1% |
| DeepSWE | — | 30.5% |
| FrontierCode | — | 30.1% |
| Aider Polyglot | 51.6% | — |
| LMArena WebDev | — | 1473 |
| SciCode | — | 47.5% |
| GSO | 4.6% | — |
| BigCodeBench Instruct | 46.8% | — |
| LiveBench Coding | 67.1% | — |
| LMArena Coding | 1343 | — |
| BigCodeBench Complete | 58.6% | — |
| CadEval | 48% | — |
| ALE-Bench | — | 886.23 |
| HumanEval+ | 81.7% | — |
| MBPP+ | 74.3% | — |
Agentic & Tool Use Claude 3.5 Sonnet leads
Claude 3.5 Sonnet: 32.1 (#61), Kimi K2.7 Code: 23.9 (#115)
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| APEX-Agents | — | 37.6% |
| TheAgentCompany | 24% | — |
| Cybench | 17.5% | — |
| BALROG | 32.6% | — |
| GBAEval | — | 0.9% |
| METR Time Horizons | 45.2% | — |
| Vending-Bench 2 | — | 5,083 |
Reasoning Kimi K2.7 Code leads
Claude 3.5 Sonnet: 22.5 (#176), Kimi K2.7 Code: 38.2 (#56)
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| SimpleBench | 41.4% | 57.9% |
| Epoch Capabilities Index | 133.55 | 150.01 |
| CritPt | — | 10% |
| Chess Puzzles | — | 21% |
| EnigmaEval | 0.9% | — |
| LiveBench Reasoning | 56.7% | — |
| LMArena Hard Prompts | 1305 | — |
| DTBench | 67.8% | — |
| LiveBench Data Analysis | 55% | — |
| Surface Evolver Bench | — | 48.8% |
| ForecastBench | 60.7 | — |
| LiveBench | 59% | — |
Math Kimi K2.7 Code leads
Claude 3.5 Sonnet: 18.0 (#282), Kimi K2.7 Code: 52.6 (#39)
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| OTIS Mock AIME 2024-2025 | 8.5% | 95.6% |
| FrontierMath (Tiers 1-3) | — | 54% |
| FrontierMath Tier 4 | — | 12.2% |
| Omni-MATH | 27.6% | — |
| LiveBench Math | 52.3% | — |
| LMArena Math | 1306 | — |
| MATH Level 5 | 56.9% | — |
| FrontierMath (Feb 2025 set) | 2.1% | — |
| FrontierMath Tier 4 (v1) | 0% | — |
Knowledge Kimi K2.7 Code leads
Claude 3.5 Sonnet: 28.5 (#236), Kimi K2.7 Code: 53.6 (#51)
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| GPQA Diamond | 55.3% | 87.9% |
| Humanity's Last Exam | 4.1% | — |
| SimpleQA Verified | — | 36.5% |
| MMLU-Pro | 77.7% | — |
| Confabulations | 19.9% | — |
| GPQA (HELM) | 56.5% | — |
| LMArena Expert | 1264 | — |
| MMLU | 87.3% | — |
Multimodal Not comparable
Claude 3.5 Sonnet: 26.0 (#115), Kimi K2.7 Code: —
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| LMArena Vision | 1125 | — |
| Video-MME | 60% | — |
| GeoBench | 62% | — |
| VPCT | 33% | — |
Multilingual Not comparable
Claude 3.5 Sonnet: 43.4 (#180), Kimi K2.7 Code: —
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| LMArena Non-English | 1283 | — |
| LMArena Chinese | 1271 | — |
| LMArena French | 1304 | — |
| LMArena German | 1297 | — |
| LMArena Japanese | 1234 | — |
| LMArena Korean | 1200 | — |
| LMArena Russian | 1305 | — |
| LMArena Spanish | 1289 | — |
Instruction Following Not comparable
Claude 3.5 Sonnet: 69.0 (#177), Kimi K2.7 Code: —
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| LiveBench Instruction Following | 69.3% | — |
| IFEval | 85.5% | — |
| LMArena Instruction Following | 1297 | — |
Long Context Not comparable
Claude 3.5 Sonnet: 39.9 (#162), Kimi K2.7 Code: —
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| LMArena Longer Query | 1311 | — |
Writing & Preference Not comparable
Claude 3.5 Sonnet: 53.0 (#159), Kimi K2.7 Code: —
| Benchmark | Claude 3.5 Sonnet | Kimi K2.7 Code |
|---|---|---|
| LMArena Text | 1298 | — |
| LMArena Creative Writing | 1292 | — |
| Short-Story Creative Writing | 80.3% | — |
| EQ-Bench Creative Writing | 1451 | — |
| WildBench | 79.2% | — |
| LMArena Multi-Turn | 1326 | — |
| LiveBench Language | 53.8% | — |
Frequently asked questions
Is Claude 3.5 Sonnet better than Kimi K2.7 Code?
Kimi K2.7 Code is the stronger model overall, scoring 43.0 to 34.4 on the Noometry Index.
Is Claude 3.5 Sonnet or Kimi K2.7 Code better for coding?
Kimi K2.7 Code scores higher on coding benchmarks: 42.4 versus 39.5 in the Noometry coding category.
How many benchmarks do Claude 3.5 Sonnet and Kimi K2.7 Code share?
5 benchmarks have published results for both models. Claude 3.5 Sonnet has 60 scored results on Noometry and Kimi K2.7 Code has 19.