Model comparison
Falcon-180B vs Olmo 2 0325 32b Instruct
Falcon-180B and Olmo 2 0325 32b Instruct score almost the same on the Noometry Index (32.2 vs 32.7), so choose on price, context window or the category you care about most.
Last verified . 6 shared benchmarks.
Technology Innovation Institute
Allen Institute for AI (Ai2)Summary
- They share 6 benchmarks with published results for both. Falcon-180B scores higher in 0 categories and Olmo 2 0325 32b Instruct in 4 categories; 4 gaps are clear of the uncertainty.
- The widest gap is in writing & preference, where Olmo 2 0325 32b Instruct leads 42.1 to 29.1.
Side by side
| Falcon-180B | Olmo 2 0325 32b Instruct | |
|---|---|---|
| Provider | Technology Innovation Institute | Allen Institute for AI (Ai2) |
| Noometry Index | 32.2 | 32.7 |
| Released | 2023-09-06 | — |
| Weights | Open | Open |
| Context window | — | — |
| Max output | — | — |
| Input $ / M tokens | — | — |
| Output $ / M tokens | — | — |
| Results tracked | 16 | 16 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding Not comparable
Falcon-180B: —, Olmo 2 0325 32b Instruct: 35.2 (#227)
| Benchmark | Falcon-180B | Olmo 2 0325 32b Instruct |
|---|---|---|
| LMArena Coding | — | 1210 |
Reasoning Olmo 2 0325 32b Instruct leads
Falcon-180B: 19.1 (#269), Olmo 2 0325 32b Instruct: 23.6 (#175)
| Benchmark | Falcon-180B | Olmo 2 0325 32b Instruct |
|---|---|---|
| LMArena Hard Prompts | 1007 | 1208 |
| Epoch Capabilities Index | 112.13 | — |
| HellaSwag | 89% | — |
| LAMBADA | 79.8% | — |
| PIQA | 84.9% | — |
| WinoGrande | 87.1% | — |
Math Not comparable
Falcon-180B: —, Olmo 2 0325 32b Instruct: 26.8 (#255)
| Benchmark | Falcon-180B | Olmo 2 0325 32b Instruct |
|---|---|---|
| Omni-MATH | — | 16.1% |
| LMArena Math | — | 1208 |
| GSM8K | 54.4% | — |
Knowledge Not comparable
Falcon-180B: —, Olmo 2 0325 32b Instruct: 19.5 (#279)
| Benchmark | Falcon-180B | Olmo 2 0325 32b Instruct |
|---|---|---|
| MMLU-Pro | — | 41.4% |
| GPQA (HELM) | — | 28.7% |
| ARC (AI2) Challenge | 67.8% | — |
| BoolQ | 89% | — |
| MMLU | 70.6% | — |
| OpenBookQA | 64.2% | — |
Multilingual Olmo 2 0325 32b Instruct leads
Falcon-180B: 25.2 (#286), Olmo 2 0325 32b Instruct: 34.8 (#248)
| Benchmark | Falcon-180B | Olmo 2 0325 32b Instruct |
|---|---|---|
| LMArena Non-English | 1000 | 1160 |
| LMArena Chinese | — | 1192 |
| LMArena Russian | — | 1187 |
Instruction Following Olmo 2 0325 32b Instruct leads
Falcon-180B: 53.4 (#286), Olmo 2 0325 32b Instruct: 61.5 (#244)
| Benchmark | Falcon-180B | Olmo 2 0325 32b Instruct |
|---|---|---|
| LMArena Instruction Following | 1047 | 1186 |
| IFEval | — | 78% |
Long Context Not comparable
Falcon-180B: —, Olmo 2 0325 32b Instruct: 36.2 (#234)
| Benchmark | Falcon-180B | Olmo 2 0325 32b Instruct |
|---|---|---|
| LMArena Longer Query | — | 1194 |
Writing & Preference Olmo 2 0325 32b Instruct leads
Falcon-180B: 29.1 (#295), Olmo 2 0325 32b Instruct: 42.1 (#236)
| Benchmark | Falcon-180B | Olmo 2 0325 32b Instruct |
|---|---|---|
| LMArena Text | 1054 | 1218 |
| LMArena Creative Writing | 1089 | 1199 |
| LMArena Multi-Turn | 1013 | 1221 |
| WildBench | — | 73.4% |
Frequently asked questions
Is Falcon-180B better than Olmo 2 0325 32b Instruct?
Falcon-180B and Olmo 2 0325 32b Instruct score almost the same on the Noometry Index (32.2 vs 32.7), so choose on price, context window or the category you care about most.
How many benchmarks do Falcon-180B and Olmo 2 0325 32b Instruct share?
6 benchmarks have published results for both models. Falcon-180B has 16 scored results on Noometry and Olmo 2 0325 32b Instruct has 16.