Model | Size | Version | Released | Code RankedAGI | Code LiveBench | Aider Polyglot | SWEBench Verified | Codeforces ELO | WebDev Arena | LiveCodeBench v5 | Code LMArena | GPQA Diamond | Reason LiveBench | ELO LMArena | AIME 2025 I & II | Math LiveBench | MATH | MMLU Pro | MMLU | MMMU | Halluc. Hughes | Aidan Bench | AIME 2024 | IF LiveBench | Data LiveBench | Avg LiveBench | IF Evaluation | Input Cost /M | Output Cost /M | Cutoff Knowledge | Context Length | Org. | License | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | Latest | Apr 16, 2025 | 78.2% | 76.7% | 79.6% | 89.2% | $10 | $40 | Jun 24 | 200K | OpenAI | Proprietary | ||||||||||||||||||||||
2 | Latest | Apr 16, 2025 | 76.0% | 80.0% | 72.0% | 91.7% | $1.1 | $4.4 | Jun 24 | 200K | OpenAI | Proprietary | ||||||||||||||||||||||
3 | Preview | May 6, 2025 | 74.9% | 72.9% | 76.9% | 1419.95 | Proprietary | |||||||||||||||||||||||||||
4 | Latest | Apr 16, 2025 | 73.5% | 77.9% | 69.1% | 2706 | 1187.08 | 1374 | 87.7% | 1411 | 88.9% | 91.6% | $10 | $40 | Jun 24 | 200K | OpenAI | Proprietary | ||||||||||||||||
5 | Old | Mar 25, 2025 | 70.2% | 58.1% | 72.9% | 63.8% | 1272.86 | 70.4% | 1352 | 84.0% | 89.8% | 1443 | 86.7% | 90.2% | 89.8% | 81.7% | 1.1% | 92.0% | 80.6% | 79.9% | 82.3% | $1.25 | $10 | Jan 25 | 1M | Proprietary | ||||||||
6 | Preview | Feb 25, 2025 | 68.2% | 68.2% | Sep 24 | Alibaba | Open Source | |||||||||||||||||||||||||||
7 | Preview | Feb 17, 2025 | 67.4% | 84.6% | 93.3% | 78.0% | 93.3% | Nov 24 | 1M | xAI | Proprietary | |||||||||||||||||||||||
8 | Latest | Jan 31, 2025 | 67.2% | 65.5% | 60.4% | 49.3% | 2130 | 1147.27 | 80.5% | 1332 | 79.7% | 89.6% | 1332 | 86.5% | 76.5% | 97.9% | 86.9% | 0.8% | 87.3% | 84.4% | 70.6% | 75.8% | $1 | $4.4 | Oct 23 | 200K | OpenAI | Proprietary | ||||||
9 | 235B | Latest | Apr 29, 2025 | 65.6% | 65.3% | 61.8% | 2056 | 70.7% | 81.5% | 85.7% | 128K | Alibaba | Open Source | |||||||||||||||||||||
10 | Latest | Apr 16, 2025 | 65.0% | 61.8% | 68.1% | 2719 | 1093.02 | 1334 | 81.4% | 1352 | 81.6% | $1.1 | $4.4 | Jun 24 | 200K | OpenAI | Proprietary | |||||||||||||||||
11 | Hanyuan T1 | Latest | Mar 21, 2025 | 64.9% | 64.9% | 69.3% | 87.2% | 78.2% | Tencent | Open Source | ||||||||||||||||||||||||
12 | 30B | Latest | Apr 29, 2025 | 62.6% | 1974 | 62.6% | 65.8% | 70.9% | 80.4% | 128K | Alibaba | Open Source | ||||||||||||||||||||||
13 | Latest | Dec 17, 2024 | 61.9% | 61.7% | 41.0% | 1891 | 1050.14 | 78.3% | 1331 | 76.0% | 91.6% | 1359 | 79.0% | 67.3% | 2.4% | 3204 | 83.3% | 81.5% | 65.5% | 73.5% | $15 | $60 | Oct 23 | 200K | OpenAI | Proprietary | ||||||||
14 | Latest | Mar 26, 2025 | 61.4% | 77.5% | 45.3% | 1385 | 1408 | 69.1% | $5 | $15 | Oct 23 | 128K | OpenAI | Proprietary | ||||||||||||||||||||
15 | Latest | Feb 24, 2025 | 60.4% | 44.7% | 64.9% | 1333 | 78.2% | 84.6% | 1306 | 49.5% | 77.5% | 75.0% | 80.0% | 78.6% | 72.8% | 74.3% | 93.2% | $3 | $15 | Oct 24 | 200K | Anthropic | Proprietary | |||||||||||
16 | 32B | Latest | Mar 18, 2025 | 59.5% | 59.5% | 66.1% | 80.0% | 74.0% | 83.0% | 90.0% | LG | Proprietary | ||||||||||||||||||||||
17 | Latest | Jan 31, 2025 | 59.1% | 58.4% | 53.8% | 42.9% | 2036 | 1100.18 | 79.2% | 1309 | 76.8% | 86.3% | 1305 | 76.5% | 97.3% | 85.9% | 79.6% | 83.2% | 66.6% | 70.0% | $1 | $4.4 | Oct 23 | 200K | OpenAI | Proprietary | ||||||||
18 | Latest | Dec 17, 2024 | 58.7% | 57.1% | 61.7% | 1914 | 0.832 | 79.0% | 77.5% | 79.3% | 2.4% | 86.0% | 81.5% | 65.5% | 72.2% | $150 | $600 | Oct 23 | 200K | OpenAI | Proprietary | |||||||||||||
19 | Preview | Feb 27, 2025 | 58.6% | 76.1% | 45.0% | 38.0% | 1362 | 71.4% | 71.1% | 1411 | 69.3% | 74.4% | 1.2% | 36.7% | 72.3% | 64.3% | 69.0% | $75 | $150 | 128K | OpenAI | Proprietary | ||||||||||||
20 | Latest | Feb 24, 2025 | 57.7% | 32.4% | 60.4% | 70.3% | 1355.76 | 1326 | 68.0% | 66.0% | 1295 | 63.3% | 71.8% | 23.3% | 76.5% | 63.4% | 65.6% | 90.8% | $3 | $15 | Oct 24 | 200K | Anthropic | Proprietary | ||||||||||
21 | Preview | Apr 17, 2025 | 57.2% | 58.4% | 51.1% | 1145.13 | 63.5% | 78.3% | 78.0% | 88.4% | 76.7% | 88.0% | $0.15 | $0.6 | Jan 25 | Proprietary | ||||||||||||||||||
22 | 671B | Latest | Jan 20, 2025 | 56.8% | 48.5% | 56.9% | 49.2% | 2029 | 1205.21 | 64.3% | 1342 | 71.5% | 83.2% | 1359 | 70.0% | 79.5% | 84.0% | 90.8% | 14.3% | 80.5% | 69.8% | 71.4% | 83.3% | $0.55 | $2.19 | Jul 24 | 128K | DeepSeek | Open Source | |||||
23 | Latest | Jan 31, 2025 | 56.0% | 50.8% | 40.8% | 1831 | 74.5% | 70.6% | 69.8% | 48.0% | 61.4% | 95.8% | 84.9% | 60.0% | 80.1% | 63.0% | 62.2% | $1 | $4.4 | Oct 23 | 200K | OpenAI | Proprietary | |||||||||||
24 | 7.8B | Latest | Mar 18, 2025 | 55.2% | 55.2% | 62.6% | 76.7% | 65.7% | 75.0% | 83.3% | LG | Proprietary | ||||||||||||||||||||||
25 | 4B | Latest | Apr 29, 2025 | 54.2% | 54.2% | 1671 | 55.9% | 65.6% | 73.8% | 32K | Alibaba | Open Source | ||||||||||||||||||||||
26 | 685B | Latest | Mar 24, 2025 | 54.2% | 40.5% | 55.1% | 1209.23 | 49.2% | 1343 | 68.4% | 65.8% | 1370 | 73.5% | 81.2% | 8.0% | 59.4% | 81.5% | 60.4% | 66.9% | $0.27 | $1.1 | Jul 24 | 128K | DeepSeek | Open Source | |||||||||
27 | Old | Dec 19, 2024 | 53.1% | 1023.36 | 1283 | 64.6% | 1369 | 69.0% | 2755 | 79.3% | 68.1% | 61.8% | Aug 24 | 32K | Proprietary | |||||||||||||||||||
28 | Old | Nov 14, 2024 | 52.4% | 54.7% | 1343 | 54.9% | 77.1% | 57.5% | 56.4% | Aug 24 | 32K | Proprietary | ||||||||||||||||||||||
29 | 32B | Latest | Apr 29, 2025 | 51.4% | 40.0% | 1977 | 65.7% | 85.7% | 81.4% | 128K | Alibaba | Open Source | ||||||||||||||||||||||
30 | Old | Dec 6, 2024 | 50.8% | 38.2% | 1022.51 | 57.0% | 1379 | 70.0% | 77.3% | 63.2% | 63.7% | Aug 24 | 2M | Proprietary | ||||||||||||||||||||
31 | Old | Nov 21, 2024 | 50.4% | 45.3% | 1365 | 62.7% | 80.2% | 57.0% | 59.0% | Aug 24 | 32K | Proprietary | ||||||||||||||||||||||
32 | Latest | Apr 14, 2025 | 50.3% | 43.0% | 53.0% | 55.0% | 1261.35 | 66.3% | 90.2% | 75.0% | 48.1% | 87.0% | $2 | $8 | Jun 24 | 1M | OpenAI | Proprietary | ||||||||||||||||
33 | Preview | Feb 5, 2025 | 49.4% | 35.3% | 1109.54 | 1298 | 64.7% | 60.1% | 1379 | 27.5% | 71.0% | 91.8% | 79.1% | 72.7% | 0.8% | 83.4% | 68.0% | 65.1% | Aug 24 | 2M | Proprietary | |||||||||||||
34 | 32B | Latest | Mar 5, 2025 | 49.2% | 43.0% | 20.9% | 63.4% | 1291 | 83.5% | 1312 | 66.0% | 64.9% | 79.5% | 81.8% | 65.0% | 72.0% | 83.9% | $0.4 | $0.55 | Sep 24 | 131K | Alibaba | Open Source | |||||||||||
35 | 2T | Preview | Apr 5, 2025 | 49.0% | 49.0% | 73.7% | 82.2% | 85.8% | 76.1% | Aug 24 | Meta | Open Source | ||||||||||||||||||||||
36 | 70B | Latest | Jan 20, 2025 | 48.8% | 46.6% | 1633 | 65.2% | 67.6% | 55.0% | 70.0% | 41.5% | 55.9% | 49.7% | Dec 23 | DeepSeek | Open Source | ||||||||||||||||||
37 | Old | Sep 12, 2024 | 48.7% | 41.0% | 1258 | 55.6% | 1312 | 73.3% | 68.0% | 1333 | 62.9% | 85.0% | 91.0% | 3.3% | 1892 | 56.7% | 74.6% | 64.0% | 69.1% | $15 | $60 | Oct 23 | 128K | OpenAI | Proprietary | |||||||||
38 | 671B | Old | Dec 26, 2024 | 48.6% | 48.4% | 42.0% | 1134 | 963.91 | 40.5% | 1279 | 59.1% | 56.8% | 1315 | 25.0% | 60.5% | 75.9% | 88.5% | 3.9% | 75.3% | 60.9% | 60.5% | 86.1% | $0.07 | $1.1 | Jul 24 | 128K | DeepSeek | Open Source | ||||||
39 | Latest | Oct 22, 2024 | 48.3% | 32.3% | 51.6% | 49.0% | 717 | 1245.4 | 39.8% | 1313 | 65.0% | 58.7% | 1282 | 3.0% | 51.3% | 78.3% | 78.0% | 88.0% | 70.4% | 4.6% | 2691 | 16.0% | 69.3% | 52.8% | 60.7% | 89.3% | $3 | $15 | Apr 24 | 200K | Anthropic | Proprietary | ||
40 | 123B | Old | Jul 24, 2024 | 47.1% | 1244 | 48.0% | 42.0% | 1250 | 43.7% | 72.0% | 65.9% | 85.0% | 4.1% | 67.9% | 44.0% | 50.4% | $2 | $6 | Jul 24 | 128K | Mistral | Proprietary | ||||||||||||
41 | 2.4B | Latest | Mar 18, 2025 | 46.6% | 46.6% | 54.3% | 73.3% | 56.4% | 65.8% | 76.7% | LG | Proprietary | ||||||||||||||||||||||
42 | Latest | Feb 17, 2025 | 46.6% | 39.7% | 34.7% | 70.0% | 79.0% | 83.0% | 69.4% | 93.0% | $0.3 | $0.5 | Nov 24 | 130K | xAI | Proprietary | ||||||||||||||||||
43 | Old | Sep 24, 2024 | 45.8% | 893.34 | 42.1% | 1253 | 59.1% | 46.0% | 1301 | 57.4% | 86.5% | 75.8% | 86.0% | 6.6% | 1515 | 70.8% | 52.3% | 55.5% | 81.9% | $1.25 | $2.5 | Sep 24 | 2M | Proprietary | ||||||||||
44 | Latest | Feb 17, 2025 | 45.3% | 37.3% | 53.3% | 1148.53 | 1335 | 75.4% | 1412 | 79.9% | 73.2% | 52.2% | $3 | $15 | Nov 24 | 130K | xAI | Proprietary | ||||||||||||||||
45 | Latest | Aug 13, 2024 | 45.1% | 1253 | 56.0% | 36.3% | 1290 | 45.8% | 76.1% | 75.5% | 87.5% | 4.6% | 826 | 69.6% | 50.3% | 54.4% | $5 | $15 | Aug 24 | 128K | xAI | Proprietary | ||||||||||||
46 | Latest | Sep 12, 2024 | 44.9% | 32.9% | 35.0% | 1650 | 1053.69 | 68.4% | 1292 | 60.0% | 77.3% | 1308 | 59.2% | 90.0% | 80.3% | 85.0% | 1.4% | 1577 | 63.6% | 65.4% | 54.1% | 53.4% | $3 | $12 | Oct 23 | 128K | OpenAI | Proprietary | ||||||
47 | Preview | Jan 21, 2025 | 44.6% | 35.7% | 1034.54 | 1290 | 74.2% | 78.2% | 1380 | 53.5% | 1.8% | 82.5% | 69.4% | 66.9% | Aug 24 | 1M | Proprietary | |||||||||||||||||
48 | Old | Jun 20, 2024 | 44.0% | 33.4% | 36.0% | 1282 | 59.4% | 58.7% | 1268 | 53.3% | 71.1% | 76.1% | 88.7% | 68.3% | 1439 | 68.0% | 56.7% | 59.7% | $3 | $15 | Apr 24 | 200K | Anthropic | Proprietary | ||||||||||
49 | Old | Jan 29, 2025 | 43.8% | 27.1% | 1331 | 57.9% | 1377 | 48.0% | 65.1% | 66.0% | 57.8% | $2.5 | $10 | Jun 24 | 128K | OpenAI | Proprietary | |||||||||||||||||
50 | 123B | Latest | Nov 18, 2024 | 42.7% | 37.3% | 1228 | 47.0% | 43.4% | 1251 | 32.6% | 72.0% | 84.0% | 4.1% | 953 | 63.7% | 46.6% | 50.0% | $2 | $6 | Jul 24 | 128K | Mistral | Proprietary | |||||||||||
51 | 32B | Latest | Jan 20, 2025 | 42.6% | 52.3% | 1691 | 62.1% | 52.3% | 72.6% | 40.9% | 45.4% | 42.9% | Sep 24 | DeepSeek | Open Source | |||||||||||||||||||
52 | Old | Dec 11, 2024 | 42.5% | 22.2% | 51.8% | 983.97 | 41.6% | 1290 | 62.1% | 59.1% | 1356 | 59.4% | 89.7% | 76.4% | 87.0% | 1.3% | 1895 | 81.9% | 59.7% | 58.0% | Aug 24 | 1M | Proprietary | |||||||||||
53 | Old | Aug 6, 2024 | 41.6% | 31.8% | 1251 | 51.0% | 54.7% | 1265 | 48.2% | 78.0% | 74.7% | 89.0% | 1333 | 68.6% | 52.9% | 55.1% | $2.5 | $10 | Oct 23 | 128K | OpenAI | Proprietary | ||||||||||||
54 | 236B | Latest | Jul 22, 2024 | 41.5% | 1226 | 42.0% | 45.3% | 1213 | 58.0% | 63.6% | 80.0% | 67.2% | 38.3% | 47.2% | $0.14 | $0.28 | Nov 23 | 128K | DeepSeek | Proprietary | ||||||||||||||
55 | Old | Aug 27, 2024 | 40.9% | 1244 | 49.3% | 1260 | 56.1% | 69.0% | 9.1% | 69.3% | 50.8% | 55.4% | Nov 23 | 2M | Proprietary | |||||||||||||||||||
56 | Old | Aug 27, 2024 | 40.6% | 1208 | 47.3% | 1227 | 59.1% | 6.6% | 73.0% | 47.9% | 50.1% | Nov 23 | 1M | Proprietary | ||||||||||||||||||||
57 | 32B | Preview | Nov 28, 2024 | 40.2% | 44.0% | 65.2% | 57.7% | 33.5% | 50.8% | 75.0% | 12.9% | 44.0% | 35.6% | 3.2% | 28.5% | Sep 24 | 32K | Alibaba | Open Source | |||||||||||||||
58 | Latest | Feb 5, 2025 | 40.1% | 26.2% | 1041.64 | 1271 | 60.1% | 55.3% | 1357 | 27.5% | 65.6% | 90.9% | 77.6% | 71.7% | 0.7% | 85.8% | 67.5% | 61.5% | $0.1 | $0.4 | Aug 24 | 1M | Proprietary | |||||||||||
59 | Latest | Jan 28, 2025 | 40.0% | 33.8% | 21.8% | 981.52 | 1291 | 60.1% | 51.4% | 1332 | 68.5% | 76.1% | 75.3% | 67.9% | 62.3% | $0.0016 | $0.0064 | Sep 24 | 32K | Alibaba | Proprietary | |||||||||||||
60 | Latest | Oct 22, 2024 | 40.0% | 28.0% | 40.6% | 1134.43 | 1263 | 41.6% | 29.3% | 1239 | 35.5% | 73.0% | 65.0% | 81.0% | 4.9% | 904 | 61.9% | 42.4% | 47.8% | $1 | $5 | Jul 24 | 200K | Anthropic | Proprietary | |||||||||
61 | Old | Sep 24, 2024 | 39.3% | 36.1% | 1210 | 51.0% | 50.0% | 1269 | 47.2% | 77.9% | 67.3% | 81.0% | 3.4% | 1047 | 78.8% | 44.2% | 50.3% | $0.075 | $0.3 | Sep 24 | 1M | Proprietary | ||||||||||||
62 | 7B | Latest | Sep 19, 2024 | 38.4% | 26.2% | 38.2% | 2.8% | 52.1% | 32.8% | 33.6% | Sep 24 | 128K | Alibaba | Open Source | ||||||||||||||||||||
63 | 72B | Latest | Sep 19, 2024 | 38.0% | 23.8% | 31.1% | 1247 | 49.0% | 46.0% | 1259 | 52.4% | 83.1% | 71.1% | 86.8% | 4.7% | 64.4% | 48.4% | 51.4% | $0.38 | $0.57 | Sep 24 | 128K | Alibaba | Open Source | ||||||||||
64 | Latest | Aug 13, 2024 | 37.5% | 1230 | 51.0% | 42.0% | 1266 | 73.0% | 72.0% | 86.2% | 69.7% | 44.3% | 51.1% | Aug 24 | 128K | xAI | Proprietary | |||||||||||||||||
65 | Latest | Dec 3, 2024 | 37.1% | 1228 | 46.9% | 32.6% | 1245 | 39.1% | 76.6% | 84.0% | 1.8% | 67.1% | 43.6% | 49.2% | 92.1% | $0.8 | $3.2 | 300K | Amazon | Proprietary | ||||||||||||||
66 | 70B | Latest | Dec 6, 2024 | 36.6% | 1232 | 50.5% | 50.7% | 1257 | 41.1% | 77.0% | 68.9% | 86.0% | 4.0% | 779 | 82.7% | 48.3% | 56.7% | 92.1% | $0.1 | $0.4 | Dec 23 | Meta | Open Source | |||||||||||
67 | 27B | Latest | Jun 27, 2024 | 35.9% | 1203 | 28.1% | 1220 | 26.2% | 42.3% | 56.5% | 75.2% | 1028 | 58.1% | 43.6% | 37.4% | Jun 24 | 8K | Open Source | ||||||||||||||||
68 | Latest | Apr 14, 2025 | 35.5% | 47.6% | 35.0% | 24.0% | 1188.56 | 65.0% | 87.5% | 73.0% | 49.6% | 84.0% | $0.4 | $1.6 | Jun 24 | 1M | OpenAI | Proprietary | ||||||||||||||||
69 | Latest | Feb 5, 2025 | 33.6% | 23.4% | 1246 | 51.5% | 50.1% | 1306 | 55.5% | 86.8% | 71.6% | 68.0% | 1.2% | 78.3% | 57.5% | 53.2% | $0.075 | $0.3 | Aug 24 | 1M | Proprietary | |||||||||||||
70 | 109B | Latest | Apr 5, 2025 | 32.8% | 895.26 | 32.8% | 57.2% | 74.3% | 69.4% | Aug 24 | 10M | Meta | Open Source | |||||||||||||||||||||
71 | 70B | Old | Jul 23, 2024 | 32.7% | 1209 | 43.0% | 40.7% | 1242 | 34.4% | 60.0% | 66.4% | 86.0% | 5.0% | 69.0% | 49.1% | 50.9% | 87.5% | $1 | $3 | Dec 23 | 128K | Meta | Open Source | |||||||||||
72 | Old | Nov 20, 2024 | 32.6% | 35.2% | 18.2% | 31.0% | 964 | 1306 | 39.0% | 53.3% | 1361 | 13.0% | 42.5% | 69.0% | 74.7% | 86.0% | 1.5% | 1269 | 64.9% | 47.2% | 53.2% | 84.6% | $2.5 | $10 | Nov 23 | 128K | OpenAI | Proprietary | ||||||
73 | 32B | Latest | Nov 12, 2024 | 32.4% | 8.0% | 903.59 | 1227 | 41.0% | 47.3% | 1220 | 46.0% | 72.0% | 79.0% | 58.7% | 43.4% | 45.6% | $0.2 | $0.2 | Sep 24 | 128K | Alibaba | Open Source | ||||||||||||
74 | 405B | Latest | Jul 23, 2024 | 32.1% | 24.5% | 811.91 | 28.4% | 1260 | 51.1% | 53.3% | 1266 | 40.5% | 73.8% | 73.3% | 87.0% | 3.9% | 778 | 75.9% | 54.5% | 58.6% | 88.6% | $5 | $15 | Dec 23 | 128K | Meta | Open Source | |||||||
75 | 236B | Old | Sep 12, 2024 | 32.1% | 17.8% | 1255 | 42.0% | 39.3% | 1221 | 55.0% | 65.8% | 81.0% | 56.7% | 45.8% | 44.7% | $0.14 | $0.28 | Nov 23 | 128K | DeepSeek | Proprietary | |||||||||||||
76 | Latest | Mar 4, 2024 | 30.3% | 22.0% | 1236 | 50.4% | 41.3% | 1248 | 43.4% | 60.1% | 68.5% | 84.0% | 10.1% | 908 | 63.9% | 54.3% | 56.2% | $15 | $75 | Aug 23 | 200K | Anthropic | Proprietary | |||||||||||
77 | 8B | Old | Sep 24, 2024 | 28.7% | 1167 | 38.4% | 33.3% | 1177 | 58.7% | 58.7% | 52.6% | 44.0% | 42.7% | $0.0375 | $0.15 | Sep 24 | 1M | Proprietary | ||||||||||||||||
78 | 12B | Latest | Jul 24, 2024 | 28.7% | 33.0% | 25.3% | 16.9% | 40.0% | 44.8% | 68.0% | 11.2% | 51.8% | 33.4% | 32.0% | $0.15 | $0.15 | Apr 24 | 128K | Mistral | Proprietary | ||||||||||||||
79 | 400B | Latest | Apr 5, 2025 | 28.2% | 16.0% | 1006.52 | 43.4% | 1265 | 69.8% | 1273 | 80.5% | 84.6% | 73.4% | $0.19 | $0.49 | Aug 24 | 1M | Meta | Open Source | |||||||||||||||
80 | Latest | Dec 3, 2024 | 27.5% | 1200 | 42.0% | 34.7% | 1217 | 30.8% | 73.3% | 79.0% | 1.8% | $0.06 | $0.24 | 300K | Amazon | Proprietary | ||||||||||||||||||
81 | 32B | Latest | Sep 19, 2024 | 27.0% | 49.5% | 83.1% | 69.2% | 83.9% | 3.0% | 79.5% | Sep 24 | 128K | Alibaba | Open Source | ||||||||||||||||||||
82 | Mercury Coder Small | Preview | Feb 26, 2025 | 25.0% | 25.0% | Inception Labs | Proprietary | |||||||||||||||||||||||||||
83 | Latest | Jul 18, 2024 | 24.1% | 25.5% | 3.6% | 1245 | 43.0% | 35.3% | 1272 | 35.6% | 75.0% | 63.1% | 82.0% | 1.7% | 56.8% | 44.5% | 43.3% | $0.15 | $0.6 | Oct 23 | 128K | OpenAI | Proprietary | |||||||||||
84 | Old | Mar 7, 2024 | 23.5% | 22.2% | 1184 | 33.3% | 25.9% | 1179 | 22.9% | 40.9% | 42.3% | 75.2% | $0.25 | $1.25 | Aug 23 | 200K | Anthropic | Proprietary | ||||||||||||||||
85 | 14B | Latest | Nov 12, 2024 | 23.4% | Sep 24 | 128K | Alibaba | Open Source | ||||||||||||||||||||||||||
86 | 9B | Latest | Jun 27, 2024 | 22.5% | 1180 | 15.2% | 19.5% | 36.6% | 52.1% | 71.3% | 10.1% | 859 | 52.6% | 35.1% | 28.4% | Jun 24 | 8K | Open Source | ||||||||||||||||
87 | Latest | Jan 14, 2025 | 21.9% | 11.1% | 35.3% | 256K | Mistral | Proprietary | ||||||||||||||||||||||||||
88 | 14B | Latest | Sep 19, 2024 | 20.9% | 45.5% | 80.0% | 63.7% | 80.0% | 4.2% | 81.0% | Sep 24 | 128K | Alibaba | Open Source | ||||||||||||||||||||
89 | 7B | Latest | Nov 12, 2024 | 20.3% | Sep 24 | 128K | Alibaba | Open Source | ||||||||||||||||||||||||||
90 | Latest | Dec 3, 2024 | 20.2% | 1181 | 40.0% | 25.1% | 1198 | 33.7% | 69.3% | 76.0% | 1.6% | $0.04 | $0.14 | 128K | Amazon | Proprietary | ||||||||||||||||||
91 | 8B | Latest | Jul 23, 2024 | 19.7% | 1174 | 27.0% | 15.3% | 1171 | 16.6% | 64.0% | 44.3% | 71.0% | 5.4% | $0.3 | $0.6 | Dec 23 | 128K | Meta | Open Source | |||||||||||||||
92 | 12B | Latest | Mar 12, 2025 | 19.1% | 19.1% | Open Source | ||||||||||||||||||||||||||||
93 | Latest | Apr 14, 2025 | 17.6% | 25.3% | 10.0% | 50.3% | 80.1% | 55.0% | 29.4% | 75.0% | $0.1 | $0.4 | Jun 24 | 1M | OpenAI | Proprietary | ||||||||||||||||||
94 | Mercury Coder Mini | Preview | Feb 26, 2025 | 17.0% | 17.0% | Inception Labs | Proprietary | |||||||||||||||||||||||||||
95 | 27B | Latest | Mar 12, 2025 | 15.2% | 25.4% | 4.9% | 1244 | 42.4% | 1339 | 89.0% | 67.5% | 76.9% | 64.9% | 3.0% | Open Source | |||||||||||||||||||
96 | 4B | Latest | Mar 12, 2025 | 11.7% | 11.7% | Open Source | ||||||||||||||||||||||||||||
97 | 3B | Latest | Nov 12, 2024 | 10.8% | Sep 24 | 32K | Alibaba | Open Source | ||||||||||||||||||||||||||
98 | 1.5B | Latest | Nov 12, 2024 | 6.1% | Sep 24 | 32K | Alibaba | Open Source | ||||||||||||||||||||||||||
99 | 1.5B | Latest | Sep 19, 2024 | 4.7% | 32.1% | 15.8% | Sep 24 | 32K | Alibaba | Open Source | ||||||||||||||||||||||||
100 | 0.5B | Latest | Nov 12, 2024 | 2.0% | Sep 24 | 32K | Alibaba | Open Source | ||||||||||||||||||||||||||
101 | 0.5B | Latest | Sep 19, 2024 | 2.0% | 14.9% | 25.2% | Sep 24 | 32K | Alibaba | Open Source | ||||||||||||||||||||||||
102 | 14B | Latest | Jan 20, 2025 | 1481 | 59.1% | 49.0% | 69.7% | Sep 24 | DeepSeek | Open Source | ||||||||||||||||||||||||
103 | Preview | Nov 20, 2024 | 1450 | 58.5% | 91.6% | 52.5% | DeepSeek | Proprietary | ||||||||||||||||||||||||||
104 | 8B | Latest | Jan 20, 2025 | 1205 | 49.0% | 50.4% | Dec 24 | DeepSeek | Open Source | |||||||||||||||||||||||||
105 | 7B | Latest | Jan 20, 2025 | 1189 | 49.1% | 59.0% | 55.5% | Sep 24 | DeepSeek | Open Source | ||||||||||||||||||||||||
106 | 1.5B | Latest | Jan 20, 2025 | 954 | 33.8% | 20.0% | 28.9% | Sep 24 | DeepSeek | Open Source | ||||||||||||||||||||||||
107 | Preview | Feb 17, 2025 | 84.0% | 90.8% | 95.8% | Nov 24 | 1M | xAI | Proprietary | |||||||||||||||||||||||||
108 | Sky T1 Preview | 32B | Preview | Jan 11, 2025 | 56.8% | 69.2% | 43.3% | NovaSky | Open Source | |||||||||||||||||||||||||
109 | 14B | Latest | Dec 12, 2024 | 56.1% | 80.4% | 70.4% | 84.8% | 63.0% | Microsoft | Open Source | ||||||||||||||||||||||||
110 | 123B | Latest | Nov 18, 2024 | 52.0% | 72.0% | 85.0% | 6.6% | $2 | $6 | 128K | Mistral | Proprietary | ||||||||||||||||||||||
111 | 32B | Latest | Mar 24, 2025 | 46.0% | 82.2% | 68.8% | Alibaba | Open Source | ||||||||||||||||||||||||||
112 | 24B | Latest | Mar 17, 2025 | 46.0% | 69.3% | 66.8% | 80.6% | 62.8% | 5.6% | 128K | Mistral | Open Source | ||||||||||||||||||||||
113 | 24B | Latest | Jan 30, 2025 | 45.3% | 70.6% | 66.3% | 81.0% | 3.1% | 82.9% | 32K | Mistral | Proprietary | ||||||||||||||||||||||
114 | 90B | Latest | Sep 25, 2024 | 43.0% | 68.0% | 86.0% | 4.3% | $0.9 | $0.9 | Dec 23 | 128K | Meta | Open Source | |||||||||||||||||||||
115 | 11B | Latest | Sep 25, 2024 | 26.0% | 51.9% | 73.0% | 5.5% | $0.19 | $0.19 | Dec 23 | 128K | Meta | Open Source | |||||||||||||||||||||
116 | 3B | Latest | Sep 25, 2024 | 1048 | 21.0% | 1093 | 49.0% | 22.2% | 63.4% | 7.9% | $0.08 | $0.1 | Dec 23 | 128K | Meta | Open Source | ||||||||||||||||||
117 | 1B | Latest | Sep 25, 2024 | 1009 | 14.0% | 1044 | 27.0% | 11.9% | 49.3% | 20.7% | $0.04 | $0.08 | Dec 23 | 128K | Meta | Open Source | ||||||||||||||||||
118 | Latest | May 7, 2025 | 77.2% | 66.1% | 89.4% | $0.4 | $2 | Mistral | ||||||||||||||||||||||||||
119 | 111B | Latest | Mar 13, 2025 | 1288 | 1310 | $2.5 | $10 | 256K | Cohere | Open Source | ||||||||||||||||||||||||
120 | R1 1776 | Latest | Feb 18, 2025 | 90.5% | 81.0% | $2 | $8 | 128K | Perplexity | Open Source | ||||||||||||||||||||||||
121 | Old | Dec 10, 2024 | 1262 | 1258 | 2.4% | Nov 23 | 128K | DeepSeek | Proprietary | |||||||||||||||||||||||||
122 | Latest | Nov 15, 2024 | Sep 24 | 1M | Alibaba | Proprietary | ||||||||||||||||||||||||||||
123 | 3B | Latest | Sep 19, 2024 | 43.7% | 7.0% | Sep 24 | 32K | Alibaba | Open Source | |||||||||||||||||||||||||
124 | 2B | Latest | Jul 31, 2024 | 15.0% | 51.3% | 7.0% | Jun 24 | 8K | Open Source |