Claude 3.5 Sonnet
OldAnthropic
•
Proprietary# 87
Released
Jun 20, 2024
# 12
Knowledge Cutoff
Apr 24
# 8
Context Length
200K
Benchmarks
# 80
Code RankedAGI
43.5%
# 40
SWEBench Verified
33.4%
# 33
LiveCodeBench v5
36.0%
# 28
Code LMArena
1282
# 18
Code LiveBench (old)
60.9%
# 59
GPQA Diamond
59.4%
# 20
Reason LiveBench (old)
58.7%
# 39
Text Arena
1268
# 25
Math LiveBench (old)
53.3%
# 28
MATH
71.1%
# 5
Human Eval
92.0%
# 12
Human Eval+
81.7%
# 21
MMLU Pro
76.1%
# 14
MMLU
88.7%
# 25
MMMU
68.3%
# 8
Aidan Bench
1439
# 35
IF LiveBench (old)
68.0%
# 22
Avg LiveBench (old)
59.7%
# 25
Data LiveBench
56.7%
# 10
Language LiveBench
53.2%
# 7
Quality Artificial Analysis
76
Pricing
# 31
Input Cost /M
$3
# 35
Output Cost /M
$15