Claude 3.5 Haiku

Latest

Anthropic

•

Proprietary

# 40

Released

Oct 22, 2024

# 6

Knowledge Cutoff

Jul 24

# 5

Context Length

200K

Benchmarks

# 28

Code LiveBench

51.4%

# 14

Aider Polyglot

28.0%

# 11

SWEBench Verified

40.6%

# 7

WebDev Arena

1134.43

# 24

Code LMArena

1263

# 53

GPQA Diamond

41.6%

# 46

Reason LiveBench

29.3%

# 43

ELO LMArena

1239

# 41

Math LiveBench

35.5%

# 26

MATH

73.0%

# 14

Human Eval

88.1%

# 29

NYT Connections

10.3%

# 5

Aider Old

75.2%

# 39

MMLU Pro

65.0%

# 23

MMLU

81.0%

# 23

Halluc. Hughes

4.9%

# 15

Aidan Bench

904

# 45

IF LiveBench

61.9%

# 50

Data LiveBench

42.4%

# 38

Lang LiveBench

35.4%

# 45

Avg LiveBench

47.8%

Pricing

# 22

Input Cost /M

$1

# 24

Output Cost /M

$5

# 6

Cached Cost /M

$0.1

Additional Details

Model ID

claude-3-5-haiku-20241022

Last Updated

3 days ago