Llama 3.1

Latest

Meta

•

Open Source

•

405B

# 75

Released

Jul 23, 2024

# 13

Knowledge Cutoff

Dec 23

# 11

Context Length

128K

Benchmarks

# 99

Code RankedAGI

32.1%

# 36

SWEBench Verified

24.5%

# 34

WebDev Arena

811.91

# 37

LiveCodeBench v5

28.4%

# 34

Code LMArena

1260

# 41

Code LiveBench (old)

42.6%

# 64

GPQA Diamond

51.1%

# 27

Reason LiveBench (old)

53.3%

# 39

ELO LMArena

1266

# 38

Math LiveBench (old)

40.5%

# 24

MATH

73.8%

# 12

Human Eval

89.0%

# 21

NYT Connections

16.2%

# 30

MMLU Pro

73.3%

# 19

MMLU

87.0%

# 21

Halluc. Hughes

3.9%

# 19

Aidan Bench

778

# 22

IF LiveBench (old)

75.9%

# 24

Avg LiveBench (old)

58.6%

# 9

IF Evaluation

88.6%

Pricing

# 32

Input Cost /M

$5

# 35

Output Cost /M

$15