Llama 3.1

Latest

Meta

•

Open Source

•

405B

# 52

Released

Jul 23, 2024

# 9

Knowledge Cutoff

Dec 23

# 8

Context Length

128K

Benchmarks

# 40

Code LiveBench

42.6%

# 16

SWEBench Verified

24.5%

# 22

WebDev Arena

811.91

# 29

LiveCodeBench v5

28.4%

# 27

Code LMArena

1260

# 39

GPQA Diamond

51.1%

# 26

Reason LiveBench

53.3%

# 33

ELO LMArena

1266

# 37

Math LiveBench

40.5%

# 24

MATH

73.8%

# 11

Human Eval

89.0%

# 22

NYT Connections

16.2%

# 14

Aider Old

66.2%

# 23

MMLU Pro

73.3%

# 10

MMLU

87.0%

# 16

Halluc. Hughes

3.9%

# 19

Aidan Bench

778

# 22

IF LiveBench

75.9%

# 27

Data LiveBench

54.5%

# 22

Lang LiveBench

45.5%

# 23

Avg LiveBench

58.6%

# 5

IF Evaluation

88.6%

Pricing

# 27

Input Cost /M

$5

# 30

Output Cost /M

$15