Llama 3.1

Latest

Meta

•

Open Source

•

405B

# 57

Released

Jul 23, 2024

# 10

Knowledge Cutoff

Dec 23

# 9

Context Length

128K

Benchmarks

# 74

Code RankedAGI

32.1%

# 20

SWEBench Verified

24.5%

# 31

WebDev Arena

811.91

# 34

LiveCodeBench v5

28.4%

# 29

Code LMArena

1260

# 41

Code LiveBench 24.11

42.6%

# 43

GPQA Diamond

51.1%

# 27

Reason LiveBench

53.3%

# 34

ELO LMArena

1266

# 38

Math LiveBench

40.5%

# 24

MATH

73.8%

# 12

Human Eval

89.0%

# 21

NYT Connections

16.2%

# 24

MMLU Pro

73.3%

# 12

MMLU

87.0%

# 17

Halluc. Hughes

3.9%

# 19

Aidan Bench

778

# 22

IF LiveBench

75.9%

# 27

Data LiveBench

54.5%

# 23

Lang LiveBench

45.5%

# 24

Avg LiveBench

58.6%

# 6

IF Evaluation

88.6%

Pricing

# 28

Input Cost /M

$5

# 33

Output Cost /M

$15