---
title: "o3 Benchmarks - RankedAGI"
description: "Detailed benchmark and metadata record for o3."
source: "https://rankedagi.com/models/o3"
---

# o3

| Field | Value |
| --- | --- |
| Organization | OpenAI |
| License | Proprietary |
| Type | Thinking |
| Version | Latest |
| Released | 2025-04-16 |
| Context window | 200K |
| Knowledge cutoff | 2024-06-01 |
| Input cost per million tokens | $2 |
| Output cost per million tokens | $8 |
| Cached input cost per million tokens | $0.5 |
| Model ID | o3-2025-04-16 |
| Last updated | 2026-05-04T00:13:11.660Z |

## Benchmarks

| Benchmark | Category | Value | Description | Source |
| --- | --- | --- | --- | --- |
| RankedAGI Coding | coding | 57.3% | RankedAGI Coding Score |  |
| SWEBench Verified | coding | 69.1% | Agentic Coding | https://www.swebench.com/index.html |
| RankedAGI Agentic | Agents | 45.4% | RankedAGI Agentic Score |  |
| BrowseComp | Agents | 49.7% | A benchmark for browsing agents | https://openai.com/index/browsecomp/ |
| LiveCodeBench v6 | coding | 75.5% | LiveCodeBench v6 - Evaluation of Large Language Models for Code |  |
| LiveBench Coding 25.5 | coding | 77.9% | LiveBench Coding Score 2025-05-30 |  |
| ChatArena (LMSYS) | coding | 1460 | ChatArena (LMSYS) Coding ELO Score with Style Control |  |
| Codeforces ELO Score | coding | 2706 | Codeforces ELO Score |  |
| Aider Polyglot | coding | 79.6% | Aider Polyglot Code Completion Benchmark |  |
| RankedAGI Reasoning | reasoning | 56.2% | RankedAGI Reasoning Score |  |
| Humanity's Last Exam | reasoning | 14.7% | Multidisciplinary Reasoning (no tools) |  |
| Humanity's Last Exam w/ Tools | reasoning | 24.3% | Multidisciplinary Reasoning (with tools) |  |
| GPQA Diamond | reasoning | 87.7% | Generalized Prefix Question Answering Score (Reasoning) PhD Level Reasoning |  |
| LiveBench Reasoning 25.5 | reasoning | 91.0% | LiveBench Reasoning Score 2025-05-30 |  |
| Text Arena | general | 1431 | ChatArena (LMSYS) ELO Score | https://arena.ai/leaderboard/text |
| AIME 2025 | math | 88.9% | AIME 2025 Competition Math |  |
| AIME 2024 | math | 91.6% | AIME 2024 Competition Math |  |
| LiveBench Math Score | math | 80.7% | LiveBench Math Score 25-05 |  |
| MMMU | imaging | 82.9% | Multimodal Understanding College-level visual problem-solving |  |
| Hughes Hallucination Evaluation | safety | 6.8% | Computed using Vectara's Hallucination Evaluation Model, also known as HHEM. This evaluates how often an LLM introduces hallucinations when summarizing a document. | https://github.com/vectara/hallucination-leaderboard |
| LiveBench IF 25.5 | instruction-following | 84.3% | LiveBench Instruction Following Score 2025-05-30 |  |
| LiveBench Average 25.5 | general | 72.4% | LiveBench Average Score 2025-05-30 |  |
| LiveBench Coding 25.4 | coding | 77.9% | LiveBench Coding Score 2025-04 | https://livebench.ai/ |
| LiveBench Reasoning 25.4 | reasoning | 91.0% | LiveBench Reasoning Score 2025-04 | https://livebench.ai/ |
| LiveBench Agentic Coding 25.5 | coding | 28.3% | LiveBench Agentic Coding Score 2025-05-30 | https://livebench.ai/ |
| MMMU-Pro | imaging | 76.4% | Multimodal Understanding |  |
| RankedAGI Math | math | 63.3% | RankedAGI Math Score |  |
| RankedAGI Overall | general | 53.1% | Overall RankedAGI score |  |
| 𝜏²-Bench Telecom | Agents | 48.2% | Agentic Tool Use |  |
| 𝜏²-Bench Retail | Agents | 80.2% | Agentic Tool Use |  |
| 𝜏²-Bench Airline | Agents | 64.8% | Agentic Tool Use |  |
| HealthBench Hard | Health | 31.6% | Open-ended Health Queries |  |
| GDPval-AA Elo | Agents | 754 | Office Tasks (Artificial Analysis) |  |
| SvelteBench old | coding | 30.0% | SvelteBench - Benchmark for Svelte (Old v1) | https://khromov.github.io/svelte-bench/v1/v1-benchmark-results-merged.html |
| Code DesignArena | Design | 1070 |  | https://www.designarena.ai/leaderboard/code |
| Toolathlon | Agents | 17.0% | Benchmark to assess language agents' general tool use in realistic environments. | https://toolathlon.xyz/introduction |

## Model Links

| Title | URL |
| --- | --- |
| Release | https://openai.com/index/introducing-o3-and-o4-mini/ |