Claude 3.5 Sonnet
by Anthropic · United StatesThe 2024 coding benchmark other labs measured themselves against.
Specification
The numbers
- Maker
- Anthropic
- Released
- 2024-10
- Parameters
- Not reported
- Architecture
- Not reported
- Context window
- 200,000 tokens
- Max output
- 8,192 tokens
- Input
- Text, Image
- Output
- Text
- Reasoning
- No
- Tool calling
- Yes
- Knowledge cutoff
- 2024-04
- Licence
- Proprietary
- Availability
- Not reported
- Weights
- Not released
Cost
Price per million tokens
- Input
- $3.00 / M tokens
- Output
- $15.00 / M tokens
- Blended 3:1
- $6.00
Standard first-party API rate, excluding batch discounts. Reasoning models bill thinking tokens as output, so cost per answer can far exceed the cost per token suggests.
Published scores
Benchmarks
Figures published by Anthropic or taken from a public leaderboard. Row last checked 2026-08.
| Category | Score | Rank |
|---|---|---|
| ReasoningGPQA Diamond | 65.0% | 48 of 83 reporting the same tests |
| MathsCompetition mathematics, graded on the final answer | Not reported | — |
| CodingSWE-bench Verified, HumanEval | 71.3% | 1 of 2 reporting the same tests |
| KnowledgeMMLU-Pro | 78.0% | 9 of 77 reporting the same tests |
| MultimodalMMMU | 70.4% | 17 of 39 reporting the same tests |
| Instruction followingObeying an exact, checkable format | Not reported | — |
| Human preferenceLMArena Elo | 1268 | 13 of 24 reporting the same tests |
A category averages every benchmark in it that Claude 3.5 Sonnet reports. The rank counts only models that report the same tests, so it never compares an average over three benchmarks against an average over one.
Every reported test
| MMLU-ProKnowledge | Rank 9 of 77 models reporting | |
|---|---|---|
| GPQA DiamondReasoning | Rank 48 of 83 models reporting | |
| AIME 2025Maths | Not reported | No figure published |
| MATH-500Maths | Not reported | No figure published |
| SWE-bench VerifiedCoding | Rank 30 of 33 models reporting | |
| SWE-bench ProCoding | Not reported | No figure published |
| Terminal-Bench 2.1Coding | Not reported | No figure published |
| Frontier-Bench v0.1Reasoning | Not reported | No figure published |
| Terminal-Bench 4.0Coding | Not reported | No figure published |
| LiveCodeBenchCoding | Not reported | No figure published |
| HumanEvalCoding | Rank 1 of 46 models reporting | |
| MMMUMultimodal | Rank 17 of 39 models reporting | |
| IFEvalInstruction following | Not reported | No figure published |
| LMArena EloHuman preference | Rank 13 of 24 models reporting |
Where these numbers come from
Every score on this page is a published figure, taken from the model's own card, system card, technical report or release post, or from a public leaderboard. CorX Labs did not run these evaluations. Most are self-reported by the lab that built the model, which means they were produced under that lab's own choice of prompt, scaffold and number of attempts — so treat them as a starting point for a shortlist, not as a settled ranking.
A score someone other than the model's maker measured is marked Independent and names its measurer. Those are the stronger numbers on this page — an outside harness has no reason to flatter anyone — and there are not many of them.
Where a figure has not been published, the cell reads Not reported rather than an estimate. Nothing here is inferred, interpolated or guessed. Each model records the month its row was last checked. Full method and caveats.
Same maker
Other models from Anthropic
| Claude Fable 5.1Anthropic | Anthropic | 1M | $10.00 | $50.00 | — | — | — | — | Proprietary |
| Claude Mythos 5.1Anthropic | Anthropic | 1M | $10.00 | $50.00 | — | — | — | — | Proprietary |
| Claude Opus 5Anthropic | Anthropic | 1M | $5.00 | $25.00 | — | — | — | — | Proprietary |
| Claude Sonnet 5Anthropic | Anthropic | 1M | $2.00 | $10.00 | — | — | — | — | Proprietary |
| Claude Fable 5Anthropic | Anthropic | 1M | $10.00 | $50.00 | — | — | — | — | Proprietary |
| Claude Mythos 5Anthropic | Anthropic | 1M | $10.00 | $50.00 | — | — | — | — | Proprietary |
| Claude Opus 4.5Anthropic | Anthropic | 200K | $5.00 | $25.00 | — | 87% | 96% | 80.9% | Proprietary |
| Claude Haiku 4.5Anthropic | Anthropic | 200K | $1.00 | $5.00 | — | 73% | 77% | 73.3% | Proprietary |
| Claude Sonnet 4.5Anthropic | Anthropic | 1M | $3.00 | $15.00 | — | 83.4% | 87% | 77.2% | Proprietary |
| Claude Opus 4.1Anthropic | Anthropic | 200K | $15.00 | $75.00 | — | 80.9% | 78% | 74.5% | Proprietary |
| Claude Sonnet 4Anthropic | Anthropic | 200K | $3.00 | $15.00 | — | 75.4% | 70.5% | 72.7% | Proprietary |
| Claude 3.7 SonnetAnthropic | Anthropic | 200K | $3.00 | $15.00 | — | 68% | 61.3% | 62.3% | Proprietary |
| Claude 3.5 HaikuAnthropic | Anthropic | 200K | $0.80 | $4.00 | 65% | 41.6% | — | 40.6% | Proprietary |
| Claude 3 OpusAnthropic | Anthropic | 200K | $15.00 | $75.00 | 68.5% | 50.4% | — | — | Proprietary |