AI editing benchmark
Which AI is best for editing?
We hid 90 mistakes in a 6,000-word short story and asked 16 AI models — the GPT models behind ChatGPT, Claude, Gemini and more — to find them.
Updated
- Smartest EditorClaude Opus 5.588/90 caught · $0.74
- Smart & EfficientGPT-6.1 Sol87/90 caught · $0.20
Mistakes caught by each AI model
Click a model to read every comment it left on the story
- Grammar & spelling
- Repetitions
- Redundancy
- Consistency
- Paler bar: all four checks in one prompt
- Claude Opus 5.5Anthropic · Oct 27, 202688/9098%1m 12s$0.74
- Claude Opus 5.5 (combined)83/9092%1m 54s$0.33
- GPT-6.1 SolOpenAI · Oct 27, 202687/9097%1m 4s$0.20
- GPT-6.1 Sol (combined)80/9089%1m 9s$0.08
- Gemini 4 ArgonGoogle · Oct 27, 202684/9093%26s$0.12
- Gemini 4 Argon (combined)80/9089%48s$0.05
- GPT-6.1 TerraOpenAI · Oct 27, 202679/9088%29s$0.07
- GPT-6.1 Terra (combined)76/9084%35s$0.02
- GLM 5.3Z.ai · Oct 28, 202678/9087%4m 11s$0.36
- GLM 5.3 (combined)70/9078%3m 4s$0.12
- Gemini 3.8 FlashGoogle · Oct 27, 202677/9086%1m 37s$0.13
- Gemini 3.8 Flash (combined)72/9080%55s$0.03
- Grok 4.6xAI · Oct 28, 202676/9084%4m 40s$0.50
- Grok 4.6 (combined)66/9073%4m 27s$0.14
- Kimi K3Moonshot AI · Oct 28, 202675/9083%58s$0.29
- Kimi K3 (combined)59/9066%1m 4s$0.07
- GPT 5.5OpenAI · Oct 28, 202675/9083%1m 50s$0.94
- GPT 5.5 (combined)67/9074%1m 50s$0.28
- Claude Sonnet 5.5Anthropic · Oct 27, 202673/9081%50s$0.33
- Claude Sonnet 5.5 (combined)73/9081%1m 32s$0.17
- Kimi K 2.6Moonshot AI · Oct 28, 202669/9077%4m 14s$0.43
- Kimi K 2.6 (combined)49/9054%3m 39s$0.14
- Gemini 3.1 ProGoogle · Oct 28, 202667/9074%49s$0.63
- Gemini 3.1 Pro (combined)51/9057%1m 28s$0.30
- Claude Opus 4.8Anthropic · Oct 28, 202665/9072%55s$0.66
- Claude Opus 4.8 (combined)65/9072%1m 1s$0.19
- GPT-6 LunaOpenAI · Oct 27, 202659/9066%1m 20s$0.02
- GPT-6 Luna (combined)60/9067%1m 49s$0.0049
- DeepSeek V4 ProDeepSeek · Oct 28, 202655/9061%1m 20s$0.09
- DeepSeek V4 Pro (combined)43/9048%21s$0.03
- Claude Sonnet 4.6Anthropic · Oct 28, 202655/9061%1m 25s$0.30
- Claude Sonnet 4.6 (combined)49/9054%1m 44s$0.10
What stood out in the results
The best AI for editing right now is Claude Opus 5.5, which caught 88 of 90 planted mistakes. GPT-6.1 Sol caught 87 at under a third of the cost ($0.20 vs $0.74), and Gemini 4 Argon is the best budget pick: 84 caught in 26 seconds for $0.12.
- Claude Opus 5.5 and GPT-6.1 Sol are near-perfect: 88 and 87 of 90
- 13 of 16 models do worse on the all-in-one prompt
- The top three caught all 15 contradictions
- Above 90%, Gemini 4 Argon is the cheapest, at $0.12
- Gemini 4 Argon is the fastest top model: 26 seconds
- GPT-6.1 Terra beats every open-weight model, for $0.07
Scores for each check
Extra flags are comments that landed on none of the planted mistakes
| Model | Caught | Grammar & spelling | Repetitions | Redundancy | Consistency | All in one | Comments | Extra flags | Time | Cost |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 | 88/90 | 44/45 | 14/15 | 15/15 | 15/15 | 83/90−5 | 114 | 9 | 1m 12s | $0.74 |
| GPT-6.1 Sol | 87/90 | 43/45 | 14/15 | 15/15 | 15/15 | 80/90−7 | 99 | 4 | 1m 4s | $0.20 |
| Gemini 4 Argon | 84/90 | 42/45 | 13/15 | 14/15 | 15/15 | 80/90−4 | 92 | 4 | 26s | $0.12 |
| GPT-6.1 Terra | 79/90 | 42/45 | 13/15 | 11/15 | 13/15 | 76/90−3 | 89 | 7 | 29s | $0.07 |
| GLM 5.3 | 78/90 | 41/45 | 12/15 | 12/15 | 13/15 | 70/90−8 | 121 | 10 | 4m 11s | $0.36 |
| Gemini 3.8 Flash | 77/90 | 41/45 | 10/15 | 12/15 | 14/15 | 72/90−5 | 81 | 0 | 1m 37s | $0.13 |
| Grok 4.6 | 76/90 | 42/45 | 12/15 | 9/15 | 13/15 | 66/90−10 | 104 | 7 | 4m 40s | $0.50 |
| Kimi K3 | 75/90 | 42/45 | 9/15 | 12/15 | 12/15 | 59/90−16 | 91 | 9 | 58s | $0.29 |
| GPT 5.5 | 75/90 | 40/45 | 10/15 | 11/15 | 14/15 | 67/90−8 | 99 | 6 | 1m 50s | $0.94 |
| Claude Sonnet 5.5 | 73/90 | 38/45 | 12/15 | 10/15 | 13/15 | 73/90±0 | 96 | 3 | 50s | $0.33 |
| Kimi K 2.6 | 69/90 | 39/45 | 9/15 | 10/15 | 11/15 | 49/90−20 | 102 | 3 | 4m 14s | $0.43 |
| Gemini 3.1 Pro | 67/90 | 40/45 | 8/15 | 10/15 | 9/15 | 51/90−16 | 87 | 0 | 49s | $0.63 |
| Claude Opus 4.8 | 65/90 | 42/45 | 3/15 | 11/15 | 9/15 | 65/90±0 | 226 | 6 | 55s | $0.66 |
| GPT-6 Luna | 59/90 | 35/45 | 11/15 | 5/15 | 8/15 | 60/90+1 | 67 | 3 | 1m 20s | $0.02 |
| DeepSeek V4 Pro | 55/90 | 39/45 | 3/15 | 4/15 | 9/15 | 43/90−12 | 101 | 7 | 1m 20s | $0.09 |
| Claude Sonnet 4.6 | 55/90 | 34/45 | 6/15 | 6/15 | 9/15 | 49/90−6 | 185 | 5 | 1m 25s | $0.30 |
How we tested the AI editors
Why these four checks
Most editing feedback is taste: two good editors can disagree on pacing or voice, so the only way to score it is another AI's opinion. Grammar, repetitions, redundancy and consistency have right answers — a verb agrees or it doesn't, a name is spelled one way or two — so each mistake can be planted and every model scored the same way.
One story, 90 planted mistakes
The draft is an original 6,000-word short story, clean apart from what we planted in it: 45 grammar and spelling errors, 15 repeated words, 15 redundancies and 15 contradictions. They're the hard kind — agreement across a long subject, a dangling modifier, a real word in the wrong place, a detail that changes chapters later — not typos a spellchecker would catch.
4 checks, one call each
Each model reads the whole story once per check: grammar & spelling, repetitions, redundancy, and consistency. The prompts are adapted from the checks in icanwrite's editor, then frozen, so every model gets exactly the same instructions.
What counts as a catch
A mistake is caught when one of the model's comments lands on it — on either side, for a repetition or a contradiction. Scoring is plain text matching: no AI judges another AI. A comment quoting more than 300 characters catches nothing, so quoting a whole paragraph can't sweep up what's in it.
All in one
The paler bar runs the same 4 checks as a single prompt, scored on its own. The gap shows how much a model loses when it's asked to look for everything at once.
Time and cost
Time is the slowest of the 4 calls, since they run side by side. Cost is what the calls cost at the provider's API prices. Smart & Efficient is the cheapest model that caught at least 97% of what the smartest did.
Questions about AI editing
Claude Opus 5.5, by a hair. On the grammar and spelling check it caught 44 of 45 planted errors, GPT-6.1 Sol 43 and Gemini 4 Argon 42. They were the hard kind: agreement across a long subject, a dangling modifier, a real word in the wrong place. Grammar is also where the models are closest: 13 of the 16 caught at least 39.
See every comment Claude Opus 5.5 left →Yes. ChatGPT runs on OpenAI's GPT models, and the best one we tested, GPT-6.1 Sol, caught 87 of 90 planted mistakes, one fewer than the top model, for $0.20. GPT-6.1 Terra caught 79 for $0.07. We called the models through OpenAI's API with the same prompts for every model, so the ChatGPT app may answer differently, and which model it uses depends on your plan.
See every comment GPT-6.1 Sol left →At the top it's close: Anthropic's Claude Opus 5.5 caught 88 of 90, OpenAI's GPT-6.1 Sol 87 and Google's Gemini 4 Argon 84. The difference is price and speed. Opus cost $0.74 and took 72 seconds, Sol $0.20 and 64 seconds, Argon $0.12 and 26 seconds. Claude is the most thorough, GPT the best value, Gemini the fastest.
Gemini 4 Argon: 84 of 90 caught for $0.12 on a 6,000-word story. GPT-6.1 Terra caught 79 for $0.07. Below that the scores fall fast: GPT-6 Luna cost two cents and caught 59.
No. We also gave every model all four checks in a single prompt, and 13 of the 16 caught fewer. Kimi K2.6 lost 20 mistakes, Kimi K3 and Gemini 3.1 Pro 16 each, and even Claude Opus 5.5 lost 5. Ask for one kind of mistake at a time: grammar, then repetitions, then redundancy, then consistency.
The best models can. The consistency check planted 15 contradictions: a ledger that turns from green to red, a grandmother called Ruth and then Rose, a Saturday that's wrong only once you count the days. Claude Opus 5.5, GPT-6.1 Sol and Gemini 4 Argon caught all 15. Eight others caught 11 to 14, and five caught 9 or fewer.
These checks come from icanwrite, a markdown editor for writers. Run them on your own draft
Want to improve the benchmark? Share your ideas