AI editing benchmark
We hid 90 mistakes in a 6,000-word short story and asked 19 AI models — the GPT models behind ChatGPT, Claude, Gemini and more — to find them
Updated
Click a model to read every comment it left on the story
The best AI for editing right now is Claude Fable 5.2, which caught 89 of 90 planted mistakes. GPT-6.1 Sol caught 87 at under a tenth of the cost ($0.20 vs $2.29), and Gemini 4 Argon is the best budget pick: 84 caught in 26 seconds for $0.12.
Free sample
The same 4 checks, plus 13 more — from dialogue to point of view. Runs on GPT-6.1 Sol, which caught 87 of 90 mistakes here. See all 17 checks
Drag & drop your manuscriptUpload your manuscript
.docx, .epub or .md
Extra flags are comments that landed on none of the planted mistakes
| Model | Caught | Grammar & spelling | Repetitions | Redundancy | Consistency | All in one | Comments | Extra flags | Time | Cost |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.2 | 89/90 | 45/45 | 14/15 | 15/15 | 15/15 | 88/90−1 | 117 | 17 | 4m 8s | $2.29 |
| Gemini 4 Carbon | 88/90 | 43/45 | 15/15 | 15/15 | 15/15 | 84/90−4 | 96 | 4 | 3m 53s | $0.83 |
| Claude Opus 5.5 | 88/90 | 44/45 | 14/15 | 15/15 | 15/15 | 83/90−5 | 114 | 9 | 1m 12s | $0.74 |
| GPT-6.1 Sol | 87/90 | 43/45 | 14/15 | 15/15 | 15/15 | 80/90−7 | 99 | 4 | 1m 4s | $0.20 |
| Gemini 4 Hydrogen | 84/90 | 41/45 | 14/15 | 15/15 | 14/15 | 68/90−16 | 89 | 1 | 42s | $0.26 |
| Gemini 4 Argon | 84/90 | 42/45 | 13/15 | 14/15 | 15/15 | 80/90−4 | 92 | 4 | 26s | $0.12 |
| GPT-6.1 Terra | 79/90 | 42/45 | 13/15 | 11/15 | 13/15 | 76/90−3 | 89 | 7 | 29s | $0.07 |
| GLM 5.3 | 78/90 | 41/45 | 12/15 | 12/15 | 13/15 | 70/90−8 | 121 | 10 | 4m 11s | $0.36 |
| Gemini 3.8 Flash | 77/90 | 41/45 | 10/15 | 12/15 | 14/15 | 72/90−5 | 81 | 0 | 1m 37s | $0.13 |
| Grok 4.6 | 76/90 | 42/45 | 12/15 | 9/15 | 13/15 | 66/90−10 | 104 | 7 | 4m 40s | $0.50 |
| Kimi K3 | 75/90 | 42/45 | 9/15 | 12/15 | 12/15 | 59/90−16 | 91 | 9 | 58s | $0.29 |
| GPT 5.5 | 75/90 | 40/45 | 10/15 | 11/15 | 14/15 | 67/90−8 | 99 | 6 | 1m 50s | $0.94 |
| Claude Sonnet 5.5 | 73/90 | 38/45 | 12/15 | 10/15 | 13/15 | 73/90±0 | 96 | 3 | 50s | $0.33 |
| Kimi K 2.6 | 69/90 | 39/45 | 9/15 | 10/15 | 11/15 | 49/90−20 | 102 | 3 | 4m 14s | $0.43 |
| Gemini 3.1 Pro | 67/90 | 40/45 | 8/15 | 10/15 | 9/15 | 51/90−16 | 87 | 0 | 49s | $0.63 |
| Claude Opus 4.8 | 65/90 | 42/45 | 3/15 | 11/15 | 9/15 | 65/90±0 | 226 | 6 | 55s | $0.66 |
| GPT-6 Luna | 59/90 | 35/45 | 11/15 | 5/15 | 8/15 | 60/90+1 | 67 | 3 | 1m 20s | $0.02 |
| DeepSeek V4 Pro | 55/90 | 39/45 | 3/15 | 4/15 | 9/15 | 43/90−12 | 101 | 7 | 1m 20s | $0.09 |
| Claude Sonnet 4.6 | 55/90 | 34/45 | 6/15 | 6/15 | 9/15 | 49/90−6 | 185 | 5 | 1m 25s | $0.30 |
Free to reuse with credit to icanwrite, under CC BY 4.0
Most editing feedback is taste: two good editors can disagree on pacing or voice, so the only way to score it is another AI's opinion. Grammar, repetitions, redundancy and consistency have right answers — a verb agrees or it doesn't, a name is spelled one way or two — so each mistake can be planted and every model scored the same way.
The draft is an original 6,000-word short story, clean apart from what we planted in it: 45 grammar and spelling errors, 15 repeated words, 15 redundancies and 15 contradictions. They're the hard kind — agreement across a long subject, a dangling modifier, a real word in the wrong place, a detail that changes chapters later — not typos a spellchecker would catch.
Each model reads the whole story once per check: grammar & spelling, repetitions, redundancy, and consistency. The prompts are adapted from the checks in icanwrite's editor, then frozen, so every model gets exactly the same instructions. Combined asks for all 4 in a single prompt, scored on its own: the gap shows how much a model loses when it's asked to look for everything at once.
Each model gets as much thinking as it can use while still checking a full chapter in a few minutes, since a writer won't wait much longer for comments. For most models that's their default effort, medium or high.
A mistake is caught when one of the model's comments lands on it — on either side, for a repetition or a contradiction. Scoring is plain text matching: no AI judges another AI. A comment quoting more than 300 characters catches nothing, so quoting a whole paragraph can't sweep up what's in it.
Time is the slowest of the 4 calls, since they run side by side. Cost is what the calls cost at the provider's API prices. Smart & Efficient is the cheapest model that caught at least 97% of what the smartest did.
Claude Fable 5.2: it caught all 45 planted errors on the grammar and spelling check, the only model to do so. Claude Opus 5.5 caught 44, Gemini 4 Carbon and GPT-6.1 Sol 43. They were the hard kind: agreement across a long subject, a dangling modifier, a real word in the wrong place. Grammar is also where the models are closest: 15 of the 18 caught at least 39.
See every comment Claude Fable 5.2 left →Yes. ChatGPT runs on OpenAI's GPT models, and the best one we tested, GPT-6.1 Sol, caught 87 of 90 planted mistakes, two fewer than the top model, for $0.20. GPT-6.1 Terra caught 79 for $0.07. We called the models through OpenAI's API with the same prompts for every model, so the ChatGPT app may answer differently, and which model it uses depends on your plan.
See every comment GPT-6.1 Sol left →At the top it's close: Anthropic's Claude Fable 5.2 caught 89 of 90, Google's Gemini 4 Carbon 88 and OpenAI's GPT-6.1 Sol 87. The difference is price and speed. Fable cost $2.29 and took four minutes, Carbon $0.83 and just under four, Sol $0.20 and 64 seconds. The cheaper Claude, Opus 5.5, caught 88 for $0.74 in 72 seconds. Claude is the most thorough, GPT the best value, Gemini the fastest at 26 seconds for Gemini 4 Argon's 84.
Gemini 4 Argon: 84 of 90 caught for $0.12 on a 6,000-word story. GPT-6.1 Terra caught 79 for $0.07. Below that the scores fall fast: GPT-6 Luna cost two cents and caught 59.
No. We also gave every model all four checks in a single prompt, and 15 of the 18 caught fewer. Kimi K2.6 lost 20 mistakes, Kimi K3 and Gemini 3.1 Pro 16 each, and even Claude Opus 5.5 lost 5. Ask for one kind of mistake at a time: grammar, then repetitions, then redundancy, then consistency.
The best models can. The consistency check planted 15 contradictions: a ledger that turns from green to red, a grandmother called Ruth and then Rose, a Saturday that's wrong only once you count the days. Claude Fable 5.2, Claude Opus 5.5, Gemini 4 Carbon, GPT-6.1 Sol and Gemini 4 Argon caught all 15. Eight others caught 11 to 14, and five caught 9 or fewer.
Free sample
The same 4 checks, plus 13 more — from dialogue to point of view. Runs on GPT-6.1 Sol, which caught 87 of 90 mistakes here. See all 17 checks
Drag & drop your manuscriptUpload your manuscript
.docx, .epub or .md
These checks come from icanwrite, a markdown editor for writers
Want to improve the benchmark? Share your ideas