Can AI be your editor? We tested the best LLMs on 90 mistakes

TL;DR: The top models now catch 93–98% of planted mistakes. Claude Opus 5.5 found the most (88/90). GPT-6.1 Sol was one behind at a quarter of the price. See the full AI editing benchmark →

So you wrote something.

A blog post. An essay. Maybe even a book.

Maybe you even call yourself a writer for the occasion.

You aren't going to publish it right away, are you?

There might be mistakes. Typos. Imperfections...

You need an editor.

Wait...

Can't AI be your editor?

It can. To some extent.

If you drop some text into ChatGPT and ask it to find typos, it will find some. But will it find all of them?

Can it find imperfections that are a bit more subtle? Like awkward repetitions?

And which model will be best to use for this kind of work?

It's hard to tell.

Most AI benchmarks are useless for writers

Yes, a website like artificialanalysis.ai has tons of benchmarks.

Most of them are focused on coding.

There is also Humanity's Last Exam. Sounds... adjacent.

Until you look at the actual questions. Like this one:

"In Greek mythology, who was Jason's maternal great-grandfather?"

AI models that ace it will probably spot a comma splice. Or they won't. Who knows...

So we decided to test the models ourselves.

Evaluating AI as an editor is tricky

A lot of what editors do comes down to judgment calls.

There is no clear right or wrong.

Is a metaphor fresh and innovative? Or forced and over the top? People might disagree.

You can't test this.

So we decided to focus on what CAN be tested.

How our AI editing test is designed

50% of the test is grammar. Things that are clear mistakes.

Agreement errors. Dangling participle. Faulty parallelism. Stuff like this.

The other 50% — repetitions, redundancies, and consistency. Names spelled differently. Words and phrases that any editor would cut...

Then we generated a synthetic text and planted 90 mistakes in it.

Then we ran 5 prompts with 16 models - ChatGPT, Claude, Gemini and others. One for each check — and one all-in-one check.

Just to test if separating editing into narrow tasks improves performance.

What we learned from benchmarking AI editors

See the results here: AI editing benchmark.

There are graphs. And tables. Fun stuff.

Everything is transparent.

You can click any model and see exactly what it missed.

Our biggest takeaways?

In just half a year, the models made a huge leap. GPT-5.5, released in April, found 75 mistakes (83%). GPT-6.1 Sol found 87 (97%).

Claude Opus 5.5 tops the ranking. But the difference is just 1 borderline mistake.

Yet it's almost 4 times more expensive.

Gemini 4 Argon is the cheapest and the fastest among models scoring above 90%.

Bottom line: if you need to fix grammar, you can rely on the top models

And that wasn't the case even a year ago.

More observations and insights — on the AI editing benchmark page.

Want to help make the benchmark better? Write to us.