You would think that I, who fashions myself as a critical reviewer of grammar software, would just shell out the $30/mo. that a company like Grammarly charges for its service, but I don't. I hate when companies simply get in the habit of taking money out of your account, even if I have let them; and besides, I don't really need it, though I'm sure you can tell from reading even this far that I could use a few pointers.
Instead, I read carefully all online reviews. I use their free service on the assumption that they'll put their best foot forward and show you how much they can do, right away (I'm not sure if this is necessarily true). And finally, I'm honest with my students, so that I know which ones use Grammarly all the time, and I critically review the papers they write (Grammarly didn't catch this?).
On the online review front, one of the biggest ones came out in 2011, by the Grammarist weblog. In fact I was unable to find an actual date of publication on the review (this is a bad sign, but quite common) but inferred from the comments that it was published around then. The Grammarist weblog was quite stinging, but because it was careful and accurate, and covered a wide variety of errors, I read it carefully. Most of the comments addressed Grammarly's bad business ethics: not announcing its terms before it collects your information, installing a free plugin that's worthless until you actually pay your money, taking payments without warning, etc. Personally, I doubt Grammarly has done much about these, and I also suspect that these people had nowhere else to complain, so they took the trouble to register with Grammarist, just in order to complain about bad business practices. But they were legion; there are lots of complaints about them.
Grammarist noticed that a wide range of contextual spelling errors (such as its/it's, there/their/they're, your/you're, all ready/already) went uncorrected. All passives were noticed and discouraged, even if they were necessary or advisable. Dangling modifiers, redundancies, etc. went unnoticed. They gave credit when Grammarly did well, but they disliked it and requested that they stop being able to advertise themselves on the Grammarist weblog. (As a sidebar, I'll point out the Grammarly is a very aggressive online advertiser - you type anything including the word "grammar" into Google, and Grammarly's paid ad will be there waiting for you. So it's no surprise that it tried to get a reputable grammar website to allow it to advertise itself, for a price, of course).
Careful reading of the comments pointed out a few other things, though. One was that, within a year or two, Grammarly had read the review, and worked on many of the problems it had pointed out. This is one thing a huge budget can do for Grammarly. It has people scanning the reviews (hello!) and responding to the errors people point out. If it is at all possible to fix them, using the technology and programming skills that Grammarly has at hand, they'll do it. At least, with their budget, they are capable of doing it, because they had millions in startup funds, and were able to jump-start a business that at least tried to manage such things.
But to get back to the review's main point, Grammarly advertises itself as the most accurate in the market; as correcting your errors and offering helpful reasons why they're wrong; as picking out the contextual misspellings; and checking for plagiarism as well. And they felt that Grammarly had not done this properly, and was therefore guilty of fraudulent advertising.
The comments by and large agreed that it was fraudulent, but more in its business practices than its grammar practices. It's a given in this field that the consumers don't know grammar, and are thus at a disadvantage when it comes to evaluating a grammar-checking service. And this applies double to international (ESL) students, who are not only unfamiliar with grammatical nuance, but also unfamiliar with shill online installation scams, or proper ways of getting revenge against such companies. The vast majority of complaints were by American writers, but some of them were quite serious consumers, submitting entire manuscripts, and as some pointed out, a service doesn't have to be right 100% of the time to be useful in at least getting you to check things that they consider could be reworded.
To our students, services like Grammarly have left the impression that passive voice is simply wrong (even when their advisors demand it); that when in doubt present tense is always best (in recent days, X become common, says one student who uses Grammarly); and that all other things the teacher points out must be simple harping, since it went under Grammarly's radar successfully. To be fair, I have no idea what they really think (why should they tell me everything?), but, I can tell you that there is still a fair amount of bad writing out there.
As for the plagiarism checking service, I'll say this: it's popular with students. They are scared to death of plagiarism, and they don't really know how to avoid it, and if something will offer them, with any accuracy at all, to point it out, they are more than grateful. Both Grammarist and the commenters pointed out that the Grammarly service was flawed, but if somebody can get a computer to take 3-6 words at a time, from every line, run each chunk through the computer database repeatedly, and simply report to the writer the times it comes up with a hit, that's more than I'm doing for them, and, presumably, it's catching at least the most egregious cases. This is an entirely separate issue from pointing out that disparate is different from desperate or affect is generally a verb, while effect is generally a noun (with a whole host of exceptions). And I believe that some think the entire service is worth its $30/month, in spite of any negative reviews.
One thing I yearn for is what many commenters have pointed out: what you want to do is line it up with other grammar software, and see which one wins. Ginger comes to mind; people like Ginger as the underdog in this matchup (not having millions in startup money) - which of these errors is Ginger catching that Grammarly is not? Are they on to each other's tricks, in terms of being able to program in some "catches" that the other has figured out? I know they are both watching Word grammar-check (the Ubiquitous, the King of all Grammar-checks) trying to do for their customers at least what Word can do. What I don't know is when they figured out how hard definately/defiantly was for people, and how to guess people's intentions by how close they got to it when they first typed it in. Our students, remember, are far more likely to get the vowels wrong, and have their first spelling be way off base, not even close to definitely. When these programs can get close to figuring out our students' intentions, I, like Grammarist, will be closer to actually recommending them.