Can Avvo's Ratings Be Gamed?
Critics of lawyer-rating site Avvo.com have found their standard-bearer in Massachusetts lawyer Leonard H. Kesten. In an article published in Massachusetts Lawyers Weekly, Kesten describes how he "impersonated" a young lawyer at his firm and manipulated his Avvo profile until he raised the lawyer's ranking to a perfect 10.
Kesten, a partner with the Boston firm Brody Hardoon Perkins & Kesten and a well-known lawyer in Massachusetts, writes that he was surprised to find that his own Avvo rating was just 6.5. By providing some basic information about himself, he was able to increase it to 8.2. That whet his curiosity and led him to embark on his experiment.
"I selected Gregor Pagnini -- a terrific young lawyer at our firm who is three years out of law school -- and decided to see what I could do to increase his marketability on Avvo," Kesten recounts. "I had no problem impersonating him on the site." Once having "claimed" Pagnini's profile, Kesten proceeded to embellish his educational credentials, falsely listing him as a graduate of Oxford and Yale Law School.
Just as falsely, he identified Pagnini as a Lawyer of the Year in 2009, an Up & Coming Lawyer in 2010, and secretary of the Minnesota chapter of the American Association for Justice. "Surprisingly, even though these recognitions were not reported in the publications that supposedly issued them, and one was listed as occurring in the future, Avvo accepted the representations without question," Kesten said.
Kesten's last step was to add his own endorsement of Pagnini's work. Only he decided to make his endorsement negative. "No problem," Kasten wrote. "His ranking went up. That's right -- up. A few more negative endorsements later, he became a 10." With this newly acquired intelligence about Avvo, Kesten revisited his own ranking and "found it easy to get to a 10."
So is Avvo's lawyer-ranking algorithm as easily manipulated as Kesten contends? We put the question to Mark S. Britton, Avvo's founder and CEO. He responds that the article is poorly researched and full of misinformation. "The Avvo Rating is working as designed," Britton said. "Kesten is just using it inappropriately." Here is what Britton says:
At its core, the Avvo Rating is a resume-scoring system. We look at the information that we both find about and receive from a lawyer, and we score it. This replicates the process that you and I do (or at least I used to do) hundreds of times a year for ourselves and our friends and family. However, the Avvo Rating delivers such rating in a much more sophisticated and broad fashion than you or I ever could.
Imagine you are doing your homework on opposing counsel for an upcoming case. You visit her website to understand what her background/resume looks like. You are impressed by what you see, and you decide to take her seriously. The gotcha is that opposing counsel has lied about her professional credentials. Following Kesten’s logic, you are a poorly designed system that adds little value to this essential task in your client’s case. He might even write an article attempting to make you look bad without calling you for comment.
In the end, Kesten’s lying is not "gaming," it’s fraud. Whether it is Bernie Madoff, Milli Vanilli, David Edmondson or OJ Simpson, many fraudsters have shown that they can dupe the system -- at least for a period of time. And that is the key -- the "period of time." Any important system must have safeguards in place designed to detect fraud, and Avvo does. We investigate every report of padded resumes (and believe me we get many), and we also have a data team led by two experienced lawyers that researches every piece of unrecognized lawyer-generated resume data to ensure that it is scored appropriately. It’s expensive, but its worth it for the long-term health of our marketplace.
One postscript to all this: Kesten's law firm's Web site no longer lists Pagnini as a member of the firm. I'd wonder whether that means he took his perfect 10 and went elsewhere, except that his perfect 10 is also no more. Avvo now rates Pagnini a 6.8.
Posted by Robert J. Ambrogi on October 19, 2009 at 01:00 PM | Permalink
| Comments (12)