IT Research Firms Get the Smackdown from the Wall Street Journal
Today, Lee Gomes' Portal column in the Wall Street Journal today entitled, "Vendors Still Paying For IT Research That Flatters Them" (subscription required) delivered a serious bodyblow to technology consulting and research outfit, The Aberdeen Group. Gomes' essential point was that companies like Aberdeen (and I'd say all other big IT and Finance research firms) are far from unbiased and actually quite conflicted in the work they do.
Let's first understand Gomes' arguments. He writes about the old Aberdeen "which was for the most part a "pay-for-praise" operation. If you saw an Aberdeen report saying that Acme MicroMacro sold world-class solutions, you could be sure that Acme had written Aberdeen a world-class check."
He then goes on to discuss how Aberdeen has repositioned itself by ending this practice and now doing "sponsored research." He writes, "The new Aberdeen is better than the old one. How much better, though, is the question, because the new reports also seem conspicuously flattering."
Basically, sponsored research lets firms pay ~$30,000 to Aberdeen who then conduct surveys with tech users. The sponsors Gomes talked to said they like to sponsor a report because it provides a "chance to rise above the noise of the marketplace by being associated with something customers consider "research"." Even sponsors know the research is of dubious value, but it gives them an air of credibility.
He goes onto describe the conflicts which are clear. Aberdeen did 212 reports last year with 4-5 sponsors each. He writes, "But if much of your top line is dependent on getting tech companies to sponsor your research reports, you've got quite an incentive to design questionnaires that will yield the kind of reports tech vendors will want to sponsor."
"In that regard," Gomes writes, "Aberdeen delivers. The reports seem to invariably discover that "best in class" companies use, or are thinking about using, or somehow embody, whatever technology the report happens to be discussing."
I hope more people will take heed of what Gomes is writing. I've written about it before in my blog as there are too many inherent conflicts that such organizations have and so their advice is far from fair and balanced. But when someone brings it up in the Wall Street Journal, it will get attention which it deserves. I'm sure the PR machines of the big research firms are going into overdrive to demonstrate how unbiased they are, but there is no way to get around these massive conflicts. Beyond research sponsorship, other conflicts include:
- Conferences are hosted with technology vendors sponsoring booths, dinners, etc. In many instances, it is these vendors who speak at these conferences. Why pay to goto a conference to hear a salesguy drone on when you could call him and he'll come to you?
- There are entire magazines which claim to provide objective advice and "best practices" to readers but whose article authors are consultants and vendors pushing concepts that they "happen" to have expertise in.
As I have argued previously in my blog, (see one posting) you have to remain skeptical because objective research on the myriad self-anointed "best practices" is often not objective and the practices they're advocating are often far from best. So before you use a report to help make the case to buy a piece of software or undertake an initiative to mirror some "best practice" within your organization, be mindful of the source.


