Michael Rasmussen (blog, bio, Twitter) is a former Forrester analyst now with his own boutique firm, Corporate Integrity. In his recent blog post The Forrester GRC ‘Ripple’ (OOOPS . . . I Mean, ‘Wave’) Michael calmly dissects the Wave methodology and makes several suggestions for improving it. It is well worth the read.
However, this SageCircle blog post is actually in response to a Twitter direct message Carter received about Rasmussen’s post: “while part of me admires Rasmussen for offering critique, why didn’t he do so while AT Forrester? Hints at sour grapes.”
Probable Answer: It was only after he left the firm could he see the problem in the methodology
Major firms are constantly tweaking their methodologies with input from the analysts. But that is typically done around the edges with the goal of increasing the efficiency, fixing minor problems, or silencing vendor complaints. Frankly, it is the rare analyst at a major firm who takes the time to do an in-depth analysis of her firm’s research methodology to see where it is really broken. This lack of observations occurs for a variety of reasons:
- Analysts are too busy with day-to-day activities
- They drank the kool-aid that what the firm does is perfect
- “If it ain’t broken, don’t fix it” attitude toward things that appear to be working
So Rasmussen should not be criticized for not criticizing the Wave methodology when he was an employee of Forrester. Rather than think of this post as “sour grapes,” it is much more likely that he did not have an “ah, ha!” moment until he was on the outside looking in. We all know that hindsight is 20-20.
- Research consumers and AR professionals should develop a deep understanding of the methodologies used by their key analysts
- Research consumers should press analysts for detail on data and methodology underlying the analysts’ recommendations
- AR should understand the weaknesses of the analysts methodologies so as to take reasonable countermeasures to limit the damage of analysts working with incomplete information or insights
Bottom Line: It is likely futile for research clients or vendor AR to try to convince analysts about the weaknesses of their methodologies. What is more important is to understand the methodologies so as to ask the right questions and provide the right ideas.
Question: Research consumers – Have you tried to ask questions of your analysts about their methodology and data only to be stonewalled?