Preface: “In 2009 we cited this article as one of the most useful articles by analysts with tips on Analyst Relations. Over the decade since then, the article’s fallen offline. In 2020, when we started the research for our book, we asked the author’s permission to cite it. To help readers see the analyst’s comments in context, we reprint it below. “ Alan Pelz-Sharpe is now senior fellow at Altimeter.”
Typically CMS Watch only gives advice to technology buyers, but every now and then we have some counsel for the vendor community as a whole. This is one of those times. I think vendors in general, and vendor analyst relations (AR) specialists in particular should avoid common pitfalls that get in the way of communicating their capabilities and strategies to analysts.
I believe some AR professionals — and their product management and marketing colleagues who work with industry analysts — simply don’t know how to deal with firms like CMS Watch. Inevitably this leads to conflict. Rather than continually pull out swords and shields, I thought I would jot down some thoughts to help the process
The Analyst As Evaluator
I have been an analyst and commentator for 10 years now. For most of that time I have written or contributed to detailed, critical evaluations of software technologies. My topic areas revolve around content technologies, ranging from ECM and Document Management to E-mail Archiving. My audience is almost exclusively buyers and implementers of these technologies, whose typical project sizes run into the high hundreds of thousands to the multiple millions. People read my research to create shortlists, and to give themselves a better chance of selecting the right product. It’s a simple model really, much like a “Consumer Reports” or “Which Guide.”
I play with the technologies, I see them in action, I talk to many users, I talk to channel partners, resellers, and also consultants and integrators. I also talk to the vendors, but more for fact checking, product demos, and gaining insights on nitpicky elements than anything else. I appreciate the help of vendors, but ultimately my research focuses on how products work and are sold in the real world, so the world of vendor marketing remains of peripheral interest.
Providing customers with honest and critical evaluations of products means highlighting all the warts, along with spotlighting all the shiny positives. If anything my job is to focus on finding the warts. Let’s be honest, it is not hard for a buyer to find the positives. They’re deluged by “white papers,” marketing collateral, and sales spin. Finding where a product’s sweet spot or drawbacks reside is much harder.
It’s my job to help buyers in that process, and if by definition that makes me unpopular at times among vendors, it still behooves AR people to work openly and professionally. Here are some specific ideas along those lines.
Twelve Do’s and Don’ts
1. Don’t assume the analyst is out to get you
You are not as important as you may think. The analyst is writing about many vendors, so you are just one in a long list. You almost certainly have no context to judge their review of your product, in light of what they have said about your competitors. You may wish to consider slowing down before jumping to bias conclusions. In my most recent report, the AR group that had the biggest and nastiest hissy fit was the vendor that has received one of the more positive reviews in the report.
2. Your reputation precedes you
The aforementioned vendor was also the vendor that had the biggest hissy fit last time they were reviewed (different product, different report, different analyst, same comprehensive treatment). They are also the vendor that analysts from rival firms share AR horror stories about. The firm has decent enough technology, but a terrible reputation for bullying or attempting to coerce analysts. That might work with some analysts, some of the time, but not all analysts, all of the time. Perhaps even more importantly, analysts learn over time which vendors and vendor staffpeople can be trusted to tell the truth (the whole truth, and nothing but the truth) — and which ones can’t.
3. The technology customer is king
If the analyst’s methodology is focused on talking to customers and partners and you have been asked to supply customer references, respond in one of two ways: politely but immediately decline, or do your best to provide references. Ignoring the request for weeks or months is not a good policy and will only arouse suspicions rather than put the analyst off the scent. By that time customers and partners have been found by the analyst and interviewed. When critical views are captured from such interviews you should not at the last minute claim “our customers love x or y or z!” We know they don’t and frankly you haven’t been able to supply any that do. Harsh as it sounds, we are not just going to take your word for it.
4. Don’t threaten analysts
If you don’t like what an analyst has written, try at least to be respectful and polite. You are far more likely to enter a dialogue that way. Provide real facts to counter their critical assertions; if you cannot provide facts and instead rely on bluster you will only dig a deeper hole for yourself. Also remember that analysts are human, threats via nasty e-mails (the coward’s way) or phone calls can backfire, and they don’t get forgotten quickly. Using such a confrontational approach does not make the AR person look important or even imperious, it makes you look unprofessional.
5. Don’t quote your own press releases or other analyst reports as evidence
There is frankly nothing sillier than to tell an analyst that they must be wrong about your firm/product because “Forrester/Gartner/IDC…ranks us as a ‘leader,’ etc.” The only thing that rivals that is to quote from your own press releases. Trust me, this happens all too often. Most of the time, this kind of response will simply result in an internal e-mail chain that shares the joke with other analysts. Bottom line: this kind of supporting evidence looks desperate, patronizes the analyst, and suggests you have simply drunk too much of your own kool aid.
6. Never say “we provided an X% ROI to our client in less than six months etc etc.”
It’s a silly thing to say, period, and a particularly daft thing to boast to buyers. It’s a little like Home Depot claiming that they dug my vegetable garden for me, when all they did was sell me a spade. You provide tools, people use the tools, and using those tools provides business benefits (or doesn’t). And just like the spade I bought from Home Depot, remember that we all know most software goes likewise goes unused.
7. Don’t kiss my ass
Flattery will get you nowhere. Neither will generous offers of fancy meals, drinks, flight tickets, five-star hotels, (none of which CMS Watch analysts can accept anyway), or the other perks that are stock-in-trade of influencing influencers. If you respect my work, do us both the favor of playing it straight.
8. Don’t ask me for advice
This one sticks in the craw of some vendors. They’ll say: “but I spent all this time with you and gave you references, and you won’t return any feedback?” Here’s the deal: advice to a vendor from an analyst that advises buyers constitutes a conflict of interest. Any advisor to you will become invested in your success, which is good for you, but bad for the analyst’s credibility and future objectivity. Just as importantly, it’s not fair to other vendors. We are observers and critics, not market-makers.
9. A demo should actually demonstrate something
AR people love to talk about their software, but are sometimes reticent to actually show it, or reluctant to show it under conditions reflecting something close to a normal enterprise environment. I think the revolving door between vendor product managers and industry analysts has helped to legitimize a culture of “briefings” completely via PowerPoint. Static slides can certainly identify the ever-beloved product strategy, but will fail utterly to convey the real experience of using an interactive application. And as we get into the demo and I ask you to show certain things and answer deeper technical questions, then…
10. Make sure you understand how your product works
Or include someone in the conversation who does. A dirty secret in our industry is that many product and marketing managers are mistaken about how their product actually functions. Or they swear that a feature they spec’ed in actually made it through a rushed development and testing period…when in fact it was left on the engineering floor without anyone ‘fessing up. Some of our more interesting moments as analysts have been pointing out to irate product managers that their own documentation or demo contradicted their marketing claims. All the more reason to…
11. Understand the difference between a fact and an opinion
For every ten vendors I evaluate there will be two that freak out. With the other eight we agree to disagree, and where there are errors (I make many, and do my best to fix them) we work together to get them corrected. I never want my reports to contain factual errors, nor do you. But my opinions are my opinions, I am paid to have opinions. To change my opinion requires a very different approach from AR. To change my opinion you need to understand why I have formed that opinion (see below) before attempting to “re-educate” me. In addition, when you claim a report is full of factual inaccuracies, and then send an annotated Word document listing differences of opinions — and can quote no factual errors at all — expect your response to be ignored.
12. Understand that customers implementing your systems have a very different perspective to share
Just as I will view your product or service differently than you, recognize that a channel partner, a user, an implementer, or a consultant will all have differing perspectives. We base our evaluations primarily on what implementers experience. When a report does not reflect your personal or corporately-mandated vision, that does not mean it is wrong. Remember, if the only research you have read is from people you directly or indirectly pay, then it won’t be surprising if you find some kind of uniformity with your own viewpoint. True outside opinions will by definition differ from your own.
Extra Credit: Don’t believe your own hype
We know it’s your job to be passionate about your company, about its product, and its services. We understand it’s your job to help sell this vision and to educate us all. But make the effort to really understand your competitive landscape too. Don’t live in a vacuum. Analysts don’t. I applaud your enthusiasm, and I wish you and your colleagues the best of luck, but I wish all your competitors the same too. I am not passionate about your company, I am passionate about ensuring that buyers and users avoid costly and sometimes disastrous mistakes. I want them to pick the right product each time, and use it to best advantage. To the extent that’s your goal, then we have the same agenda.
Analyst Relations is hard
Some of this advice is clearly specific to dealing with CMS Watch. Granted we may have an unusual research model. Having been in the industry for 10 years and having run research practices and undertaken extensive competitive intelligence, I am well aware that the vast majority of “traditional” analyst firms are dependent on vendor funding of one form or another to pay the bills, and that clearly flavors their dealings with and reporting on vendors. It’s the way it is — whether I like it or not.
I also understand that AR professionals have a very tough job to do. Frankly I do not envy your role. You have to try to keep everyone happy all the time, and that is an impossibility. I have deep admiration for many AR professionals. I have deep admiration for any technology vendor who stays in business more than six months. I suspect, though, that the vendors who really thrive in the coming years will be those that figure out how to relate to the marketplace — not just analysts — in a more transparent way.
{A previous version of this article was first published by the Institute of Industry Analyst Relations.]
