Analystanalyst’s Weblog

Who analyzes the analysts? – analystanalyst.com

Let’s collect examples

Anyone got any examples of ‘predictions’ by IT analysts that ‘didn’t quite hit the mark’?

Anyone seen some real massive faux-pas where someone clearly doesn’t speak to people in the real world?

Got any good starts for my ‘research’ in this area?

Let’s hear ’em…

Advertisements

April 24, 2008 - Posted by | Uncategorized

5 Comments »

  1. The list is endless. I remember being in a Forrester briefing where they predicted a large multinational ISP would go bust.
    The point is though that analysts often get it wrong but they can also provide real insight into market trends, contribute on messaging and positioning and help on procurement. It’s not all about getting in the top right corner of an MQ

    Comment by Jonny Bentwood | April 24, 2008 | Reply

  2. I agree it’s impossible for Analysts to always get it right although one has a strong agument that the amount of money I’m paying for the data they should be.

    My main issue is that I have serious doubts the info provided being written by someone who is a true subject matter expert on the technology/sector they are writing about.

    As a customer I’m placing a huge amount of faith (and money) in information that may not have been written, or more importantly validated/verified, by a true expert in the field. This effectively makes the info considerably less valuable to my organization.

    Finally, if it’ not all about being in the top right of the MQ, why are so may companies obsessed with technology vendors being in it?

    Comment by sha512 | April 24, 2008 | Reply

  3. It’s not ALL” about being in the top right”. But simply put, it’s a desired position because analysts and the end customers that place a huge emphasis on it have driven this to be the case. Vendors must have the wherewithall and strength to focus on driving customer value instead of their placement in the quadrant. When value is delivered to the customer, favorable placement in the proper ‘quadrant’ will follow.

    ***

    Sha512 – I recommend you use these tools as one data point if you wish, but as with any evaluation criteria, do not put all your money on one thing.

    Others – if you have been let down by these graphs in the past, give them another chance. if you fail to, you are unnecessarily narrowing the amount of information in your evaluation.

    ***

    The Gartner MQ, Forrester Waves, and other ‘summaries’ should be used as a data point. The fact is the analysts do not intend for buyers to always choose the vendor in the upper right corner every time. That would be like saying the best product always wins.

    We all know the best product, and similarly the ‘leader’ does not always win the business. There are other intangible factors that must go into choosing a vendor for any solution. If the leader happens to be a vendor you had a horrible experience with in the past, are you inherently choosing that vendor, no, not necessarily. If the vendor in the bottom left is one you have a trusted relationship with, you may choose that one even though the analyst seems to be steering you differently.

    Having been a vendor of multiple solutions from multiple companies, and having shown up in every quadrant and in every variation of other ‘market graphs’, placement is sometimes spot on, and other times it is too favorable or off the mark in a negative way.

    The funny thing is, it’s falsely presumed that showing up in the bottom left is a bad thing. My thought is this: If a vendor is in the bottom left, and it is an accurate representation, it’s a good thing for both the end buyer and the vendor. The end buyer most likely will not seriously consider that vendor. AND, the vendor will not have to waste time trying to win that selection when it will undoubtedly be a long, arduous sales cycle, with a potentially negative outcome in terms of customer support moving forward. Vendors should be happy to be selected out of an opportunity if it is one they are truly not meant for. When they want to pursue said market, they can work to prove themselves as a challenger or visionary.

    regards,
    dbmrktprof

    Comment by dbmrktprof | April 24, 2008 | Reply

  4. Interesting premise for a website. I’ll put a plug in for you on the SageCircle blog.

    A couple of quick comments:

    1) Most analyst research is prone to being misused because the firms do not have any systematic approach to training their clients. That is why we wrote a series of “Consumers’ Guide to Analyst Research” SageNotes for vendors to give to customers and prospects. We also posted “IT managers, it’s never, ever only about the upper right dot when it comes to Forrester Waves or Gartner Magic Quadrants” (http://tinyurl.com/37d3fa) with the same goal in mind. So in some situations, it is not that the commentary is wrong, it is just that the client does not know how to use it properly.

    2) To really do this right, it will require domain expertise, access to the full research paper and serious work to analyze analyst predictions. It is not enough to say the analyst got something wrong, there has to be analysis backed with data on exactly how the prediction or research was wrong. Too often folks leaving comments say something like “Gartner sucks,” which is somewhat less than useful. One example of someone who did a good job was theARpro as a comment on the ARmadgeddon post “Reviewing Gartner’s Top Predictions – who will step up?” It was a nice analysis of the prediction (reprinted in whole) backed with specific data points.

    http://armadgeddon.blogspot.com/2006/12/reviewing-gartners-top-predictions-who.html

    Comment by sagecircle | April 24, 2008 | Reply

  5. Sagecircle – thanks!!! (your comment got held up in spam filtering, sorry…)

    I love the piece on magic quadrants, however the ARmadgeddon piece turned into a rant against Gartner and died which is what I desperately want to avoid, and have been helped to do so by other in recent posts.

    Comment by analystanalyst | April 28, 2008 | Reply


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: