Info

GO BIG ALWAYS

We just signed up to be clients for both these analyst firms in January. It’s the first semester and the experience couldn’t be more different. Granted, we’ve just begun relationships with both firms, so things are bound to change. report-card.jpg

Before we became clients

Gartner

  • Recognized our space via Magic Quadrant last October. That said, it was filled everything under the sun. Apple, oranges, fruit cake.
  • A cool analyst in their Media division proactively called us to learn more. It would have been nice if someone from the “High Performance Workplace” (the collaboration-focused group) had done that.
  • We were hounded by their sales rep to become clients.

Grade: B-
Forrester

  • A number of analysts contacted us to learn more (Josh Bernoff, Charlene Li, Rob Koplowitz)
  • Kept assuming we were clients and it didn’t seem to matter to much to them that we weren’t. Ultimately, I had to ask them to become one.
  • Analysts freely shared insight across the blogosophere. Specifically, Charlene Li (though she seems to have stopped), Jeremiah Owyang, and Josh Bernoff.
  • Forrester hadn’t done a “Wave” (equivalent to Gartner’s Magic Quadrant) on our space. They had, however, produced many visible reports and contribution to both the Online Community and Enterprise Collaboration spaces.

Grade: B+

After we became clients

Gartner

  • Jive folks haven’t requested many Gartner reports. I can only assume that means there’s not much specific relevance yet.
  • Our account manager has disappeared. We’ve been trying to schedule briefings since January. We’ve emailed a ton and filled out online forms to no avail.
  • My longest discussion with them was an hour orientation on how to use their website (which actually needs an hour orientation.)
  • I invited the analysts to a customer advisory board meeting. That’s a meeting with VPs at our top Fortune 100 clients who share exactly how they’re using our software, what works and what doesn’t. They told me they could come but it would cost me tens of thousands of dollars.
  • They haven’t posted a blog in over a year
  • Value so far: Unclear.

Grade: D
Forrester

  • High demand from Jivers for their reports. They’re relevant and topical.
  • Their account manager proactively sends me reports they think we’d like.
  • I invited them to the same customer advisory board meeting. Rob and Jeremiah made it. That said, they stayed for their 45 min presentation but didn’t seem curious to attend more to hear what our customers had to say. It’s too bad, our customers had unbelievably cool and insightful things to share.
  • Where’s the Forrester Wave on our market?
  • Value so far: The online community reports have been valuable. The Analyst availability has been good (Rob Koplowitz made himself available for a video), Jeremiah is a content and reference animal. The man is constantly in the conversation online.

Grade: B-

Overall Grades

Gartner C-

Forrester B

The smaller guys

Probably not too dissimilar from us, the smaller pure-play firms are more engaged and accessible. Analysts who really do seem to want to learn and produce great stuff:

Questions

  • Why, when presented with the opportunity to listen to huge companies actually using social software, would analysts not fall over themselves to listen to them?
  • Has anyone else experienced a report card like this?
  • Do enterprise companies still really use these reports to make buying decisions?
  • Will insightful and credible people from Blogosphere make analysts irrelevant?
  • http://blogs.forrester.com/groundswell Josh Bernoff

    Hey, Sam, I’m interested in big companies using social software. I will fall all over myself to listen to them. Send me the contacts!

    Do enterprise companies still really use these reports to make buying decisions? I can’t speak for all, but I sure talk to a lot of them about it (and now that Jeremiah is our community expert, he’ll be talking to lots more).

    As for your last question, I’m all for the wisdom of crowds and undifferentiated opinions. But when it comes to buying stuff that affects your company’s future and image, I doubt we’ll become obsolete. People like to talk to someone whose full time job is keeping on top of this stuff.

    P.S. I wonder how long you’ll wait before Gartner comments on this post?

  • http://www.ddmcd.com Dennis McDonald

    I think a lot depends on how you use the service. It’s nice to have a good relationship with a rep who understands your business, but ultimately you have to take responsibility for figuring out how your interests map to their capabilities, then pursue those interests by establishing ongoing relationships with the key industry or technology experts inside whose insight/opinions/thinking you respect.

    If you have to wait in line to schedule a one hour conference call with an expert who is shielded from establishing ongoing relationships based on your subscription level, you’ll never get the value you need, nor will you be able to effectively influence the source’s research directions.

  • http://thepaisano.wordpress.com paisano

    Fantastic idea! A report card for analysts. I wish more would do this because you know it would change the game if they knew we were all going to share reviews of their performance (or lack there of) in a public manner! It’s like ebay ratings. Nice! Totally agree about the perplexing habit of “leaving money on the table” by not listening to the actual users of software or a client’s services! It’s really mind-numbingly stupid if you ask me!
    Thanks for always thinking outside the box, which is just as important as thinking big!

    pai

  • http://blogs.the451group.com/information_management/ Kathleen Reidy

    Hi Sam, I don’t recall an invite to the customer advisory board meeting…but that’s ok ;) Maybe next time?

  • http://www.sagecircle.com Dave Eckert

    Great post – very insightful. It is too bad that more clients do not provide this sort of feedback to their analyst firms. It would improve the overall community.

    Obviously you were focused on the social media aspects, but all clients need to address the analyst firm responsiveness related to their markets and the way the those markets are addressed.

    Dave

  • Pingback: Doing unto analysts what they do unto vendors « SageCircle Blog

  • http://Website Dennis Howlett

    Analysts represent one of many data points in the decision making process so treat their effectiveness with care. Gartner is still the big elephant in the room but it has to be said they are much more focused on transaction based systems. As Sam knows, there is value in bringing others to the table and I had the privilege of attending the meeting to which he refers.

    What it does for people like myself is allow us to parse what we hear the vendors saying against what customers are demanding in an environment where we can surface that information back out to the wider body of readers. It is therefore much more in the interpretive tradition which folk like myself believe is where the real action lays. Not just on decisions but also on understanding for the future.

  • http://sagecircle.wordpress.com/2008/03/18/doing-unto-analysts-what-they-do-unto-vendors/ Carter Lusher

    Kudos on the post. There is now an article on SageCircle’s blog commenting on the implications of your report card.

    It is interesting to see how savvy analysts like Josh and AMR’s Jonathan Yarmis responded to you so quickly. I wonder how many more calls you will get requesting a briefing? I hope you have hired that AR manager to do all the work setting up the briefings. ;->

    http://sagecircle.wordpress.com/2008/03/18/doing-unto-analysts-what-they-do-unto-vendors/

  • http://sagecircle.wordpress.com/2008/03/18/doing-unto-analysts-what-they-do-unto-vendors/ Carter Lusher

    BTW, have you communicated your client/sales service issues to the Gartner Ombudsman? I bet they would like to help.

    ombudsman@gartner.com

  • http://comingsoon Oliver Marks

    - a fascinating and informative post from a vendor perspective! In the past I experienced having my brains picked by analysts in the parallel business universe of ‘blended learning’ and was dismayed to find months later that I would have to buy the results of our conversations…seperately there’s also an irony in paying for analyst reports on collaboration after you’ve collaborated with them to create those reports…

    I’m currently seeing this from the inside of a vast corporation (although not for much longer) and did see the Gartner magic quadrant last year. Those things are sometimes good for those vendors sitting in the quadrant cross hairs when you are looking for budget money, but as Sam succinctly states,’apple, oranges, fruit cake…’. There are so many facets to this space that many reports, quadrants etc can be dangerously misleading in their generality…and confusing when you are crafting solutions to specific problems.

    I’m being a little hard on analysts here, but I do think the spirit in which we freely share information, particularly as in this blog and others, is often cynically profiteered by large analyst firms. Surely the objective is to all profit from our shared knowledge…

  • http://www.forrester.com/rb/search/results.jsp?N=0+11646 Oliver Young

    Hey Sam, Thanks so much for the feedback. I actually just moved to the Bay Area (last week) specifically so I could spend more time attending customer meetings, participating in vendor events, and generally spending more face time with vendors like Jive. You can be sure I will attend the next customer advisory session and, for the record, I always stick around to hear what others have to say (just ask the Atlassian guys)!

  • http://web-strategist.com/blog Jeremiah Owyang

    Wow Sam

    Thank you for the feedback. I fully agree that a client (regardless of who they a customer of) should give feedback about their experience. This is the promise of social media (providing the company improves)

    I promise to do whatever I can to improve, and thank you for the feedback.

    Sorry, I couldn’t stay at the event, but some of the folks in the room are my clients, and I already am quite aware of what they are working on (and what they think of your product too)

    To Carter’s comment, in no way does this feedback impact our objective outlook

  • http://www.biztechtalk.com/2008/03/shh-secret-webi.html Dan Keldsen

    Perhaps the need to have paid, ongoing relationships, is part of the problem?

    What’s the percentage of the time/credits that subscribers/clients actually USE versus what they’re paying for?

    That and disconnects between drive-by selling and what is actually on the analyst’s editorial calendars are some long-standing issues. Very common in enterprise software, as well as in the analyst world.

    BTW – would’ve been interested in the customer advisory board meeting as well. With enough warning, I could be persuaded to visit Oregon, it’s been almost 20 years!

    And setting up an analyst briefing with our Market Intelligence unit is pretty simple. E-mail, IM, twitter, Facebook message me or call me… :)

    FYI – Our research on Enterprise 2.0 will be released next week, incidentally. Fully public distribution, and a companion free webinar discussing the webinar, happening on Thursday, March 27th. Approaching 400 people pre-registered for the webinar. Should be fun!

  • http://Website Rob Koplowitz

    Good stuff. Thanks for the honest feedback Sam. Was I invited to stay? : )

    BTW, always grateful for the excellent access Jive provides to their customers. It’s been invaluable to my research.

  • Pingback: Gartner & Forrester’s report card « The Whole Nine Yards

  • http://www.mzinga.com/aaron Aaron Strout

    Sam – looks like you’ve struck a nerve with the public on this one. Kudos to the Forrester gang for chiming in (Josh in particular who commented within minutes of your posting.)

    It’s interesting that nobody from Gartner has weighed in yet. Commenting on your post would be a great opportunity for them to earn a little bit of street credibility.

    Keep it coming!

    Best,
    Aaron (@astrout)

  • Pingback: People Over Process » links for 2008-03-19

  • Pingback: People Over Process » links for 2008-03-19

  • Pingback: The Benefits of Blogs: Feedback (Why Sam is sending me back to school)

  • http://www.web-strategist.com/blog/2008/03/19/the-benefits-of-blogs-feedback-why-sam-is-sending-me-back-to-school/ Jeremiah Owyang

    Sam

    Yesterday I was a little caught off guard, I first saw your post on Twitter while in the back of a NYC taxi cab (yeah, I’m that connected) On the plane, I reviewed what your report card, and spent some time thinking about your post.

    I’ve responded from my blog at

    http://www.web-strategist.com/blog/2008/03/19/the-benefits-of-blogs-feedback-why-sam-is-sending-me-back-to-school/

  • http://www.burtongroup.com Mike Gotta

    Sam, just a quick note to thank you for the acknowledgement. I try to be as open as possible (there are situations when confidentiality and business model constraints do limit things). Burton Group has been very open-minded when it comes to personal analyst blogs as complimentary to our group blogs.

    I agree that transparency is a good thing for our industry. Analysts should be engaged in broader community conversations – it benefits all involved.

  • http://www.gobigalways.com/ sam

    First off, I’m pretty excited that this has been so positively received. It one thing to have so many folks supporting the open dialogue online (this post, twitter, other people’s posts) but quite another to also hear it from Gartner and Forrester via phone.

    As I mentioned on Stowe’s post, It’s going to be interesting to see how the Analyst market evolves faced with social software and the changing landscape of expertise. Personally, I think they have a massive opportunity to engage in a wholly new way and like many companies, some will embrace and others won’t.

    The truth is, most companies don’t track social software nearly as much as our echo-chamber does and they will put value on *people* who are credible analysts to provide further insight. Who those people are and how they plan to deliver it is what seems to be in flux.

    Analysts are freaking smart. Many have insight the rest of us, don’t. They get to talk to vendors, customers, media, and gain perspective that only those relationships (along with data) could provide.

    The trick is, how to open up, include and fully engage with others. This whole thing is about relationships now–typical and atypical. They’re no idiots, creme rises to the top.

  • Pingback: Between the Lines mobile edition

  • http://www.conversationjones.com Richard Telofski

    “Will insightful and credible people from Blogosphere make analysts irrelevant?”

    Theoretically possible, I suppose. But it would require client brains akin to the best sponges out there, and clients with unlimited amounts of time.

    Analysts provide the knowledge absorption and capacity of hours, allowing clients to actually run their businesses.

  • http://www.gartner.com Ray Valdes

    Hey Sam, very interesting post.

    A small point of information: Gartner first covered Jive Software back in August 2006, mentioned in my overview report on corporate blogging technology, now dated but still available at http://www.gartner.com/DisplayDocument?id=495227

    Can’t speak for others, but speaking for myself, I am always interested in talking to customers and end-users about their use of social software. I probably have about five hundred such conversations in the course of a year, mostly through the inquiry process but also through other means such as formal one-on-one sessions and informal conversations at vendor/industry conferences.

    There are now other colleagues at Gartner who are spending more time than me covering the category of enterprise social software. However, I am still interested in hearing about your company, its products and relevant customer experiences. Let’s keep the conversation going.

  • http://www.gobigalways.com/ sam

    Kudos to you, Ray, for reaching out. That’s very cool.

    I should be clear that as it applies to you guys, it’s not about the analysts. You guys are super smart and are doing your thing.

    Most of what seems to be broken is the processes and fatty tissue on the relationship side. All that said, this is just a point in time. I spoke to your Sales reps today and I think they understand the issues, so hopefully productive stuff will come out of it.

  • Pingback: Grading the analysts - The Network Hub: A SearchNetworking.com blog

  • Pingback: Gotta Conversation Jones?

  • http://www.ComHacker.org Trisha

    Jeremiah has really made Forrester research a household name. Heck – HE is a household name (at least so long as someone in the house runs a small business or does anything online =P )

    I don’t believe enterprises use these reports, however. What are assistants for, if not to research? Maybe they buy some reports from time to time, but I doubt an annual membership.

    In all honesty, I use Aberdeen and my personal assistant to get at any information I need.

  • http://www.susanitsa.wordpress.com Susan Scrupski

    Hi Sam. It amazes me how much our F500/G2000 clients are still reliant on the traditional analyst house’s opinions and reports. Did I say amazes? Looking for a stronger word… It’s somewhat of a frustration for me. If you’ve not seen it, a great post/debate on old school analysts vs. the digerati analysts was posted last fall by the HP analyst relations guy (whom I’m told since moved on). The comments further the debate. See for yourself: http://tinyurl.com/3ddybw

    Kudos to you for grading the analysts and provoking some conversation on this topic.

  • Pingback: Why vendors mistake analysts’ primary task as being forecasting : Analyst Equity - Lighthouse’s action research blog

  • Pingback: The Social Enterprise » Le Ferrari delle Community Online

  • Pingback: Traditional Analysis Firms, It’s All About Dollars. But Are Blogs Heading the Same Way? at diversity.net.nz