10 Important Considerations When Selecting A Social Media Measurement Vendor

1 Jul

The majority of conversation today within the measurement community pertains to social media* measurement.  And the number of firms offering tracking, monitoring or social media measurement solutions is exploding, doubling or more from 2007 to 2008.  So, how do you select a firm that best meets your needs or the needs of your clients?  Here are ten items to consider as you start down that path. 

 

Underlying Philosophy & Capabilities – There are many different approaches and underlying philosophies driving the various social media measurement vendors.  Some of the approaches translate directly into unique capabilities.  For example, CoreX, a company focused on defensive monitoring, is driven by a belief that participating in a discussion may unintentionally influence its outcome.  Therefore your identity is hidden when you use the tool.  Compete tracks a consumer’s online behavior/clicks while most other vendors track consumer artifacts like comments or posts.   Kaava focuses on communities and not blogs and believes they better represent consumer opinion.  Pick a partner with a philosophy that is compatible with your own views and needs.

Metrics and Analytics – Give some thought to the specific metrics you want to track and the analytics that will give you the best data upon which to make decisions.  While most firms track basic metrics like volume of posts or comments and number of links, more complex or algorithmic metrics vary widely.  There are many different interpretations of Influence for example.  Also evaluate the range of analytics that are available. If links analysis or social network analysis is important to you, some social media vendors offer these while others do not. The good news is the analytics and visualization tools available are rapidly becoming more sophisticated and useful.

Content – It is important to understand how content is brought into the social media measurement site, and how many different sources are looked at in order to find relevant content.  Some sites claim 10 million sources while others claim 30 million.  You also want to ensure the sites trolled are the ‘right’ sites for you or your clients.  Develop a list of your 100 most important sites and ask the prospective vendor to verify whether or not each of the 100 is included in their universe of content.  Also note which categories of content are tracked – discussion groups, blogs, review sites, traditional online media outlets, etc.      

Language Support – Are you most interested in the United States, Europe or perhaps China?  How you answer may well lead you to consider an entirely different set of vendor alternatives.  If Europe is most important to you, you might consider a European specialist like London-based Attentio, or perhaps Onalytica.  If you are most interested in social media conversations in China, you might consider a vendor like CIC that offers a deep understanding of Chinese pop culture and works in Shanghainese as well as Mandarin and Cantonese.  Just offering language translation is vastly easier than understanding cultural nuances of language usage within a given country.  If your requirements are global, either make sure your analysis partner supports the languages that are important to you, or select two or three regional partners that collectively can support your global requirements.

Real-time Orientation – Do you need information for crisis or issues monitoring or perhaps investing (Collective Intellect)?  If so, you will need a vendor that provides near real-time updates of information.  The search orientation model offered by many vendors is more akin to batch processing.  You define terms you are interested in and then bots go and gather the information for your review and analysis.  If crisis or issues monitoring is your primary requirement, make sure your vendor also offers capabilities like email or phone alerts.    

SaaS or Consultative Models – Do you want a platform to do your own social media monitoring or do you want a firm to do the research and prepare a monthly report for you?  Hands-on or hands-off?  Some vendors (BuzzLogic, NetMap Analytics, Radian 6 and Visible Technologies to name a few), offer a software-as-a-service model while, at the other end of the spectrum, others (MotiveQuest) are very oriented toward consulting – helping to answer the “so what should we do about it” question.  Most of the firms offering a SaaS use a dashboard as their main interface.  If creating dashboards for reports and/or having a dashboard interface is important to you, make sure to ask whether or not this capability exists and have it demonstrated to you to assess the overall ease-of-use and utility.

Automated or Human Analysis – One of the ongoing controversies in the measurement field is what can be successfully automated in content analysis and what must or should be left to human analysis.  A few vendors, Umbria for example, are fully automated, even for sentiment analysis.  The majority of social media measurement vendors employ a hybrid approach, with simple items like post or comment counts and number of links fully automated, but sentiment analysis left to human analysis.  Some vendors perform the sentiment analysis while others leave the user to define sentiment for themselves.   

Ability to Directly Engage with Consumers – Some monitoring and analysis vendors allow you to directly engage in consumer discussion without leaving their platform.  Visible Technologies and Digital Influence Group are two examples.  This may be advantageous in situations where you are attempting to ‘Listen and Engage’, rather than just Measure online conversations.

Customizability – As you look at specific vendors and the tools and metrics they offer, you would be wise to ask whether or not any of the approaches, content sources, metrics or algorithms can be altered to better meet your needs.  For example, if specific social media sites are critical to you, make sure they are being included in the content population.  Filtering capability allows you to exclude sources you may not want to consider.  Some tools, like Radian 6 for example, allow the user to customize (via a graphic equalizer-like interface), the weighting given to specific elements when calculating Influence.  You can put more of less emphasis on elements like Number of On-topic Posts or Number of Links to better fit your own definition of online Influence.

Cost – Costs vary greatly, driven by many of the factors above.  Obviously human analysis and consulting are major cost drivers.  Tools range from free (always a strong price point!) to one to two thousand dollars a month, to $100,000 or more per year.  Decide what you can reasonably spend before you do too much vendor analysis.  For example, there is no sense in looking at vendors oriented toward consulting if you have only $1000 per month to spend.

 

There are many additional factors you may want to consider when comparing firms.  Number of years in business, clients/customers they do business with, and their ownership structure to name a few.  I hope you will find the ten considerations presented useful as you try to find the best social media analysis for you.  Happy hunting!

 

For more complete information to help you select the right social media measurement company, please visit Social Target.  Founder Nathan Gilliatt produces the industry’s best reference guide of social media analysis vendor information.  His reference guide has informed much of what you see above.  His 2008 reference guide will be available soon. You can order it here.

*I am using ‘social media’ broadly to include blogs, online discussion groups, forums, review sites, CGM sites and social networking sites

Disclaimer – My employer’s parent company, IPG, has a non-exclusive agreement with Radian 6.   

32 Responses to “10 Important Considerations When Selecting A Social Media Measurement Vendor”

  1. Richard McInnis July 1, 2008 at 10:19 pm #

    Hi Don,
    Great post, should be a valuable resource for those looking for a measurement vendor.

    It is important for people to also consider the breadth of coverage a vendor offers to deliver a full picture of the social web including not just blogs but; boards, images, video and microblogging.

    Thanks for the mention!

    Cheers,
    Richard – Radian6

  2. Don Bartholomew July 1, 2008 at 10:25 pm #

    Good point, Richard. I tried to address that thought with this: “Also note which categories of content are tracked – discussion groups, blogs, review sites, traditional online media outlets, etc.”, but you stated the breadth notion more clearly. Thanks, DB

  3. Blake Cahill July 1, 2008 at 10:43 pm #

    Don,

    Nice post and break down of functional areas for consideration/debate while looking at different vendors. Glad to see that you mention engagement. Additionally, in the SaaS area I think myself and the other vendors you mention all have reporting on-demand verses monthly. It think that Cymfony, Umbria, or BuzzMetrics deliver reporting more on a monthly or project based perspective.

    Blake
    VisibleTechnologies

  4. Nathan Gilliatt July 2, 2008 at 1:09 am #

    Don,

    Nice set of points. Are you looking for some writing work on the side? ;-) Thanks for the plug!

    The second edition will include a quick reference that goes into more detail on breadth of coverage, so that will be easy to sort out. Most of the companies cover most of the media types I asked about, which seems to be a change from last year.

    Nathan

  5. Don Bartholomew July 2, 2008 at 2:38 am #

    Nathan.
    Thanks for your kind words. I found the first edition to be excellent, and it sounds like the upcoming edition will be even better. I’m looking forward to it very much. Keep up the great work!
    -Don B

  6. Mike Spataro July 2, 2008 at 11:52 am #

    Don,

    Great industry overview and analysis Watch out, Forrester Research may come looking for you. One other key point to include in the content section (because we get asked it all the time) would be the collection and filtering of relevant comments (not just original posts) because that’s where so much of the valuable insights are contained. With the growth of community across so many sites, that will be an increasingly important part of the business.

  7. David Rabjohns, Founder July 2, 2008 at 1:37 pm #

    +1 nice article, very well written.

    I think you have nailed the essense of the key vendors MotiveQuest included.

    One more important consideration may be to ask what problem you are trying to solve? Are you just looking to supplement your broader data gathering needs? Are you trying to develop a pan European campaign? Are you trying to boost sales in Q4? Different questions also lead to different vendors.

  8. Don Bartholomew July 2, 2008 at 1:54 pm #

    Mike,
    Your point on the value of comments and not just posts is very well taken. After all, the commenters are almost always the real targets, not the posters. Therefore, on-topic comments are absolutely relevant and of great interest. I also believe that tracking re-commenters may be a more valid measure of community than number of unique commenters per se.
    -Don B

  9. Don Bartholomew July 2, 2008 at 1:59 pm #

    David,
    Could not agree more that different questions lead to consideration of a different vendor set. I glossed over this a bit (“Pick a partner with a philosophy that is compatible with your own views and needs.”)
    and probably should have emphasized the point more strongly.
    Thanks for your comment, Don B

  10. Nick Sowden July 2, 2008 at 5:52 pm #

    Don,

    Good outline. Thanks for including us. We’re really excited to see so many competitors doing good work; there is a big market here somewhere.

    Keep it up,
    Nick Sowden
    Collective Intellect

  11. Don Bartholomew July 2, 2008 at 6:45 pm #

    Thanks for your kind words, Nick. I believe the big market is just around the next corner :-)

    -Don B

  12. Katie Paine July 2, 2008 at 7:54 pm #

    Don, I’m hurt! I know Nathan Gilliatt left us out of his list because he didn’t know any better, but you’ve seen our dashboard and our work! Just because we’re not a pure technology company, doesn’t mean that we’re not in this space and don’t compete with Cymfony, Visible and the rest of them!
    KDPaine & Partners (www.kdpaine.com) has been monitoring and measuring consumer generated media longer than all these guys combined. We started with newsgroups in 1995! As far as I know we are the ONLY ones that are measuring Facebook as well as YouTube and Twitter AND combining it with traditional media AND providing correlations to outcomes. You saw what we did for Georgia Tech and STILL didn’t include us. What’s up with that???
    Or are you implying that factors like the ability to tie traditional and social media into a single dashboard and to correlate on that same dashboard your social media activity to desired outcomes aren’t important? Either way, I expected better of you.

  13. Don Bartholomew July 2, 2008 at 8:28 pm #

    Katie, Katie, Katie;

    Are you having a bad day? I hope your comment is just a little bit tongue in cheek or perhaps false indignation to better provide a platform for you to stump KDPaine & Partners (I’m OK with either of these by the way :-) ) My post was obviously not an attempt to name all the vendors in the space nor an attempt to highlight the best companies in the space. I believe Nathan’s first edition contained 32 companies. My post mentions 16 vendors that I thought were examples of the various points I was trying to make. Not necessarily the best examples/firms, but valid ones.

    To your last question, no, I was not trying to imply anything of the sort (but you knew that). You may also note that I also did not mention Cymfony (calling Jim Nail, calling Jim Nail)…

    – DB

  14. Kristin Wadge July 2, 2008 at 10:35 pm #

    You make so many good points, Don, I’m tempted to respond to each and every one of them. In the interest of brevity (and sanity), however, here are the highlights:

    We (www.metrica.net) completely agree with Katie that mainstream media and social media should be intrinsically linked — they are all just part of the media mix. We’ve developed all our products from the eyes of the consumer. Although all media have different degrees of influence, the average consumer doesn’t think, “ooh, now I’m reading a blog, now I’m reading the national press,” and neither should we. Our learnings over the last 15 years of media analysis mean that we’ve created social media products to avoid all the usual bear traps…

    All offerings should be completely customisable for every client. Some clients just want to know what it’s all about, some want to watch (in real time, usually) and some want to engage. The range of analytics is as broad as the ocean is deep. We wrap metrics snugly around our clients’ PR objectives. Just like MSM, we should fit our measures around what they are trying to achieve, rather than trying to jam them into a rigid system. The same concept still applies, it all starts with the question, “what are you trying to achieve?”

    We source, we accept other sources, we add insight, help our clients engage and output real-time information on our interactive dashboards. But all this whizzy stuff is meaningless unless you put the right analysis into the equation. It’s true that if you put rubbish in, you get rubbish out. So we support a hybrid solution of computers where they can’t be wrong (yes / no, black / white measures) and our sector-expert human analysts where we need the nuances of communication (namely sentiment analysis — messages, favourability etc). And international is key. Global language capabilities are essential.

    I think we’re all (PR professionals, media analysis experts and anyone else who wants to join the party) in for a very exciting couple of years. We do need to bear in mind though — just as the media analysis industry experienced all those years ago — this is just another media channel and there are no right answers or universal metrics. It all depends what the client is trying to achieve, and the winners will be those who focus the hardest on client needs and deliver the most effective and targeted solutions (and all at a proportionate and sustainable cost — we must not lose sight of the fact that the chaff to wheat ratio is far higher in CGM than in mainstream or indeed almost any other form of media).

    Not as brief as intended, but from the heart! Best, Kristin Wadge

  15. Katie Paine July 2, 2008 at 11:10 pm #

    Kristin’s points are very well taken, and no Don, I wasn’t having a bad day till I read your post! :) just out of curiosity, what would you advise your client to do if, as part of their measurement program, you came across an enGadget post that mentioned all the coolest new phones from Nokia, Motorola and never mentioned Samsung?

  16. Don Bartholomew July 3, 2008 at 12:30 am #

    Katie,
    The intent of the article was not to name all the vendors one should consider nor was it intended to rate or endorse vendors in any way, shape or form. The vendors were merely mentioned as examples of the points I was trying to make. It should not be construed that these are the only vendors for whom the point applies, not should one conclude the vendors mentioned are necessarily the best choices. If anything, it is more about vendor classification than vendor qualification. I could have made the same post without mentioning any vendors and not changed the core intent of the post – it just would have been less interesting.

    If you feel strongly about this, Katie, I will be happy to add a disclaimer to the post that says the above.

    I hope you know I believe you and your firm are at the very cutting edge of social media measurement.and I learn from you regularly. -DB

  17. Sam Flemming July 3, 2008 at 1:16 am #

    I am flattered to have my company mentioned among these other great companies in the space.

    I think David’s point is very important. What does the client want? Depending if you they come from a PR, market research, marketing or competitive intelligence standpoint, you will likely get very different answers.

    Also, we would have obviously strong opinions about the need for local knowledge which is not limited to language. Understanding the local market, local culture (inlcuding net culture), and how the local netizens use the Internet and social media are essential to provide true understanding of the words. we have plenty examples of this on my blog (www.seeisee.com/sam) and in our white papers (http://www.slideshare.net/CIC_China/slideshows).

  18. David Rabjohns, Founder, MotiveQuest July 3, 2008 at 7:41 pm #

    “As far as I know we are the ONLY ones that are measuring Facebook as well as YouTube and Twitter AND combining it with traditional media AND providing correlations to outcomes.”

    errrr we do that and I think others do also. Sorry to make you bad day worse.

  19. Thomas Aedy July 6, 2008 at 11:27 pm #

    Hi Don,

    I’m a student working on a project about social media. I was wondering if you could point me to any websites/links that might show me:

    1) Which kinds of social media are the most frequently used?

    2) How big social media is compared to other websites?

    3) Which social media measurement companies are the biggest (market share/revenue)?

    4) How big the social media measurement space is?

    If you can help with any of the above, I would extremely grateful – such information is hard to find surfing the web!

    Thanks in advance,

    Thomas Aedy

  20. Don Bartholomew July 7, 2008 at 4:42 pm #

    Hi Thomas,
    No easy answers to some of your questions, but the following may help a little:
    1. You will want to decide how you are defing social media and usage. Alexa is a good source for size/usage information (http://www.alexa.com/site/ds/top_sites?cc=US&ts_mode=country&lang=none)
    2. The Alexa site above can help answer this. Of the top 20 sites in the U.S., 7 are social media sites:
    3. MySpace
    4. Youtube
    5. Facebook
    8. Wikipedia
    11. Craigslist
    15 Photobucket
    19. Flickr
    3. Most are privately held or subsidiaries of larger companies and don’t list revenues. I have never seen a list.
    4. Again, I have no way of knowing. Perhaps one of the vendors will share their assumption. I would be surprised if the market is much more than $50MM at this time, but again, that is a guess. Perhaps Charlene Li at Forrester could help you with this data.

    Good luck! -Don B

  21. Don Bartholomew July 7, 2008 at 8:52 pm #

    Thomas,
    On my answer to Q4 above, I just learned Charlene Li is leaving Forrester on July 18. Perhaps Jeremiah Owyang could help you. -Don B

  22. Joseph Fiore July 7, 2008 at 8:57 pm #

    Hi Don,

    Thanks for including us in this great outline. Please note that since Nathan’s published work, we have changed our company name to RepuMetrix Inc.

    @Thomas, I thought the following might be useful for your project:

    Note: This chart reflects mentions on specific company’s and/or brands.

    Also, if you haven’t already come across it, there is a great wiki resource that might useful for your research.

    Joseph

  23. Joseph Fiore July 7, 2008 at 10:06 pm #

    Hmm, image didn’t show so here is the link to the chart.

  24. Thomas Aedy July 9, 2008 at 9:50 pm #

    Thanks, Don – very helpful.

  25. Mark Weiner August 3, 2008 at 2:39 am #

    As a regular reader of MetricsMan, I believe that in addition to providing a useful profile of some of the best prospective partners, he has uncovered the key to generating responses to the blog. For those of you who blog and wonder where your readers are, here’s a list of seven pointers taken from this exchange:

    1. Have an opinion and be willing to name the names of those with a commercial interest
    2. Don’t mention all the names and select even fewer for compliments
    3. Accept incoming gratitude from named suppliers with grace
    4. Prepare to separate “pure gratitude” from “commercial gratitude” which is unique in its desire to both offer thanks and to expand upon your good but incomplete information
    5. Get ready to assuage some egos among those omitted
    6. Feel good about initiating some meaningful dialogue
    7. Enjoy your role as a “market shaper” (but that’s the beauty of “owning the printing press,” isn’t it?)

    Don: thank you for this useful lesson. Thanks to you, I may have a future in blogging.

  26. FefeSevieteGus August 3, 2008 at 2:44 am #

    Thanks for the post

  27. Don Bartholomew August 4, 2008 at 1:19 pm #

    Mark,
    Very funny comment, although I wish you had not disclosed my secret strategy for all to see. :-) Thanks for reading. -Don B

Trackbacks/Pingbacks

  1. Public Relations Nightmares » Blog Archive » 10 Important Considerations When Selecting A Social Media … - July 1, 2008

    [...] post by metricsman Share These icons link to social bookmarking sites where readers can share and discover new web [...]

  2. 3 steps to choosing your social media metric vendor : Social Media Mafia - July 3, 2008

    [...] found a really good blog about measuring the value or ROI of public relations and a post on selecting your metric vendor. I think it will answer many of the questions people have about [...]

  3. links for 2009-02-26 « Brendan Cooper, your friendly neighbourhood social media strategist - February 26, 2009

    [...] 2008 July « Proving the Value of Public Relations Really useful list of providers (tags: monitoring onalytica influence collectiveintellect attentio corex compete kaava cic buzzlogic netmap radian6 visible motivequest measuring sentiment) [...]

  4. Social Media Question of the Week: Finding a Social Media Vendor — Regator Blog - April 1, 2009

    [...] “10 Important Considerations When Selecting A Social Media Measurement Vendor” hits on important points ranging from cost to language to content and focuses on measurement of benefits. [...]

  5. Beyond Social – What’s Next? | B2B Voices - November 24, 2010

    [...] media, but also point to an enormous accumulation of data. The challenge for communicators will be choosing the correct vendors to work with based on strategy and content. Your choices will be so you consolidate services with [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 170 other followers

%d bloggers like this: