Don’t Let the Tool Tail Wag the Measurement Dog

19 Jul

Social media listening and measurement tools are sexy.  Well, at least to those of us in research and measurement – it’s all relative right?  In the last three years or so there has been an explosion of social media tool vendors and platform choices.  Tools are sexy and important, but in the grand scheme of things are being overemphasized to some degree.  We are letting tools decide what we can measure without giving sufficient thought to what we should measure.  We are letting the tool tail wag the measurement dog.

There are several steps and decisions that should be addressed prior to selecting a tool or suite of tools.  Consider this diagram as a starting point to help you think through these interim considerations and decisions:

OBJECTIVES

Proper social media objectives should be measurable (indicate change in metric of interest and timeframe) and aligned with desired organizational outcomes.  Understanding the social media objectives will suggest broad parameters the measurement program, and ultimately the tool decision, must operate within.  For example, geographic coverage requirements, type of content to be considered and on-platform engagement capability may all be strongly suggested based on a review of social media objectives.

PROCESS

In addition to comprehending organizational or business outcomes, it is essential to understand the business process the social media program will address or drive.  If the program is marketing oriented, the sales funnel process (Awareness/Consideration/Preference/Sales/Loyalty) may be most appropriate.  For a brand-building campaign, the brand pyramid (Presence/Relevance/Performance/Advantage/Bonding) is what you want to measure your program impact against.  Other business processes that are commonly addressed by social media programs include customer service and support, CRM, corporate reputation and lead generation.

METRICS

Understanding the requisite business process the social media program is driving is crucial because each business process drives specific metrics.  For example, the sales funnel drives a specific metrics set:  percentage of unaided or aided awareness; percentage of the target audience who would consider the product/company; percentage who prefer the product/company; incremental sales revenues; percentage who would purchase the product again number or the number/amount of repeat purchases.  For B2B companies, the lead generation process would drive a different set of metrics: number of incoming leads; percentage/number of qualified leads; lead conversion rate; sales revenues generated.  In addition to the business process metric sets, there are other metrics areas like Exposure and Engagement we will want to address.  Reach/opportunities to see, share of positive discussion, comments/post ratio, number of @ mentions and RTs per 1000 followers are examples of ‘standard’ metrics that might be applicable for many social media programs.

Understanding how the social media program drives a specific business process is also important to our ability to describe the impact or, in some cases, return on investment the program has created.

DATA SETS

Each metric has data requirements, usually two pieces of data per metric – a numerator and a denominator.  Examine the set of metrics you have defined for your social media program.  Catalog all the specific pieces of data you need to compute the various metrics.  For example, the data needed to compute the basic sales funnel metrics and some ‘standard’ metrics might include:

  • Number of individuals in the target audience
  • Number of survey respondents
  • Number of respondents ‘aware’ of the product/company
  • Number of respondents who would consider/seriously consider purchasing the product/doing business with the company
  • Number of respondents purchasing the product
  • Amount of sales revenue directly attributable to the program
  • Number of purchasers who purchased again
  • Total branded mentions
  • Volume of positive and negative mentions
  • Number of posts
  • Number of comments
  • Number of RTs and @ mentions
  • Number of followers

TOOLS

Armed with an understanding of all the data needed to calculate the metrics required to measure the social media program, you will be able to assess which tools or classes of tools best deliver the data you need.  Pick the best three to five tools for further evaluation.  You most likely will find no one tool can deliver the complete data set you need.  It is common to need two or more tools, e.g. web analytics package and social content analysis platform, in order to fully meet data requirements.  Budgetary constraints may also limit your ability to capture the entire data set required.

By addressing the interim steps leading up to tool selection, you will be able to make a more informed tool decision.  You also will have a much better chance of measuring what you should measure rather than settling for what you can measure.  No tool before its time.  Let the big dogs run.

17 Responses to “Don’t Let the Tool Tail Wag the Measurement Dog”

  1. Wendy July 19, 2010 at 10:15 am #

    Well said. It is tempting to want new, pretty tools but you’re right – what matters is whether you can garner the data you need. Objectives have always led (or should have led!) strategic marketing decisions (media? creative?) and measurement and research is no different in that respect.

  2. metricsman July 19, 2010 at 10:26 am #

    Hi Wendy,
    Thanks for stopping by and for your kind words. -Don B @Donbart

  3. Lars July 20, 2010 at 12:43 am #

    Hey Don – another great post! I just did a 3-hours session on measurement last week and the main questions in the beginning was “WHY… do you want to measure” while the expection of most delegates was to talk tools…
    Here’s the preso if you find it useful: Measuring the Effectiveness of PR Efforts: Are You Busy or Indispensable? http://bit.ly/cui22i

  4. Michelle July 20, 2010 at 9:48 am #

    Absolutely. Working for a social media monitoring tool, I have to say it’s pretty cool that I get to work with a team that customizes our tool to each client. Not every brand/company has the same objectives or strategy, and how they want to measure, how they want to share these measurements, and what they want to do afterwards is different for each one. I just saw, though, that the IAB is looking to standardize measurements for PR professionals. It’ll be interesting to see what comes out the next 2-3 years.

  5. John Carraway July 21, 2010 at 9:02 am #

    I was just wondering how accruate and up to date these tools are. Do they allow for real time analysis and are they 100% accurate? Will I instantly know when a product is being mentioned on Twitter or will I find out the next day or week.

  6. metricsman July 21, 2010 at 1:04 pm #

    Hi Lars,
    Thanks for stopping by and leaving a comment. I reviewed your presentation – lots of good stuff in there. Two items I might have a different view on:
    – Value of PR is primarily in Outcomes not Outputs IMO
    – Another major reason to measure is for diagnostic info – what’s working and what isn’t.

    Thanks again, Don B

  7. metricsman July 21, 2010 at 1:10 pm #

    Hi Michelle,
    Thanks for leaving a comment. There are parallel efforts to create standardization in measurement – not familiar with the IAB effort – perhaps IABC? For some of the reasons you point out – company/brands have differing/unique objectives, market conditions, reputations, etc. – many believe standardization in measurement is either impossible or ill-advised. I call this the snowflake school of measurement. It will indeed be interesting to watch and participate in the conversation around standardization. Cheers, Don B

  8. metricsman July 21, 2010 at 1:18 pm #

    Hi John,
    Thanks for your comment. Could you define accurate? Could mean degree to which all content is captured, could pertain to sentiment analysis, etc. Not sure what you mean.

    I believe the leading tools are best described as ‘near real time’. Some are delayed by only a matter of seconds, others refresh every X minutes, with X often around 15. Some tools allow you to change how often they are refreshed. Most of the leading tools allow you to program automated alerts so you can receive an email when certain terms appear in your platform (‘brand name + fire’ for example).

    Again, accuracy can mean many things. With respect to automated sentiment analysis, most listening platform companies hesitate to state an ‘accuracy’ figure on the record, but most claim 70 – 85% accuracy privately. In my experience, it depends a lot on the type of content. For an online news article this might be a good range, but with Twitter, the chances are it will be less, perhaps much less. Maybe even ‘coin-flip’ accuracy.

    Hope this helps a little. Thanks again, Don B @Donbart

  9. mot August 2, 2010 at 2:40 am #

    a nice tips, maybe other people don’t know about this simple tips but very useful for us

  10. Kelly Watson August 6, 2010 at 6:03 am #

    Another brilliant post from Metrics Man! I totally agree. When looking over the analytics and the analysis, it’s easy to forget the real agenda. I find myself doing this even when looking at the traffic stats of my blog. I get sucked into the numbers and forget the value of the conversations and feedback that are taking place.

  11. Rob Leavitt September 23, 2010 at 11:47 am #

    Great post, Don. As in so many areas of marketing and social media, strategy and objectives have to drive selection and use of tools – a simple principal that it’s all too easy to forget in the crush of “getting stuff done.”

Trackbacks/Pingbacks

  1. Define the Job, Then Find the Right Tool | Fleishman-Hillard | Point of View - July 19, 2010

    [...] Then Find the Right Tool In the last few years, the growth in digital communications has led to an explosion of social media tools and platform options. But research specialist Don Bartholomew wonders if marketers are giving sufficient thought to what [...]

  2. Sosiaalisen median seuranta nyt ja huomenna « Republic Of Communications - September 3, 2010

    [...] kehittyy ja mitä trendejä on nousemassa esiin. Etenkin USA:ssa käsitellään paljon sosiaalisen median analysointi ja tiedon hyödyntämistä [...]

  3. Weekly Roundup: The Good, Bad and Ugly of PR | Waxing UnLyrical - September 19, 2010

    [...] Don’t let the tool wag the measurement dog, from Don [...]

  4. Measuring Public Relations: ROI and Unicorns | Waxing UnLyrical - November 16, 2010

    [...] For a really detailed overview, read Don Bartholomew’s excellent post, don’t let the tool tail wag the measurement dog. [...]

  5. Shonali Burke Consulting, Inc. | Measuring Public Relations: ROI and Unicorns - August 10, 2014

    […] For a really detailed overview, read Don Bartholomew’s excellent post, don’t let the tool tail wag the measurement dog. […]

  6. Shonali Burke Consulting, Inc. | Weekly Roundup: The Good, Bad and Ugly of PR - August 10, 2014

    […] Don’t let the tool wag the measurement dog, from Don […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 171 other followers

%d bloggers like this: