Don’t Let the Tool Tail Wag the Measurement Dog

Social media listening and measurement tools are sexy.  Well, at least to those of us in research and measurement – it’s all relative right?  In the last three years or so there has been an explosion of social media tool vendors and platform choices.  Tools are sexy and important, but in the grand scheme of things are being overemphasized to some degree.  We are letting tools decide what we can measure without giving sufficient thought to what we should measure.  We are letting the tool tail wag the measurement dog.

There are several steps and decisions that should be addressed prior to selecting a tool or suite of tools.  Consider this diagram as a starting point to help you think through these interim considerations and decisions:

OBJECTIVES

Proper social media objectives should be measurable (indicate change in metric of interest and timeframe) and aligned with desired organizational outcomes.  Understanding the social media objectives will suggest broad parameters the measurement program, and ultimately the tool decision, must operate within.  For example, geographic coverage requirements, type of content to be considered and on-platform engagement capability may all be strongly suggested based on a review of social media objectives.

PROCESS

In addition to comprehending organizational or business outcomes, it is essential to understand the business process the social media program will address or drive.  If the program is marketing oriented, the sales funnel process (Awareness/Consideration/Preference/Sales/Loyalty) may be most appropriate.  For a brand-building campaign, the brand pyramid (Presence/Relevance/Performance/Advantage/Bonding) is what you want to measure your program impact against.  Other business processes that are commonly addressed by social media programs include customer service and support, CRM, corporate reputation and lead generation.

METRICS

Understanding the requisite business process the social media program is driving is crucial because each business process drives specific metrics.  For example, the sales funnel drives a specific metrics set:  percentage of unaided or aided awareness; percentage of the target audience who would consider the product/company; percentage who prefer the product/company; incremental sales revenues; percentage who would purchase the product again number or the number/amount of repeat purchases.  For B2B companies, the lead generation process would drive a different set of metrics: number of incoming leads; percentage/number of qualified leads; lead conversion rate; sales revenues generated.  In addition to the business process metric sets, there are other metrics areas like Exposure and Engagement we will want to address.  Reach/opportunities to see, share of positive discussion, comments/post ratio, number of @ mentions and RTs per 1000 followers are examples of ‘standard’ metrics that might be applicable for many social media programs.

Understanding how the social media program drives a specific business process is also important to our ability to describe the impact or, in some cases, return on investment the program has created.

DATA SETS

Each metric has data requirements, usually two pieces of data per metric – a numerator and a denominator.  Examine the set of metrics you have defined for your social media program.  Catalog all the specific pieces of data you need to compute the various metrics.  For example, the data needed to compute the basic sales funnel metrics and some ‘standard’ metrics might include:

• Number of individuals in the target audience
• Number of survey respondents
• Number of respondents ‘aware’ of the product/company
• Number of respondents who would consider/seriously consider purchasing the product/doing business with the company
• Number of respondents purchasing the product
• Amount of sales revenue directly attributable to the program
• Number of purchasers who purchased again
• Total branded mentions
• Volume of positive and negative mentions
• Number of posts
• Number of comments
• Number of RTs and @ mentions
• Number of followers

TOOLS

Armed with an understanding of all the data needed to calculate the metrics required to measure the social media program, you will be able to assess which tools or classes of tools best deliver the data you need.  Pick the best three to five tools for further evaluation.  You most likely will find no one tool can deliver the complete data set you need.  It is common to need two or more tools, e.g. web analytics package and social content analysis platform, in order to fully meet data requirements.  Budgetary constraints may also limit your ability to capture the entire data set required.

By addressing the interim steps leading up to tool selection, you will be able to make a more informed tool decision.  You also will have a much better chance of measuring what you should measure rather than settling for what you can measure.  No tool before its time.  Let the big dogs run.

A New Model for Social (and traditional) Media Measurement

In November 2007 I suggested the current Outputs, Outtakes, Outcomes model and taxonomy for public relations measurement was confusing and therefore often misunderstood and misapplied (Let’s put Outputs, Outtakes and Outcomes in the Outhouse).  At that time I suggested a simpler, more descriptive approach was in order and offered the following:

“What we need is a metrics taxonomy that is easier to understand and explain.  Perhaps simple and descriptive enough that we could skip the need for explanation altogether.   I propose the following three terms:
* Exposure – to what degree have we created exposure to materials and message?
* Influence – the degree to which exposure has influenced perceptions and attitudes
* Action – as a result of the public relations effort, what actions if any has the target taken?”

Since November, I have given a lot of thought to the E-I-A construct and how to improve upon it.   Some of the feedback to the model was the gap between Exposure and Influence was too great, and perhaps there should be an interim step called Understanding or Relevance.  There is also the social media dynamic to consider since the measurement model should be flexible enough to work for both traditional and social media.

What seems to fit best between Exposure and Influence, and adds richness to social media measurement, is the concept of Engagement.  Not only is it one of the hotter topics in social media, it is consistent with the desire to have more descriptive and easily understood metrics.   With Engagement we now have an category that nicely contains such emerging key metrics as view-thrus, duration spent with content, repeat commenters and comments/posts ratio.  It also works well for old school metrics like recall and retention.  Engagement is what helps set the stage for Influence to occur.  Engagement is necessary for communities to form.  Engagement is fundamental to brand.

Here’s a graphic that shows the new model and sample metrics that might be used at each stage:

Would love your feedback on this new Exposure — Engagement — Influence — Action model.

There are still a few challenges in adoption of the model, not the least of which is that there is no consistent definition of Engagement.  Current definitions range from the simple comments to post ratio used by BusinessWeek in their Reader Engagement Index, to the 8-term formula for Engagement offered by Eric T Peterson.  The next year should bring more clarity and consistency to our understanding and use of Engagement.  At least there is modest agreement on the specific metrics contained within the category of Engagement.

Thanks for reading, Don B