Three Fundamentals of Great Social Media Measurement

If you want to evaluate the robustness and effectiveness of your approach to social media measurement, ask yourself these three fundamental questions:

• Does the approach measure the ‘right’ things in order to show the business impact of the programs and initiatives? 
• Will stakeholders of the report receive the data and actionable insights required to make strategic decisions?
• Are the data and insights presented in a clear and concise manner that tells a story and makes it easy to understand and act upon?

Measuring the ‘Right’ Things

Social media metrics are derived from three primary sources:

Ideally, a robust social media measurement program will have a rich metrics set that contains metrics from all three areas. Metrics tied to program objectives allow for direct measurement of program success. Fundamentally, measurement is about assessing performance against objectives. It is surprising how often social program objectives are slanted toward channel-specific metrics (e.g. Likes or Followers) and not the specific outcomes desired for the program – what you hope to accomplish by implementing the program. Also, relying too heavily on channel metrics limits you to what you can measure rather than what you should measure. Business outcome metrics are used to connect the dots between social media programs and the business results they are designed to drive. Social programs that cannot answer, or at least address, the management question, “How is this impacting my business”, are more susceptible to resource allocation scrutiny (#pleasecutmybudget). Stated another way, if management asks how we’re doing in social media and we reply, “great, post virality is up 6.1% this month”, we make it difficult for that individual to understand how social media/business initiatives are helping move the business forward.

Getting to Data and Insights that Inform Strategic Decisions

Expectations for social media measurement and analysis have risen. In addition to sound analysis and reporting of performance against key metrics and KPIs, understanding audience dynamics and developing actionable insights are rapidly becoming de rigueur. Insights may be defined as synthesizing and interpreting data to provide actionable information and knowledge that informs strategic decisions. Too many social media measurement programs take a social-centric rather than a business-centric approach to insights. They often feature insights and recommendations that are tactical in nature – the best time of day or how many times to tweet, or what type of content seems to be most successful. Ideally, insights and recommendations in social measurement reports would be operating one level above this, informing strategic decisions about how social programs and conversations are impacting, or could impact, the business. To do this requires an understanding of the business function (e.g. marketing, customer service) impacted by the social program and an ability to ask the right questions prior to starting a social media analysis. 

For example, let’s say Company X plans to introduce a new video game. A social listening program has been implemented to analyze the early consumer reaction to the game. Based on the listening analysis, changes to the packaging, marketing or even the product itself are possible. If you are in charge of the marketing campaign for the game, what are the types of social media insights you need to make decisions about the game and the marketing campaign?

• What is the level of buzz about the game?  What is the overall sentiment? How does this compare to previous game launches?
• What are people talking about in social media – availability, cost, specific features of the game, packaging, marketing campaign?
• What features of the game do consumers seem to like most?  Least? Specifically, what do they like or dislike?
• What are the most influential gaming enthusiasts saying about the product?
• Who are the promoters and detractors? What is the ratio of promoters to detractors? How does this compare to promoters and detractors from previous game launches?
• How much social media conversation contains recommendations or expresses purchase intent?  How does this compare to previous launches?

Answering these types of questions provides actionable insights that provide context and can inform strategic marketing decisions.          

Presenting Results

Dashboards have gotten a bit of a bad rap – not because dashboards are not useful, but because some have used them as THE measurement report rather than just one aspect of a good report. I’m a dashboard proponent for a few reasons:

• Deciding which metrics to feature on a dashboard is a good strategic exercise requiring you to focus on the very most important and relevant metrics for the intended audience
• Online, dynamic dashboards are an effective user interface that can be used as a launching- off point for drilling into data to understand the underlying story
• Good dashboards present a snapshot of overall performance that is easily absorbed and understood.   

A dashboard-driven social media measurement report is versatile and effective in many situations. A typical report might consist of one of more dashboards and then a deeper dive on each of the key metrics featured on the dashboards, along with audience insights, strategic insights and recommendations. This format provides a quick snapshot (dashboard) of results, ideal for those stakeholders interested only in topline data, and provides sufficient depth to satisfy those more interested in the underlying drivers of the metric  

Social media measurement programs that are built around metrics tied to business outcomes and show how programs are performing against objectives are important. Reports that deliver clear insights that inform strategic decisions are important. And delivering those reports in a compelling format that enhances usability and effectiveness is important. How do your programs stack up?

Bringing Some Clarity to Social Media Influence

The emphasis on influencer marketing in social media has reached a fever pitch in 2011 and with it a flood of tools and opinions on how to navigate the influence waters. This is interesting in that one of the most powerful aspects of social media marketing is the ability to establish connections and relationships directly with prospects and customers and not have to go through an intermediary to communicate. But we’ll leave that to the social strategists to reconcile and justify. Influencer marketing is hardly a new strategy. Through the years, much work in traditional public relations utilized influencer targeting (e.g. market analysts, financial analysts, KOLs, other customers) to help amplify and endorse a brand or a company’s products and services. So why is there so much discussion and confusion about influence in social media? Let’s explore.       

Influence Basics

A definition I like for influence is: effecting change in another person’s attitudes, opinions, beliefs and/or behavior. I believe the most overlooked word in this definition is change.  Without change influence has not truly occurred. One challenge here is influence can happen without any resulting short-term observable action. Influence takes hold primarily between the ears, not necessarily with hand on mouse or wallet. This creates fundamental challenges when trying to measure the degree to which influence has occurred.

Another challenge we face is that influence is contextual not absolute. People who influence others do so primarily in areas where they have specific expertise or authority. It is common to be influential in one area but have little or no influence in others. One of the main issues with current influence tools are they do a relatively poor job of establishing contextual relevance.

The distinction between creating influence within a target audience and who/what has influence over the target has a tendency to get muddled. To clarify, determining who has the potential to influence the target audience, (the influencers), is a targeting question. Have we created influence, (changed attitudes, opinions, beliefs and/or behavior) is a measurement question.   

Influence is purposeful. In real life or digital life, when we set out to change the opinion, attitude, beliefs or behavior of another person or group, we do so with a downstream motivation – for them to take a specific action. The list of possible actions is lengthy – buy a product, visit a website, tell a friend, vote, wave a sign and donate to name a few. Of course, not all desired actions are equal in terms of amount of influence required for change. Opinions might be easier to change than an attitude. An attitude is easier to change than a belief. Behavioral change can range from relatively easy to nearly impossible depending on the specific behavior. In marketing, the ultimate behavior or action we try to influence is purchase behavior. It is important to think through the specific actions you hope the target will take as a result of being influenced. This is also the sweet spot for influence measurement.

While creating an action/behavior change is the ultimate reason for influencing someone, it is helpful to think of the process of influence as two stages – opinion, attitude or belief change – and then, because of this change, did an action occur or was a behavior changed. Stated another way, the opinion change is an intermediate or micro outcome and the desired action is a final or macro outcome. Depending on the type of purchase decision there may be a time lag between the micro and macro outcomes that make it difficult to connect the dots. In his book The Business of Influence, Philip Sheldrake presents a concept called the “Maturity of Influence Approach”. Basically it melds two important concepts to use when thinking about influence measurement – focus on the influence, not the influencer (Philip refers to this as “influence-centric), and to start at the macro outcome/action and trace the path of influence back to the source(s) of influence. One simple example of this in a B2B context would be to ask the prospect at the time they are ready to make a purchase, “what sources of information or opinion were most valuable to you in making your decision to buy our product?” A similar question or two can be asked using a pop-up survey in an ecommerce situation. 

Influence and Engagement Confusion

A primary source of influence confusion is failing to distinguish between a simple act of engagement and the process of being influenced. If someone in my Twitter stream sends out a tweet and I retweet it, have they influenced me to retweet or have I simply engaged with that individual’s content? Many who have written about social media influence have suggested that in RTing the tweet, I have been influenced to do so. I do not believe that is the case. I have engaged with the content, but have there been any true changes in my attitudes, opinions, beliefs or behavior? Again, the operative word here is change. Does the act of RTing constitute a behavioral change? Probably not. Engagement – yes, influence - no.

Engagement is a necessary pre-condition to Influence. (This social media measurement model addresses the distinction) Without engagement you don’t have the opportunity to influence. Influence, however, only occurs if that engagement leads to a change in attitudes, opinions, beliefs and behavior.   

Influence, Popularity and Celebrity Confusion

There also seems to be some confusion about the difference between influence, popularity and celebrity. Although related, and in some cases overlapping, they are three distinct concepts. In my opinion, at least some of the confusion stems from Klout and other influencer tools that seem to measure popularity but call it influence. So what is the difference?

Popularity is the state of being popular – widely admired, accepted or sought after.

Celebrity is a famous person, renown and fame. 

If popularity is about being well-liked and celebrity is about being well-known, influence is more about being well-respected, with associations like knowledge, persuasion and trust. Some of the confusion lies in the fact that some celebrities do have influence over the types of behaviors that make the cash register ring. Oprah comes to mind. Other celebrities, while very popular, don’t really have the ability to create meaningful influence. They can get content re-tweeted (WINNING!) but do they have any influence over the types of actions brands really value?

Keeping Online Influence in Perspective  

As we discuss the intricacies of digital influence we should also keep in mind the majority of influence occurs in the analog world. I’ve seen estimates ranging from 70 – 90% of influence occurring by offline WOM. It’s personal. It’s about real family and friends and not Twitter friends. Influence is about a relatively small number of people (Dunbar’s Number suggests humans have a finite cognitive capacity to have around 150 social relationships with other humans), and not mass influence. The fact that most influence happens offline presents another significant measurement challenge.

In summary, I’ll leave you with a few sound bites on social media influence:

• Influence is about change
• Engagement leads to influence
• One can be popular but not influential
• Measure the influence not the influencer
• Don’t forget offline when measuring online influence.

Thanks for reading.  See it a different way?

Measurement 2020 and Other Fantasies

At the 3^rd European Summit on Measurement held in Lisbon in June 2011, standardization, education, ROI and measurement ubiquity emerged as the key themes in response to a call to set the Measurement Agenda 2020.  Delegates to the conference voted on 12 priorities they thought were most important to focus on in the period leading up to 2020.  The top four vote-getters became the Measurement Agenda 2020:

1. How to measure the return on investment of public relations (89%)
2. Create and adopt global standards for social media measurement (83%)
3. Measurement of PR campaigns and programs needs to become an intrinsic part of the PR toolkit (73%)
4. Institute a client education program such that clients insist on measurement of outputs, outcomes and business results from PR programs (61%)

For a very nice overview of the Lisbon session and the Barcelona Principles that came before, read this post from Dr. David Rockland of Ketchum who chaired the Barcelona and Lisbon sessions.  David pretty much said it all on these sessions, so I’ll just add a couple of comments and share a few thoughts on what I believe the future of measurement 2020 could be.

The rallying cry coming out of Barcelona has been focused and loud – death to AVEs!  Will there be a similar thematic coming out of Lisbon and what might it be?  My money is on standardization, borne out of cross-industry cooperation.  As David points out in his post, and in the words of AMEC Chairman Mike Daniels, “The Summit identified some significant challenges for the PR profession to address by 2020.  However, what we also accomplished in Lisbon beyond setting the priorities was to harness the commitment and energy of the industry to agree what we need to do together.”  The current cooperation and collaboration between industry groups – AMEC, Institute for Public Relations, PRSA and the Council of PR Firms is unprecedented in my time in this industry and is focused on tangible outcomes.  Cross-organization committees are already at work developing standard metrics for social media measurement for example.  The spirit of cooperation is uplifting.  While the outward thematic appears to be standardization, cooperation is the enabling force.  

I was also struck by the symmetry of the call to end AVEs in Barcelona and the call to codify ways to measure ROI in Lisbon.  One follows the other.  In my opinion the primary reason AVEs exist is because PR practitioners feel pressure to prove the value of what they do, and quite often they are asked to describe the impact in financial terms.  AVEs are perceived as a path of least resistance way to express financial value.  Except, as we all know, AVEs don’t really have anything to do with the impact public relations creates.  They are a misguided proxy for financial value.  Hence the need for research-based methods to determine true return on investment.

All of the priorities coming out of Lisbon are excellent goals for the industry.  And like David Rockland, I believe they will be achieved, and be achieved before 2020.  Here are three other items on my wish list for Measurement 2020:

Word of Mouth/Word of Mouse Integration: For those of us focused in social media and other digital technologies, we can’t allow our digital lens to color what is fundamentally an analog world.  Research studies suggest the majority of word of mouth happens in real life.  From an influence perspective, I don’t think too many would argue that word of mouth from a trusted friend or family member is more powerful than word of mouse from someone you follow on Twitter.  Digital cross-platform research is difficult enough, but when one huge platform is ‘real life’, we have significant challenges in measurement.  WOMMA and others have made early attempts to define measurement approaches for offline WOM, but much work remains.  We need ways to assess its impact and then we need to think about ways to attribute value to that impact.  Mobile is a wild card here as it becomes the preferred platform for online activity.  The need to triangulate online, mobile and ‘real life’ measurement presents significant challenges today, and may still by 2020.

Cookie Wars: We all know the measurement versus privacy showdown is coming, right?  The first shots have already been fired.  The collection of source-level personal data, enabled by cookies, is crucial to measurement and insights but has the potential for misuse or unintended disclosure.  Some sophisticated consumers have had their fill of cookies.  Although the broader issue might be framed as social sharing versus privacy control, how it plays out will have a direct impact on digital analytics and measurement.

Integrated Measurement across the Paid Earned Shared Owned (PESO) Spectrum: Measurement has increasingly become integrated.  It began with integrated traditional (Earned) and social media (Shared) measurement and then progressed rapidly to Earned, Owned and Shared, which is where most integrated measurement programs are today.  Many leading-edge integrated programs today also include advertising or Paid media.  By 2020, integrated measurement across the PESO spectrum will most likely be the norm and not the exception.  A key enabling element here in my view is some base level of agreement on how each area should be measured and standard metrics for each.  It will take significant cooperation between industry groups, vendors, agencies and major customers/clients for cross-discipline standardization to move forward effectively.  We are at the beginning of this movement in 2011.  By 2020, it will be fascinating to look back and see how all this plays out.

When looking ahead to 2020, I am reminded of a measurement discussion pulled together by PRWeek a couple of years ago.  Many of the Measurati attended.  In response to a question of where measurement will be in five years, David Rockland replied (paraphrasing here), ‘Who knows?  Five years ago who would have guessed we would all be focused on how to measure social media?’  So, there is a certain fantasy element to discussing 2020 challenges in measurement.  What are your measurement fantasies?

Don’t Let the Tool Tail Wag the Measurement Dog

Social media listening and measurement tools are sexy.  Well, at least to those of us in research and measurement – it’s all relative right?  In the last three years or so there has been an explosion of social media tool vendors and platform choices.  Tools are sexy and important, but in the grand scheme of things are being overemphasized to some degree.  We are letting tools decide what we can measure without giving sufficient thought to what we should measure.  We are letting the tool tail wag the measurement dog.

There are several steps and decisions that should be addressed prior to selecting a tool or suite of tools.  Consider this diagram as a starting point to help you think through these interim considerations and decisions:

OBJECTIVES

Proper social media objectives should be measurable (indicate change in metric of interest and timeframe) and aligned with desired organizational outcomes.  Understanding the social media objectives will suggest broad parameters the measurement program, and ultimately the tool decision, must operate within.  For example, geographic coverage requirements, type of content to be considered and on-platform engagement capability may all be strongly suggested based on a review of social media objectives.

PROCESS

In addition to comprehending organizational or business outcomes, it is essential to understand the business process the social media program will address or drive.  If the program is marketing oriented, the sales funnel process (Awareness/Consideration/Preference/Sales/Loyalty) may be most appropriate.  For a brand-building campaign, the brand pyramid (Presence/Relevance/Performance/Advantage/Bonding) is what you want to measure your program impact against.  Other business processes that are commonly addressed by social media programs include customer service and support, CRM, corporate reputation and lead generation.

METRICS

Understanding the requisite business process the social media program is driving is crucial because each business process drives specific metrics.  For example, the sales funnel drives a specific metrics set:  percentage of unaided or aided awareness; percentage of the target audience who would consider the product/company; percentage who prefer the product/company; incremental sales revenues; percentage who would purchase the product again number or the number/amount of repeat purchases.  For B2B companies, the lead generation process would drive a different set of metrics: number of incoming leads; percentage/number of qualified leads; lead conversion rate; sales revenues generated.  In addition to the business process metric sets, there are other metrics areas like Exposure and Engagement we will want to address.  Reach/opportunities to see, share of positive discussion, comments/post ratio, number of @ mentions and RTs per 1000 followers are examples of ‘standard’ metrics that might be applicable for many social media programs.

Understanding how the social media program drives a specific business process is also important to our ability to describe the impact or, in some cases, return on investment the program has created.

DATA SETS

Each metric has data requirements, usually two pieces of data per metric – a numerator and a denominator.  Examine the set of metrics you have defined for your social media program.  Catalog all the specific pieces of data you need to compute the various metrics.  For example, the data needed to compute the basic sales funnel metrics and some ‘standard’ metrics might include:

• Number of individuals in the target audience
• Number of survey respondents
• Number of respondents ‘aware’ of the product/company
• Number of respondents who would consider/seriously consider purchasing the product/doing business with the company
• Number of respondents purchasing the product
• Amount of sales revenue directly attributable to the program
• Number of purchasers who purchased again
• Total branded mentions
• Volume of positive and negative mentions
• Number of posts
• Number of comments
• Number of RTs and @ mentions
• Number of followers

TOOLS

Armed with an understanding of all the data needed to calculate the metrics required to measure the social media program, you will be able to assess which tools or classes of tools best deliver the data you need.  Pick the best three to five tools for further evaluation.  You most likely will find no one tool can deliver the complete data set you need.  It is common to need two or more tools, e.g. web analytics package and social content analysis platform, in order to fully meet data requirements.  Budgetary constraints may also limit your ability to capture the entire data set required.

By addressing the interim steps leading up to tool selection, you will be able to make a more informed tool decision.  You also will have a much better chance of measuring what you should measure rather than settling for what you can measure.  No tool before its time.  Let the big dogs run.

Inflationary Twitter Audience Numbers Hurt Social Media Credibility

In yesterday’s New York Times, you may have read the article, Spinning the Web: P.R. in Silicon Valley, an interesting although not overly insightful piece.  From a social media measurement perspective, two items caught my eye.  The first, referring to Brian Solis, Principal of FutureWorks, about how he calculates social media audience figures:

“Instead of calculating the impressions an article gets by estimating a publication’s circulation and pass-along rate, Mr. Solis counts the number of people who tweeted about a company and their combined following, the number of retweets or clicks on links, as well as traffic from Facebook and other social networks.”

Toward the end of the article, we learn:

“By 6:30 p.m. on the day Wordnik went live, Brew’s staff calculated that 1.43 million people had seen tweets about it.”

Setting aside for a moment that the article and these sorts of audience metrics take a broadcast-oriented view of Twitter (Mr. Solis discusses the shortcomings of the NYT viewpoint here), the emerging view of audience measures for Twitter is to calculate the Followers of each person tweeting about the subject of interest, and then adding Follower numbers for each person retweeting the subject and so on.  The issue here, much as it is in traditional public relations, is that the audience figure that results from these sorts of calculations grossly overstates, by one or two orders of magnitude or more, the actual “audience” for these tweets.  It is a hypothetical number that assumes everyone that possibly could see a tweet has in fact seen it, and everyone who sees it is relevant to you/your brand.  This is fantasy of course.

On the issue of relevant audience, here is a quick example.  At the time I pulled these figures, the audited circulation of the New York Times was 4,974,000.  Most PR practitioners getting a ‘hit’ in the NYT would claim this as their audience.  However:

• If you were only trying to reach a C-Suite audience with your message, the actual audience reached would be 598,000 or 12% of the total circulation
• If you were trying to reach Women, your audience would be 1,937,000 or 39% of the total
• If you were trying to reach 25 – 54 year old Men, your potential audience would have been 2,930,000, or 59% of the total number.

There is a large difference between how many people theoretically can see a tweet, versus how many actually saw it/read it, versus how many of those seeing the tweet find it relevant to them, versus how many engaged with it by hitting a link or retweeting.  Part of my issue with this is the language we use to report the figures.  For the Brew staff to use these numbers to estimate 1.43 million people “had seen tweets about it” is wrong.  If they had said 1.43 million people had an opportunity to see the tweet, it would have been more realistic, although still greatly overstating actual relevant audience.

This problem of audience inflation has already been institutionalized in public relations.  The use of Impressions as an output metric does not mean a true impression in the branding sense, but rather an opportunity to see the content.  To make matters worse, many PR practitioners believe Impressions should be factored by either dubious pass-along readership figures and/or use of a multiplier to account for the mythical credibility advantage PR enjoys over impressions generated from advertising.  The simple fact is there is no research-supported, fact-based argument for using any adder or multiplier in public relations when calculating potential audience (here’s an IPR white paper on this subject I co-authored).

For Twitter and other social networks we lack demographics and data about tweet readership averages (i.e. what is the probability that any one tweet is actually read) that would allow for more precise audience estimates.  In the absence of data, believable assumptions should be used:

• Out of all the opportunities to see, how many actually read the tweet?  10%?
• Of those reading the tweet, how many find it relevant to them (or from the other perspective, how many of the readers are in your intended target audience)?  Maybe 10% again?

You can see how our audience estimate has already been reduced by a factor of 100.  This may well still be overstating the actual, relevant audience.  The issue here is that unrealistic and overstated audience figures have the potential to hurt credibility and call into question other data and metrics that may be more grounded in fact.  Actually the more meaningful metrics pertain to engagement or outcomes rather than exposure/outputs.  It is more meaningful that 40,000 visited the Wordnik website as a result of the campaign discussed in the NYT article than the overstated 1.43 million who were estimated to have seen the tweets.  40,000 is real.  1.43 million is fantasy.

A New Model for Social (and traditional) Media Measurement

In November 2007 I suggested the current Outputs, Outtakes, Outcomes model and taxonomy for public relations measurement was confusing and therefore often misunderstood and misapplied (Let’s put Outputs, Outtakes and Outcomes in the Outhouse).  At that time I suggested a simpler, more descriptive approach was in order and offered the following:

“What we need is a metrics taxonomy that is easier to understand and explain.  Perhaps simple and descriptive enough that we could skip the need for explanation altogether.   I propose the following three terms:
* Exposure – to what degree have we created exposure to materials and message?
* Influence – the degree to which exposure has influenced perceptions and attitudes
* Action – as a result of the public relations effort, what actions if any has the target taken?”

Since November, I have given a lot of thought to the E-I-A construct and how to improve upon it.   Some of the feedback to the model was the gap between Exposure and Influence was too great, and perhaps there should be an interim step called Understanding or Relevance.  There is also the social media dynamic to consider since the measurement model should be flexible enough to work for both traditional and social media.

What seems to fit best between Exposure and Influence, and adds richness to social media measurement, is the concept of Engagement.  Not only is it one of the hotter topics in social media, it is consistent with the desire to have more descriptive and easily understood metrics.   With Engagement we now have an category that nicely contains such emerging key metrics as view-thrus, duration spent with content, repeat commenters and comments/posts ratio.  It also works well for old school metrics like recall and retention.  Engagement is what helps set the stage for Influence to occur.  Engagement is necessary for communities to form.  Engagement is fundamental to brand.

Here’s a graphic that shows the new model and sample metrics that might be used at each stage:

Would love your feedback on this new Exposure — Engagement — Influence — Action model.

There are still a few challenges in adoption of the model, not the least of which is that there is no consistent definition of Engagement.  Current definitions range from the simple comments to post ratio used by BusinessWeek in their Reader Engagement Index, to the 8-term formula for Engagement offered by Eric T Peterson.  The next year should bring more clarity and consistency to our understanding and use of Engagement.  At least there is modest agreement on the specific metrics contained within the category of Engagement.

Thanks for reading, Don B