Archive | Industry News & Views RSS feed for this section

Social Media Measurement at a Crossroads

25 Jul

Please join me on August 6, from 2:00 – 3:00pm EDT for CARMA’s fourth quarterly webinar, co-sponsored by PRNews, titled “Social Media Measurement at a Crossroads.” Here is a little more information on the webinar:

With social media clearly entrenched as a mainstream business activity, the need to measure the impact on the organization has never been greater. While social media practitioners talk about Like or Follower growth, organizations want to understand how social media is helping drive the business or cause forward. 

Another challenge in social media measurement is the lack of standard definitions, approaches and metrics. In response, a cross-industry push to define social media standards was initiated and initial standards recently published. Social media measurement is clearly at a crossroads where new thinking and approaches are emerging.

In this session you will learn:

  • How to align KPIs and metrics to demonstrate organizational impact and value. 
  • What industry efforts are being made toward standardization and the implications for how you approach social media measurement
  • New models, metrics and frameworks you can use today to develop more effective social media measurement programs.

I will be joined by PRNews Group Editor Matthew Schwartz, who will moderate the discussion and lead a Q&A session.

Here is a link to register for the session. Feel free to leave a comment with any questions you would like answered during the webinar and I will do my best to address them. Hope you can join!

A New Framework for Social Media Metrics and Measurement

12 Jun

Last week in Madrid, AMEC (International Association for the Measurement and Evaluation of Communications) held their 5th European Summit on Measurement. This one was entitled, Unlocking Business Performance – Communications research and analytics in action. One underlying premise of the program this year was that the time for talk is behind us and the time for action and putting into place the principles and practices of sound measurement is upon us. The later part of the program featured an update from Salience Insight Commercial Director Mike Daniels on social media standards including the recently published standardization effort from the cross-industry group called The Conclave which may be found here.

Once Mike discussed where we are with respect to standardization, Richard Bagnall (@richardbagnall), Chairman of the AMEC Social Media Measurement Group, and I as his vice-chair, presented a session on creating a new recommended framework for social media metrics and measurement. Essentially we tried to answer the question: how do we take the standards work coming from The Conclave and operationalize it to create proper social media measurement? Here is an overview of what we presented and what we are encouraging everyone to adopt and use. The framework templates, usage guide and a short video synopsis will be available for download from the AMEC website, Social Media Measurement section, in the next week or so.

Valid Metrics Framework and Social Media

Our journey begins with the Valid Metrics Framework, a measurement planning framework and template developed under the auspices of AMEC. The framework was designed to be flexible enough to address multiple aspects of public relations within a consistent measurement framework and approach.

VMFII

Some of the most positive aspects of the Valid Metrics Framework are that it:

  • Provides a mechanism to link activities to outputs to outcomes
  • Tracks through the familiar sales funnel
  • Helps create a focus on outcomes.

One of the applications of the Valid Metrics Framework was for use with social media programs. Two potential issues were surfaced by early adopters of the framework in social programs. The intermediary effect, which in traditional public relations is the impact on the media, seemed at odds with the social world of direct interaction between consumers and brands, and consumers with each other. And use of the marketing sales funnel, while familiar, was only relevant in a percentage of social media use cases and perhaps not the best way to model common uses of social media like customer relations and building relationships with stakeholder groups. Also, thought leaders like Forrester Research and McKinsey & Company had noted the traditional communications funnel was not necessarily funnel-shaped in social media. They described the discovery process that occurs when investigating companies and brands that often cause the consideration set to expand rather than be reduced, and the fact that a lot of engagement around brands happens post-conversion event. For all of these reasons our task was develop and recommend a better framework and approach.

Models and Frameworks

When we use the word model, we are referring to a representation of a system, in this case social media. In the original Valid Metrics Framework, the model used was the traditional sales funnel. A framework adds additional dimensions to the model and is operationalized with metrics. In the Valid Metrics Framework the additional dimensions are the phases – activities, intermediary effects and target audience effects. We looked at both of these aspects individually and collectively when considering alternative approaches.

We studied and evaluated about fifteen different social media and communications models. A couple of common patterns emerged. Several of the models, including Forrester’s Customer Lifecycle and McKinsey’s Customer Journey showed a post-purchase engagement/experience step. We judged this important to include in our recommended approach. And we considered the importance of Engagement and Influence, as two key concepts in social media marketing and measurement, and decided to try to make these two elements explicit in our model as well.

Suggested Social Media Metrics Model

The model we developed is derivative of the categories chosen by The Conclave (Note: Richard Bagnall and I also participate in The Conclave) to organize social media metrics and definitions. We took a slightly different perspective on the front end of the model and reordered the back-end to create this model for our new framework. The descriptions of the stages use the definitions from the smmstandrards.org work wherever possible.

New Model

You will note Engagement includes both interactions with owned social channels as well as earned social conversation of people ‘talking about you’ in social channels. The definition of Influence is clear and concise, the result of a lot of discussion and prevailing clear thinking. The concept of Impact includes the outcomes of social initiatives as well as the Value those initiatives created. (I usually advise to always attempt to measure impact, and attribute value when it is feasible and makes sense.) Advocacy includes a very helpful definition and conditions that exist with advocacy.

Creating the Framework

To create the framework, we explored various ways to address the ‘phases’ of the Valid Metrics Framework. Two ideas stood out:

  • Use a simple structure that captures social media metrics from three key perspectives – programmatic-level, channel-specific and business. Programmatic metrics are those directly tied to social media objectives. Channel-specific metrics are just that, the metrics that are unique to specific social channels – tweets, RTs, Followers, Likes, Talking About This, Pins, Re-Pins. Business metrics are used to show the business impact of the campaign or initiative.
  • Use Paid, Owned, and Earned media metrics for integrated programs containing these elements. Borrowing the definitions from Forrester, Paid are social channels you pay to leverage (e.g. promoted tweets, display ads), Owned are channels you own and control (e.g. website, Facebook page) and Earned is where customers become the channel (e.g. WOM, viral)

There are certainly other ways to think about this (e.g. Business Performance Management) and we intend to possibly add others based on industry feedback and suggestions.

The AMEC Social Media Valid Framework

Currently we have developed both versions with sample metrics (taken from the smmstandards.org work where applicable). We are calling them The AMEC Social Media Valid Framework. Here is the version with Program, Channel and Business Metrics shown.

Valid Metrics Framwork

Where Do We Go From Here?

Look for the completed frameworks on the AMEC website shortly. We encourage you to adopt the frameworks for use by your company or clients. If you like them and find them useful, please help promote them widely. And please provide your feedback on the proposed framework on this blog, or through the social channel of your choice. We’re listening and looking forward to the dialogue.

Where is Your Organization on the Social Media Listening Maturity Model?

23 Jul

Quite often I am asked to consult with a company on their social media listening strategy. Their initial question more times than not is about the listening platform they should use. But it is increasingly common for the questions to be more sophisticated and the ambition behind them to be much greater. Companies with experience in social listening know that it is all too easy to focus on rudimentary analysis of brand mentions and topics, Followers and Likes and never get to the truly actionable insights that lead to marketing or business actions. Experience in listening is an important element here but you also need a path to follow. I thought a maturity model approach to social media listening could provide a possible path to consider and would provide a construct that could be used in consulting with a company on their social listening strategy.

Maturity models are sort of hot – there seems to be a proliferation in the last two years or so. One that I find particularly insightful and helpful when thinking about social listening is Forrester’s Social Maturity Model.  Two really important points the folks at Forrester make is that listening is not the goal, social intelligence is, and that social intelligence informs actions taken by marketing or some other area of the business. Action being the operative word here. Social intelligence is a closely related topic to social business, and if social business is more your thing the Dachis Group has an interesting social business maturity model.  Big data more your bag? Check out IBM’s big data governance model. After looking at the models out there, I could not find one specific enough to social media listening so I took a stab at creating one.

Social Media Listening Maturity Model 

There are five stages in the Social Media Listening Maturity Model, beginning with reactive alerts and ending with social intelligence. Let’s take a brief look at each stage and some of the overarching differences or changes one sees with social listening maturity.

Reactive Alerts – Many companies or brands begin by establishing a reactive alert system that notifies them whenever their brand is mentioned or is mentioned with specific keywords. Think Google Alerts. Companies in this stage may only periodically check social media channels to see what may have changed or is new since the last check-in.

Monitoring Social Media – At the next stage, the company has begun active monitoring of all ‘owned’ social embassies. They also are monitoring social media conversations, often focused on trying to detect any ‘bad’ news, mentions or conversations.

Companies in these first two stages generally have a reactive stance toward social media, viewing it as another way to find out about news and circumstances that may harm or otherwise impact the organization. It is common for companies in these stages to use one or more of the various free tools available to gather web and social media data.

Social Listening – The third stage is most likely where the largest percentage of companies reside today. Companies in stage three are listening to social conversations about their company, brands and products. They are tracking mentions of competitors and calculating share of conversation. Many also track issues and topics that are important to their brands/products/company. At this stage many begin to put additional emphasis on ‘who’ is talking (source) not just what is being said (post). Most companies in the social listening phase have transitioned from free tools to paid platforms.

Companies in the first three stages often suffer from having too much data and not enough insights. They are up to their necks in ‘big data’ but lack the experience and expertise to analyze the data and reduce it down to crisp, actionable insights supported by the data. They look for the Insight button on the tools they use but increasingly realize insights are the product of human analysts, not tools or data.

Strategic Listening – The transition to strategic listening brings with it a bias toward ‘listening with a purpose’. I first heard this turn of phrase from my friends at Radian6 and use it often. Listening with a purpose is just that – listening to specific sets of conversations with a specific goal or objective in mind. Often in insight work, the goal or objective may take the form of a hypothesis we are trying to test. Here are a few examples of listening with a purpose:

  • Listening for conversations of consumers in a particular phase of the buying decision process
  • Listening to customers whose subscriptions or policies are about to expire that are expressing thoughts of changing vendors
  • Identifying, tracking and building relationships with key influencers
  • Listening for consumer reactions to new packaging or product features
  • Mining the emotional content of specific stakeholder groups to determine potential risk around a sensitive issue.

During this phase, an Enterprise listening strategy is often developed and implemented. Some also begin to integrate data from sources beyond social media – search, web analytics and customer data for example.

Social Intelligence – Forrester defines social intelligence as the process of turning social media data into actionable marketing and business strategy. Social intelligence therefore is not about the best times to tweet or whether or not a twitter party would be an effective tactic, it is about informing strategic decisions that impact the company’s success. For me, three concepts are crucial:

  1. Action – social intelligence is designed to drive true actions.
  2. Integration – although the definition focuses on social media data and insights, the fact is that true insights often require more than just social data. Integrating data from multiple data sources – consumer survey, behavioral tracking, social posts, search analytics, advertising data, customer records, scan/sales data – allows for greater understanding and richer insights. Integration of multiple data types often requires multiple tools and platforms to aggregate and analyze the data.
  3. Sharing – For social intelligence to truly take root within an organization, the data and insights should involve cross-disciplinary groups that can look at the data from different perspectives and collectively arrive at better insights than any one group could in a vacuum. The insights then need to be systematically shared broadly across the organization so they may be acted upon in a manner that will create the most impact. Social intelligence can be a catalyst to the silos within an organization tumbling down.

Since the social listening and social intelligence ‘markets’ are relatively immature, this model will continue to evolve and be refined.

Where is your company today on the social media listening maturity model?

Time to Get Real About Social Media Audience Reporting

12 Jun

Though almost everyone would agree that social media is about engagement and not eyeballs, too much of digital and social media measurement is focused on audience size. How many Followers do we have? How can we get a million Likes? How many unique visitors did we have to our site this month? And unfortunately, audience size estimates in social media grossly overstate the actual relevant audience. We seem fixated and oriented toward ‘how many’, while our focus should be on ‘who’ and specifically, ‘who within our target audience’. Generally speaking, the advertising industry has led the way with audience measures and is ahead of where the public relations and social media camps are with respect to level of sophistication.

In television advertising, the concept of Target Rating Points is a refinement of Gross Rating Points where you only measure and get ‘credit’ for the percentage of the gross audience that meets your target audience criteria. In an effort to keep refining the audience data available, Nielsen has evolved from diary-based data to electronic data to software at the set-top box level that allows operators to monitor channels choices and changes. In audio-based media, Arbitron’s Portable People Meter recognizes today’s mobile world and begins to address cross-platform measurement. It is also interesting to reflect on the U.S. Congressional involvement in television audience ratings accuracy (or lack thereof as it were) that led to the formation of what is now known as the Media Rating Council in the early 1960’s. The time has come for social media audience research to greatly increase in sophistication, accuracy and relevance.

When we think about social media audience size measures today, the emphasis is on Opportunities To See (OTS), although almost never by this name. We might call them Impressions or Reach, but what we really mean is how many people had the potential to see this content item. There are two overarching issues here:

  • Opportunities to see are not the same as actually seeing
  • The metrics count all possible members of the audience, regardless of whether or not they are part of the targeted audience or can even buy the product or service.

OTS is also a prevalent metric in the public relations industry which has always focused on stating the highest possible audience measures. In traditional media we know the probability of any one person in the audience actually seeing the article in question is a fraction of the total audience – a reasonable estimate is 10% or less. So OTS greatly overstates the actually number of people who saw a given article. To compound the audience overstatement, we have the practice of using audience multipliers to ‘credit’ earned media for either a perceived credibility advantage over advertising or to account for pass-along circulation (see this IPR white paper for more on multipliers). Thankfully the practice of applying multipliers (and its evil cousin AVEs) is out of favor and rapidly on a path toward extinction.

In social media one can make the case the audience metrics situation is actually exacerbated in that the probability of any one follower seeing any one tweet, for example, is most likely an order of magnitude less than in earned media – my guesstimate is 1% or less. Before you call BS on this guesstimate, play around with a few Twitter factoids – the recent Pew Research study suggesting only 8% of Twitter users use it daily, the perishable nature of most individual’s twitter streams, and the fact that a reasonably high percentage of Followers of a brand are bots, and the reality is that only a small fraction of twitter followers actually see tweets, let alone find it interesting enough to share or comment on. And, of course, not all Facebook Likes see every post you make either. Riffing on the old, ‘if a tree falls in the forest…’, if you tweet into the twitterverse and no one sees it does it make an impact?

Evolving from ‘opportunities to see’ to ‘relevant audience’ measures.

Most social media campaigns have a specific target audience in mind, often described with demographics (Female, age 18 – 34), psychographics (who worry about feeding their family healthy food on a budget) and behavioral (access deal and coupon sites regularly) dimensions. Yet when it comes to reporting and measurement we take credit for the entire audience (total OTS) rather than the percentage of the audience that meets our targeting criteria. Trying to promote lingerie to 22 – 29 year old ladies? No worries, count all your Twitter Followers and all the visitors to your website – the men, the young and the old – everybody counts. Trying to sell camo clothing to male hunters? No worries, everybody counts – male, female, hunters, non-hunters and PETA members, too. Of course this all seems a little silly and strange and I suppose it would be if it wasn’t the way most social audience reporting is done today. It is unusual to see someone in social media, or PR for that matter, report only the relevant audience opportunities to see. Why is this? I believe there are three primary reasons:

  1. Legacy – the PR industry has historically reported gross potential audience size rather than the relevant audience size. When social media came around, this same orientation toward gross audience measurement was used.
  2. Data – there is a lack of consistent social media demographic and psychographic audience data available and it often resides in channel silos rather than cross-platforms. And often the audience data from one platform (e.g. ComScore) does not match the data available from another platform (e.g. Compete).
  3. Standards – there are no standards for social media audience metrics and no codified best practices for audience measurement.

Where do we go from here?

First, we need a change in mindset of how we think about audiences. From ‘how many people theoretically had the potential to see our content’ to ‘how many of the people we were targeting actually saw our content’. Big audience numbers are irrelevant. Relevant audience numbers are big.

Next, as the demand for audience data that contains demographic, psychographic and behavioral data grows, it is reasonable to assume one or more of the large media data companies might start to aggregate and make the data available. Privacy concerns, cookies and other issues are also in play here.

And last but not least, industry standards for social media audience and engagement metrics and definitions are necessary for transparency and replicability that will increase credibility of social media measurement and reporting. 2012 will go down as the year that serious cross-industry progress on social media metrics standards began and gained momentum. There has already been a lot of progress (See this post from Katie Paine), and this week in Dublin at the 4th AMEC European Summit on Measurement the theme is around attempting to define standards for social media metrics and measurement. To tune into the debate as it occurs in Dublin, monitor #SMMStandards and #AMEC2012.

What  are your thoughts on the need for social media metrics standards and the use of target rather than gross audience size estimates?

Measurement 2020 and Other Fantasies

23 Sep

At the 3rd European Summit on Measurement held in Lisbon in June 2011, standardization, education, ROI and measurement ubiquity emerged as the key themes in response to a call to set the Measurement Agenda 2020.  Delegates to the conference voted on 12 priorities they thought were most important to focus on in the period leading up to 2020.  The top four vote-getters became the Measurement Agenda 2020:

  1. How to measure the return on investment of public relations (89%)
  2. Create and adopt global standards for social media measurement (83%)
  3. Measurement of PR campaigns and programs needs to become an intrinsic part of the PR toolkit (73%)
  4. Institute a client education program such that clients insist on measurement of outputs, outcomes and business results from PR programs (61%)

For a very nice overview of the Lisbon session and the Barcelona Principles that came before, read this post from Dr. David Rockland of Ketchum who chaired the Barcelona and Lisbon sessions.  David pretty much said it all on these sessions, so I’ll just add a couple of comments and share a few thoughts on what I believe the future of measurement 2020 could be.

The rallying cry coming out of Barcelona has been focused and loud – death to AVEs!  Will there be a similar thematic coming out of Lisbon and what might it be?  My money is on standardization, borne out of cross-industry cooperation.  As David points out in his post, and in the words of AMEC Chairman Mike Daniels, “The Summit identified some significant challenges for the PR profession to address by 2020.  However, what we also accomplished in Lisbon beyond setting the priorities was to harness the commitment and energy of the industry to agree what we need to do together.”  The current cooperation and collaboration between industry groups – AMEC, Institute for Public Relations, PRSA and the Council of PR Firms is unprecedented in my time in this industry and is focused on tangible outcomes.  Cross-organization committees are already at work developing standard metrics for social media measurement for example.  The spirit of cooperation is uplifting.  While the outward thematic appears to be standardization, cooperation is the enabling force.  

I was also struck by the symmetry of the call to end AVEs in Barcelona and the call to codify ways to measure ROI in Lisbon.  One follows the other.  In my opinion the primary reason AVEs exist is because PR practitioners feel pressure to prove the value of what they do, and quite often they are asked to describe the impact in financial terms.  AVEs are perceived as a path of least resistance way to express financial value.  Except, as we all know, AVEs don’t really have anything to do with the impact public relations creates.  They are a misguided proxy for financial value.  Hence the need for research-based methods to determine true return on investment.

All of the priorities coming out of Lisbon are excellent goals for the industry.  And like David Rockland, I believe they will be achieved, and be achieved before 2020.  Here are three other items on my wish list for Measurement 2020:

Word of Mouth/Word of Mouse Integration: For those of us focused in social media and other digital technologies, we can’t allow our digital lens to color what is fundamentally an analog world.  Research studies suggest the majority of word of mouth happens in real life.  From an influence perspective, I don’t think too many would argue that word of mouth from a trusted friend or family member is more powerful than word of mouse from someone you follow on Twitter.  Digital cross-platform research is difficult enough, but when one huge platform is ‘real life’, we have significant challenges in measurement.  WOMMA and others have made early attempts to define measurement approaches for offline WOM, but much work remains.  We need ways to assess its impact and then we need to think about ways to attribute value to that impact.  Mobile is a wild card here as it becomes the preferred platform for online activity.  The need to triangulate online, mobile and ‘real life’ measurement presents significant challenges today, and may still by 2020.

Cookie Wars: We all know the measurement versus privacy showdown is coming, right?  The first shots have already been fired.  The collection of source-level personal data, enabled by cookies, is crucial to measurement and insights but has the potential for misuse or unintended disclosure.  Some sophisticated consumers have had their fill of cookies.  Although the broader issue might be framed as social sharing versus privacy control, how it plays out will have a direct impact on digital analytics and measurement.

Integrated Measurement across the Paid Earned Shared Owned (PESO) Spectrum: Measurement has increasingly become integrated.  It began with integrated traditional (Earned) and social media (Shared) measurement and then progressed rapidly to Earned, Owned and Shared, which is where most integrated measurement programs are today.  Many leading-edge integrated programs today also include advertising or Paid media.  By 2020, integrated measurement across the PESO spectrum will most likely be the norm and not the exception.  A key enabling element here in my view is some base level of agreement on how each area should be measured and standard metrics for each.  It will take significant cooperation between industry groups, vendors, agencies and major customers/clients for cross-discipline standardization to move forward effectively.  We are at the beginning of this movement in 2011.  By 2020, it will be fascinating to look back and see how all this plays out.

When looking ahead to 2020, I am reminded of a measurement discussion pulled together by PRWeek a couple of years ago.  Many of the Measurati attended.  In response to a question of where measurement will be in five years, David Rockland replied (paraphrasing here), ‘Who knows?  Five years ago who would have guessed we would all be focused on how to measure social media?’  So, there is a certain fantasy element to discussing 2020 challenges in measurement.  What are your measurement fantasies?

AVEs Don’t Describe the Value of Media Coverage, They Sensationalize It.

26 Jun

Saturday, Wall Street Journal columnist Carl Bialik, The Numbers Guy, addressed the subject of advertising value equivalency (AVE).  This is perhaps the first example of a mainstream media publication shining a light of the controversial practice of AVEs.  (You can read the story here.)

The primary reason advertising value equivalents exist are because they are perceived to be a way to attribute value to programs that would otherwise be difficult to value directly.  They are a path of least resistance approach to return on investment calculations, but not a valid one.  Let’s take a deeper dive into the three specific examples in the WSJ story, ask the tough questions and discuss more valid ways to think about value attribution and ROI.

American Airlines  

You can enjoy both questionable valuation techniques and hyperbole in this article.  American Airlines stands to “make boatloads of cash” and “the airline company could gain as much as $95.9 million of exposure”.  Of really, let’s take a closer look.

The most incredible part of this financial calculation is the financial calculation itself.  The calculation is apparently based on sign placement within the arena and presumably the ‘impressions’ the brand will receive when people attending the venue see the signage and when TV cameras catch the signs when showing the scoreboard or during the action.  This is a very passive form of advertising that should have as its objective either creating top of mind awareness or perhaps creating more brand affinity.  Rather than using an advertising equivalency model that has no validity, a true measurement of the value created by naming rights would ask a series of questions designed to determine the actual, tangible (or even intangible) impact on the business:

  • Revenue: Can incremental revenue generation in the form of higher passenger miles be directly attributed to the exposure created by the naming rights?  Is it possible that incremental revenue would actually be realized on a game by game basis, or would any positive impact be realized over a longer time horizon?  Have new customers been created as a direct result of the exposure generated by the naming rights?
  • Brand: Can the increased exposure lead to people perceiving the brand differently and can the difference translate into higher transactional revenues generated or increased brand loyalty?

So where exactly are the ‘boatloads of cash’ American Airlines made?  Are they hitting the income statement in the form of incremental revenue or enhanced brand loyalty (repeat business)?  Are they residing on the balance sheet in terms of brand goodwill?  Given that American’s parent company AMR lost $11.5B dollars in the first decade of the 21st century, its last profitable year was 2007 and they are projected to lose money in 2011 and 2012, they could use the cash.  Perhaps they could use it to fund a ’bags fly free’ program or for enhancing their Advantage program to create more brand loyalty.  I would strongly suspect American’s shareholders would prefer a do-over on the investments made on naming rights to the ‘boatloads of cash’ they are now enjoying from the investment.

Couple Won’t Cash In on Kiss

15 minutes of fame is rarely worth $10 million.  In this case, the celebrity agent is suggesting the news value of the coverage generated by the kiss is somehow equivalent to advertising value and assigns what appears to be an arbitrary and ridiculously high value to it.  (He later admits he just made the number up.)  Just how was the couple going to monetize their 15 minutes of fame?   Yes, they turned down a few talk show opportunities and perhaps the National Enquirer would have thrown a few dollars their way for an exclusive, but the assertion that any major brand would have paid them to endorse their product is wildly speculative.  I would guess that if you did a survey after the event, a small number of people would remember seeing the coverage, and a very small percentage of the people who did see it would have recalled Scott Jones’ name.  So perhaps Mr. Jones walked away from tens of thousands of potential dollars in the short-term, but nowhere near the sensationalized estimate of $10 million.  15 minutes of fame might be worth 10 thousand dollars, but certainly not $10 million.

Obama Enjoys a Guinness

So Guinness is a winner and received $20 million worth of “free publicity”?  What was the outcome of the publicity?  Again, in order to determine the value of the “free publicity” (this term is despised in the PR industry by the way), Guinness would have to be able to measure incremental revenues directly attributable to the publicity generated.  Did sales of Guinness increase as a result?  Were new customers created?  Did existing customers feel compelled to drink even more?  What was the value of the incremental sales?  These are much more difficult questions to answer but are the correct ones to ask in order to measure the publicity.  Not by focusing on the mythical value of the coverage as measured by flawed advertising equivalency, but measuring the outcome or what happened as a result of the publicity.  The assertion that President’s Obama’s image was softened and will help keep him in the public’s favor is highly dubious thinking.  Perhaps it helps him in Boston, but in the grand scheme of things, this is a Presidential image non-event.

Beginning last Summer in Barcelona,  the public relations industry has come together to publicly state advertising value equivalency is not a valid measure of public relations.  The so-called Barcelona Principles are explicit against AVEs and also call for a focus on measuring outcomes and not (just) outputs.  While it will take some time for the PR industry to totally leave AVEs behind, there is a lot of momentum right now to make this happen sooner rather than later.  No serious measurement effort can use advertising value equivalency to attribute value and be credible.  

Social Media Measurement 2011: Five Things to Forget and Five Things to Learn

30 Dec

It has been said that social media came of age in 2010.  Not so for social media measurement.  But the mainstreaming of social media marketing brings with it a heightened call for accountability.  The need to prove the value of social media initiatives has never been greater.  So, perhaps 2011 will be the year that social media measurement matures and comes of age.

As we look to the next year, here are five things to forget and five things to learn about social media measurement in 2011.

Things to Forget in 2011

1. Impressions

The public relations industry has historically measured and reported success through the lens of quantity not quality.  The most common PR metric today is Impressions.  While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek.  Impressions also (greatly) overstate actual relevant audience.   Impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content.

For Twitter, many folks use the sum of all first generation followers as ‘impressions’ for a particular tweet.  The obvious problem here is that the probability that any one follower sees any one tweet is quite small.  I don’t have good data on this (please share if you do), but an educated guess might put the percentage at less than 5%.  Similarly for Facebook, use of impressions as a metric is also problematic.  Facebook impressions do not indicate unique reach and you don’t have any idea who, if anyone, actually viewed the content.

Number of Impressions is a flawed, unwashed masses metric for social media measurement.  Any time you are tempted to use the word ‘impressions’ in social media, think about ‘potential reach’ or ‘opportunities to see’ instead.  Or better yet, concentrate on Engagement and Influence.

____________________________________________________________

2. Vanity Metrics – Fans and Followers

Most social media measurement efforts place far too much emphasis on Fans/Likers and Followers.  For Twitter, the number of Followers is seen as a key metric, thought by many to relate to potential influence.  For Facebook it is the number of Fans/Likers many companies/brands attempt to maximize.  While these may be the vanity metrics of choice, they fall far short of being adequate for rigorous measurement.  The largest disconnect of course is these numbers really don’t describe potential audience size very well and they have nothing to do with interactions/engagement.

For Twitter, there is a growing amount of evidence (read the Million Follower Fallacy paper) that number of Followers really has little to do with Influence.  Number of Followers may be an indication of popularity but not influence.  Influence talks more to one’s ability to start conversations and spread ideas.  For Facebook, number of Fans bears little semblance to average daily audience size and tells you nothing about engagement of the community.  All Fans are not created equally.  Some are engaged, some never return.  Some are your best customers, others are there only to trash you.

Number of Fans and Followers are metrics you probably should include in your overall metrics set, but should be de-emphasized and not be a primary area of focus.

________________________________________________________

3. Standardization

Measurement standardization is always an interesting topic to debate.  On one side you have the folks who believe standards are absolutely necessary for measurement to proliferate, and on the other side you have the snowflake measurement disciples who believe each program is unique and therefore requires unique objectives/metrics.  I fall somewhere between the two extremes.

In June 2010 IPR, AMEC, PRSA, ICCO and The Global Alliance got together in Barcelona for a conference intended to create an atmosphere for measurement consistency/standardization around a codified set of principles of good measurement.  The Barcelona Principles as they have come to be called are basic statements of good measurement practice – focus on outcomes not outputs, don’t use AVEs, etc.  Absolutely nothing to disagree with in the Principles.  However, the heavy lifting of standardization comes at the metrics-level.  Subcommittees have been formed that are taking the Principles all the way down to the metrics level.  I have reviewed the work of the social media committee and believe there is a lot of good work being done.

But in 2011, I expect a lot of debate but not a lot of progress in creating social media measurement standardization.   One to watch is the Klout score for online influencers which is being integrated as metadata in social media listening and engagement platforms.  There are issues with the Klout score (read this post), and I question the type of ‘influence’ it is measuring – there is a big difference between motivating someone to action (e.g. retweeting your content) and motivating someone to purchase which is ultimately the type of influence many companies and brands are most interested in effecting.

__________________________________________________________

4. Ad or Media Equivalency

One of the truly insidious aspects of public relations measurement is the use of advertising or media equivalency (AVEs – advertising value equivalency) to assign financial value to public relations outputs.  It is a highly flawed, path of least resistance attempt to calculate return on investment (ROI) for public relations.  There are many reasons why using ad equivalency as a proxy for PR value is not advisable.

To make matters worse, the practice has clearly moved into social media measurement as well.  For example, research studies that monetize the value of a Facebook Fan/Liker by attributing an arbitrary $5 CPM value from the advertising world.  Online media impact rankings also utilize equivalent paid advertising value to assign monetary value to online news and social media.  The true value of social media is not how much an equivalent ad would have cost but in the impact it has on brand, reputation and marketing.

__________________________________________________________

5. Return on Engagement/Influence/etc.

Not a day goes by without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks.  A recent Google search for “Return on Engagement” returned 192,000 results.  “Return on Influence” returned 68,300.

Most of the folks who use these terms either don’t understand ROI or don’t know how to obtain the data necessary to calculate it.  Many confuse the notion of impact with ROI (addressed in Things to Learn).  Engagement creates impact for a brand or organization, but may or may not generate ROI in the short-term.  Creating influence – effecting someone’s attitudes, opinions and/or actions – creates impact but may or may not create ROI in the short-term.  It often is better to think about measuring impact first and then deciding whether or not you have the means and data necessary to attribute financial value.

__________________________________________________________

Things to Learn in 2011

1. Measurable Objectives

There are many issues and challenges in the field of social media measurement.  The easiest one to fix is for everybody to learn how to write measurable objectives.  Most objectives today are either not measurable as written or are strategies masquerading as objectives.  (For example, any sentence starting with an action buzzword like leverage is a strategy.)

‘Increase awareness of product X’ is not a measurable objective.  In order to be measurable, objectives must contain two essential elements:

  • Must indicate change in metric of interest – from X to Y
  • Must indicate a timeframe for the desired change – weeks, months, quarter, year, specific dates tied to a campaign (pre/post)

Therefore, properly stated, measurable objectives should look more like these:

  • Increase awareness of product X from 23% to 50% by year-end 2011
  • Increase RTs per 1000 Followers from 0.5% in Q1’11 to 10% by the end of Q2’11.

__________________________________________________________

2. Impact versus ROI

ROI is one of the most overused and misused term in social media measurement.  Many people say ‘ROI’ what they really just mean results or impact.  ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.

ROI is a form of impact, but not all impact takes the form of ROI.  Impact is created when people become aware of us, engage with our content or brand ambassadors, are influenced by engagement with content or other people, or take some action like recommending to a friend, writing a review or buying a product.  Impact ultimately creates value for an organization, but the value creation occurs over time, not at a point in time.  Value creation is process-oriented.  It has both tangible and intangible elements.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

Most social media initiatives today do not (or should not) have ROI as a primary objective.  Most social programs are designed to create impact, not ROI, in the short-term.  There is also the notion that many social media initiatives are in an investment phase, not a return phase of maturity.

__________________________________________________________

3. Hypothetical ROI Models

One important step in determining how a social media initiative creates ROI for an organization is to create a hypothetical model that articulates the cascading logic steps in the process, as well as the data needed and assumptions used.  The model is most useful in the planning stages of a program.  It helps address the proverbial question, “If I approve this budget, what is a reasonable expectation for the results we will achieve?”  Let’s take a look at a simple Twitter example:

Program: Five promoted tweets are sent with a special offer to purchase a product on an e-commerce site.

Hypothetical ROI Model:

  • (Data)                   Total potential unduplicated reach of the five tweets is 1,000,000 people
  • (Assume)            10% of the potential audience will actually see the tweet = 100,000 people
  • (Assume)            20% of the individuals who see the tweet find it relevant to them = 20,000 people
  • (Assume)            10% of those finding it relevant will visit the site = 2,000 people
  • (Assume)            10% of those visiting the site will convert and buy the product = 200 people
  • (Data)                   Incremental profit margin on each sale is $50
  • (Data)                   Total cost of the social media initiative is $2,400

ROI Calculation: (200 x $50) = $10,000 – $2,400 = $7,600/$2,400 = 3.17 x 100 = 317% ROI

Our model suggests this program will be successful and generate substantial ROI.  If in reviewing a model with someone who needs to approve a program, they conceptually buy into the model but challenge the assumptions, that is a positive step.  Negotiate different assumptions and rerun the numbers.  Hypothetical models help you think through the data requirements your research approach must address in order to actually measure the ROI of the program after implementation.

__________________________________________________________

4. Integrated Digital Measurement

The definition of public relations is fluid, and rapidly evolving to encompass a much broader and more integrated view of communications and how we connect, engage and build relationships with consumers and other stakeholders.  Digitization in all its forms has driven and accelerated this important change.  Communicators should now take a more content and consumer-centric view of the world, orchestrating all the consumer touch points available in our increasingly digital world.  At Fleishman Hillard, we capture this expanded scope and integration in a model we refer to as PESO – Paid/Earned/Shared/Owned.  Here is how we define the elements of our model:

Paid – refers to all forms of paid content that exists on third-party channels or venues.  This includes banner or display advertisements, pay-per-click programs, sponsorships and advertorials.

Earned – includes traditional media outreach as well as blogger relations/outreach where we attempt to influence and encourage third-party content providers to write about our clients and their products and services.

Shared – refers to social networks and technologies controlled by consumers along with online and offline WOM

Owned – includes all websites and web properties controlled by a company or brand including company or product websites, micro-sites, blogs, Facebook pages and Twitter channels.

The social media measurement Holy Grail in many ways is to be able to track behavior of individuals across platforms, online and offline, tethered and mobile, understanding how online behavior impacts offline behavior and vice-versa.  We also seek to understand how the PESO elements work together synergistically.  For example, how exposure to online advertising impacts conversions within social channels.  To address this, your measurement strategy should be to take a holistic, integrated approach using a variety of methodologies, tools and data.

_________________________________________________________

5. Attribution

If you are not already familiar with value attribution models, prepare to hear much more about them in 2011.  Value attribution models attempt to assign a financial value to specific campaigns and/or channels (e.g. advertising, search, direct, social) that are part of a larger marketing effort.  So rather than giving all the conversion credit to the last click in a chain or even the first click, the model attributes portions of the overall value across the relevant campaigns and/or channels.

A simple model might look at the following metrics for each channel:

  • Frequency – the number of exposures to a specific marketing channel or campaign
  • Duration – time on site for exposures referring to the conversion site
  • Recency – credit for exposures ranging from first click to last click, with last click typically receiving more credit.

Value attribution models require human analysis and expertise.  This factor is often cited in studies as the reason more companies do not pursue attribution modeling.

_________________________________________________________

Here’s wishing you and yours an exciting and prosperous 2011!

The Barcelona Principles: Leaders Speak

23 Jun

Well, the Second European Summit on Measurement held last week in Barcelona has come and gone, but its impact may be felt for some time to come.  The Summit was organized by the International Association for the Measurement and Evaluation of Communication (AMEC) and the Institute for Public Relations.  The most notable outcome of the Summit was the creation of the ‘Barcelona Declaration of Research Principles’.  The Principles were debated and voted upon by about 200 delegates representing 33 countries and five global PR and measurement organizations (AMEC, IPR, PRSA, ICCO, The Global Alliance).  David Rockland, Ph.D. chaired the debate.

Here are the ‘Barcelona Declaration of Research Principles’:

1. Goal setting and measurement are fundamental aspects of any PR programs.
2. Media measurement requires quantity and quality – cuttings in themselves are not enough.
3. Advertising Value Equivalents (AVEs) do not measure the value of PR and do not inform future activity.
4. Social media can and should be measured.
5. Measuring outcomes is preferred to measuring media results.
6. Business results can and should be measured where possible.
7. Transparency and Replicability are paramount to sound measurement.


I asked three of the leaders of the conference to comment on four questions regarding the Summit and what it may mean for the future of measurement.  The leaders are:

Barry Leggetter, FPRCA, FCIRR is Executive Director of AMEC  (barryleggetter@amecorg.com)

Pauline Draper-Watts, is Chairperson of the Institute for Public Relations, Commission for Public Relations Measurement and Evaluation  (pauline.s.draper@gmail.com)

David Rockland, PhD, is Partner/CEO, Ketchum Pleon Change and Managing Director, Global Research (David.Rockland@ketchum.com)

Here are their thoughts on the Summit:

Q1. For those not able to attend the Summit, how would you briefly describe what they missed?

BL: A milestone moment when delegates from 33 countries agreed to take program measurement more seriously, starting with the abandonment of AVE’s

DR: Missed a great opportunity to network with colleagues from 33 countries, hear some engaging speakers, and be part of a moment in time where the industry first adopted a set of measurement principles.

Q2. From your perspective, what are the two or three most significant outcomes of the Second European Summit on Measurement?

BL:

  • For AMEC to be successful in getting five global organisations on the same platform for the first time and talk from the same page about the need for the PR and media intelligence industry to act – not just talk – about improved methods of program measurement.
  • That the Summit achieved its own break-through status in receiving speaker support from senior level clients from global organisations such as FedEx Corporation, Yahoo, Royal Philips Electronics, Nissan, Telefónica, Banco Santander and others.

PD-W: The percentages (voting) in favor for each of the Barcelona Principles following the discussion

DR: To me a significant outcome was a gathering of the industry in a manner where ideas were shared, friendships and partnership extended, and we agreed as an industry to look ahead to how we can do what we do better and professionalize the practice of public relations.

Q3. How do you hope agencies, companies and organizations operationalize the seven principles?

BL: I introduced quality management processes when a Director of Porter Novelli in the UK in the 90’s – the first agency in the world to make this commitment. It became part of our agency’s way of working. I hope agencies, companies and organisations will similarly make the same commitment to the Barcelona Principles and introduce more stretching methods of program measurement on all programs.

PD-W: Integrating them into the culture and corporate language within the organization so that they are lived out in practice.

DR: My hope would be that first the principles are widely talked about and become SOP for what we do. Second, that the term AVE disappears along with the incredibly counter-productive debate around this subject that has distracted the industry from its own development. And third, that each organization adapts the Principles into their own words and practices; when we see them translated into the languages of the 33 countries represented at the Summit, we’ll know it worked.

Q4. Please complete this sentence: A year from now, we will know the Second European Summit on Measurement was successful …….

BL: …if when I judge my next PR Awards schemes PR consultancies and company in-house PR teams are putting more effort into the Program Measurement heading on the award entry as they are now doing to demonstrating creativity!

PD-W: … the Barcelona principles (modified to reflect the comments made at the Summit and submitted afterwards) are not only adopted but also put into practice throughout the industry.

DR: …if we don’t hear PR practitioners continue to complain we don’t have a “seat at the table” because we lack the metrics and measurement approaches other disciplines have.

Keeping it real…and transparent

  • I am a member of the Institute for Public Relations, Commission on Public Relations Measurement & Evaluation
  • My agency, Fleishman Hillard, is a member of AMEC
  • Ketchum is a sister Omnicom agency

Will Barcelona Measurement Debate Shake Up the Industry?

26 May

In mid-June, the Second European Summit on Measurement will be held in Barcelona.  The Summit is jointly organized by the International Association for Measurement and Evaluation of Communication (AMEC) and the Institute for Public Relations.  The highlight of the conference will be a debate to “articulate and agree on standard metrics and measurement techniques…” according to a press release issued last week.

David Rockland, head of research for Ketchum, will lead the session.  Mr. Rockland has a lofty vision and high hopes for the debate, “We regard this as the industry’s ‘Commitment Conference’.  This is a very powerful moment in time in the history of public relations.  Until now, public relations has been undervalued due to its inability to measure itself.  The goal of this summit is to establish consistency in order to increase credibility.”

The three-day conference is expected to be attended by about 150 people, including many measurement thought leaders.  It should be interesting to see what comes out of this event.  Will it serve as a wake-up call to the industry?  Time will tell.  I’ll post some thoughts once the summit concludes.

Keeping it real…and transparent

  • I am a member of the Institute for Public Relations, Commission on Public Relations Measurement & Evaluation
  • My agency, Fleishman Hillard, has agreed to join AMEC (pending paperwork)
  • Ketchum is a sister Omnicom agency

Public Relations Measurement 2010: Five Things to Forget & Five Things to Learn

29 Jul

(This post is a re-purposing of a speech I gave to the FPRA/PRSA-Orlando on July 23, 2009.  You can download the slides here.)

Public relations measurement is at a crossroads.  Old techniques are no longer sufficient.  Old metrics are no longer applicable.  Old thinking must be replaced by new.  The need for accountability, and to prove the value of PR and social media programs, has never been greater.

As we look to the next year, here are five things to forget and five things to learn about public relations measurement in 2010.

Things to Forget in 2010

1. Media Relations Focus

A focus on media relations fails to capture several important aspects of PR – brand, reputation, crisis, employee communication and DTC to name a few.  Also, the importance of traditional media is declining.  Numerous studies have shown people don’t trust what they read in the media, they trust each other.  I believe it was Hauser and Katz who coined the term ‘you are what you measure’ in 1998.  If measurement is focused on media relations that is how the public relations function will be judged.

__________________________________________________________

2. Outputs

The need to put PR results in a business context has never been greater.  We need to be able to address the question – what are we doing to help drive the business?  If you are focused on output metrics like impressions or message delivery, you will always have a hard time explaining business impact.  Instead, we need to focus on outcomes and answer the question – what happened as a result of our program or coverage?  Understanding outputs has primary benefit as a diagnostic tool rather than a ‘scorecard’.

__________________________________________________________

3. Impressions (and Multipliers!)

The most common PR metric today is Impressions.  While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek.  Impressions also (greatly) overstate actual relevant audience.  Generally only a fraction of any particular magazine or newspaper’s circulation meets your target audience demographics.  And impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content.  To compound the problems, many PR practitioners use a multiplier on impression numbers to account for pass-along readership or a mythical credibility advantage PR has over other communication tools.  The simple fact is there is no factual basis (e.g. research proof) that multipliers should be used in any case.

__________________________________________________________

4. Ad Equivalency (AVEs)
There are many reasons why using ad equivalency as a proxy for PR value is not advisable.  Here are five good reasons they should be avoided:

  • AVE calculations vary and there are no standards.  Tonality, article length, competitive mentions and other factors are handled differently.
  • AVE results can be misleading.  AVEs may be trending up while metrics like message communication, share of favorable positioning and share of positive press are falling.
  • AVEs reduce PR to just the media dimension by only assigning a value in this area.
  • AVEs only apply to traditional media.  What is the AVE of a positive conversation about your company on a leading blog?
  • How much is it worth for a troubled company to not appear in the Wall Street Journal?  AVEs cannot address this.

__________________________________________________________

5. Return on (Engagement/Influence/etc.)
Not a day goes by on Twitter without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks.  Wrong and wrong.  Most of these folks either don’t understand ROI or don’t know how to obtain the data necessary to calculate it.   There is also a lot of confusion between creating value and ROI.  Generating awareness creates value, for example, but may not immediately result in demonstrable ROI.

__________________________________________________________

Things to Learn in 2010

1. Total Value of PR

Microsoft PowerPoint

The majority of current PR measurement efforts focus on marketing/sales and output metrics.  The Total Value Cube is a way to visualize and think about all the potential value your PR and social media efforts deliver.  Beyond marketing to include brand and reputation, beyond outputs to include engagement, influence and action, and beyond revenue generation to include cost savings and cost avoidance.

__________________________________________________________

2. A New Model for Measurement
Many public relations practitioners regularly get their Outputs confused with their Outtakes or Outcomes.  Outtakes is not often used in the U.S. – it seems much more prevalent in Europe.  The overall terminology is confusing and is defined in different ways by different practitioners.  Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.

What we need is a metrics taxonomy that is easier to understand and explain.  I like this one.

Social Media Model.pptx

Exposure – to what degree have we created exposure to content and message?

Engagement – who, how and where are people interacting/engaging with our content?

Influence – the degree to which exposure and engagement have influenced perceptions and attitudes

Action – as a result of the PR/social media effort, what actions if any has the target taken?”

__________________________________________________________

3. Three Zones of Measurement

PRSA.FPRA.07.23.ppt-3

From the left, companies or brands control, own or manage websites  – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with.  This zone is measured primarily by web analytics.  In the middle are the actual social networks and conversations between individuals.   In this zone we are interested in data sets that cannot be gathered solely using web analytics packages.  How often is the brand being mentioned in conversation?  What is the sentiment of the comments?  How often is the brand being recommended and by whom?  Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone.  The third zone represents all the real-world, offline transactions that may be of interest.  Did someone visit the store or attend or event?  Did they buy a product?  Did they recommend the brand or product to a friend over coffee?  Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.

Your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones.  The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.

__________________________________________________________

4. New Metrics

PRSA.FPRA.07.23.ppt-2

_________________________________________________________

5. The Difference Between Impact/Value and ROI
ROI is a form of value/impact, but not all value takes form of ROI.  ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.  ROI lives on the income statement in business terms.

Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product.  Value creation occurs over time, not at a point in time.  Value creation is process-oriented.  Value lives on the balance sheet.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

Follow

Get every new post delivered to your Inbox.

Join 160 other followers