Archive | Industry News & Views RSS feed for this section

Social Media Metrics & Measurement Continue to Evolve

9 Jun

This week on 11/12 June, AMEC is holding their next International Summit on Measurement. Many of you will recognize AMEC as the framer of the Barcelona Principles at their annual meeting in 2010. The theme this year is upping the game to deliver relevant insights along with traditional measurement reporting on performance against objectives and KPIs, in order to provide a richer environment and context for making strategic business decisions.

summit-header

 

 

In this post, I wanted to shine the light on a workshop during the Summit called, Metrics that matter: Making sense of social media measurement. The session, led by Richard Bagnall, promises to look at the latest social media measurement trends, provide a look ahead at what might be right around the corner in the next 12 months, and unveil a revised and enhanced AMEC social media measurement framework guide that should make it easier to implement the frameworks in your own planning environments.

At last year’s Summit in Madrid, we presented the initial work in developing models and frameworks to support rigorous and valid social media measurement. It consisted primarily of three elements – a new model for social media measurement, a couple of alternatives ways to think about populating the model with relevant metrics, and a social media measurement planning framework/template. The initial metrics approaches focused on two alternatives, metrics focused across programmatic, channel and business dimensions as well as an approach based on the Paid, Earned, Shared and Owned integrated channel metrics approach which enjoys traction in many organizations. Based on using and customizing the approach for multiple clients, here are a few things to consider as you review and think about using the new frameworks and usage guidance to be unveiled this week.

Model.PESO

As campaigns become increasingly integrated across media types it makes sense to also reflect this integrated view in measurement. Ideally, measurement should reflect a similar level of integration as the campaign or program being measured. As we measure performance from the four channels, we should keep in mind what we really would like to understand is how the channel efforts amplify or build on each other.

The work presented at last year’s Summit showed example or illustrative metrics for each of the media types across the measurement model. Expect this year’s version to take a stronger view of the most relevant or best metrics to use when employing this approach.

Another key development would be the presentation of different metrics models to use with the measurement model. The intention all along was there could be a range of choices to fit different corporate cultures and planning environments.

 

Measurement Planning Template

The measurement planning template is the heart of your social media measurement planning effort. It is best used by conducting a facilitated discussion with all measurement stakeholders around each of the key elements in the planning template. Setting aside half-a-day to complete the exercise is not excessive. Here is how we think about using the template in our social media measurement planning.

Planning.Framework

Business or Organizational Objectives: The agreed upon overarching organizational objective(s) the social media effort is designed to impact. These objectives may be given to the social media team or they may be the result of conversations and negotiations.

KPIs (Key Performance Indicators): One or two high-level metrics that are aligned with the business/organizational objectives and are outcome-oriented. Again, the KPIs may be given to the measurement team or the measurement team can help guide the conversation to develop them.

Program Elements: Outline the major elements of the program. The elements should reflect the scope and integration of the campaign. They could range from a simple social media program to one that includes social media, online advertising, influencer outreach, e-mail marketing and other elements.

Program Objectives: Capture or write the measurable objective associated with each major aspect of the program above. Frequently you will need to help rewrite the metrics to make them measurable. You may also need to rewrite them to actually make them objectives (what) and not strategies (how).

Measurement Story: This is an attempt to marry the concepts of measurement and story telling. At the end of the program, what measurement story would you like to be able to tell your key stakeholders? This should be based on accomplishing your specific KPIs through the success of the programs used and your ability to prove that through data and measurement. We typically develop the measurement story right after settling on the business objectives and KPIs, and use it as a guidepost to ensure we have the means and methods to tell the desired story.

Key Metrics: The metrics directly tied to program objectives. Should be aligned with the objectives as well as the higher order KPIs. The most important KPIs and metrics and often captured on a dashboard for monitoring and/or reporting.

 

Determining the importance of metrics

A question sometimes comes up about the best way to determine the most important metrics – what rises to the level of inclusion on a high-level dashboard? Here’s a few tips to consider:

Place metrics in their respective place in the measurement model (Exposure/Engagement/Influence/Impact/Advocacy) Metrics that appear toward the right side of the model are generally more compelling than those addressing just Exposure or Engagement. These generally get to the outcomes rather than just outputs of program activities.

Examine how closely aligned each metric is with program objectives. Metrics that directly support the objective are generally more important than those that indirectly support the objective.

Look at the degree to which the metric is explicitly part of the Measurement Story. Metrics directly aligned with the Measurement Story represent better potential dashboard metrics than those that are tangential to the story.

Metrics that provide context are generally stronger than those that don’t. For example, RTs per 1000 Followers tells you much more than just the Number of RTs – it gives an indication of community engagement. Likewise, Engagement Rate (Number of Engagements divided by Total Reach) is a better metric that just the Number of Engagements.

Remember the audience – at the end of the day, when you’re showing your stakeholders what you accomplished this year, what would they be most excited to hear? What will they want to see that ultimately demonstrates the best use of their dollars?

 

After the Summit I’ll post a reaction to Professor Jim Macnamara’s unveiling of his new his new paradigm and model for measurement and evaluation. Jim is incredibly smart and thoughtful so I’ll be curious to see and discuss what he proposes. Enjoy the Summit in person or through social media and contribute to the dialogue.

Thanks for reading.

 

 

I would like to acknowledge our colleagues at R/GA for their contributions to the thinking you see here.

Social Media Measurement at a Crossroads

25 Jul

Please join me on August 6, from 2:00 – 3:00pm EDT for CARMA’s fourth quarterly webinar, co-sponsored by PRNews, titled “Social Media Measurement at a Crossroads.” Here is a little more information on the webinar:

With social media clearly entrenched as a mainstream business activity, the need to measure the impact on the organization has never been greater. While social media practitioners talk about Like or Follower growth, organizations want to understand how social media is helping drive the business or cause forward. 

Another challenge in social media measurement is the lack of standard definitions, approaches and metrics. In response, a cross-industry push to define social media standards was initiated and initial standards recently published. Social media measurement is clearly at a crossroads where new thinking and approaches are emerging.

In this session you will learn:

  • How to align KPIs and metrics to demonstrate organizational impact and value. 
  • What industry efforts are being made toward standardization and the implications for how you approach social media measurement
  • New models, metrics and frameworks you can use today to develop more effective social media measurement programs.

I will be joined by PRNews Group Editor Matthew Schwartz, who will moderate the discussion and lead a Q&A session.

Here is a link to register for the session. Feel free to leave a comment with any questions you would like answered during the webinar and I will do my best to address them. Hope you can join!

A New Framework for Social Media Metrics and Measurement

12 Jun

Last week in Madrid, AMEC (International Association for the Measurement and Evaluation of Communications) held their 5th European Summit on Measurement. This one was entitled, Unlocking Business Performance – Communications research and analytics in action. One underlying premise of the program this year was that the time for talk is behind us and the time for action and putting into place the principles and practices of sound measurement is upon us. The later part of the program featured an update from Salience Insight Commercial Director Mike Daniels on social media standards including the recently published standardization effort from the cross-industry group called The Conclave which may be found here.

Once Mike discussed where we are with respect to standardization, Richard Bagnall (@richardbagnall), Chairman of the AMEC Social Media Measurement Group, and I as his vice-chair, presented a session on creating a new recommended framework for social media metrics and measurement. Essentially we tried to answer the question: how do we take the standards work coming from The Conclave and operationalize it to create proper social media measurement? Here is an overview of what we presented and what we are encouraging everyone to adopt and use. The framework templates, usage guide and a short video synopsis will be available for download from the AMEC website, Social Media Measurement section, in the next week or so.

Valid Metrics Framework and Social Media

Our journey begins with the Valid Metrics Framework, a measurement planning framework and template developed under the auspices of AMEC. The framework was designed to be flexible enough to address multiple aspects of public relations within a consistent measurement framework and approach.

VMFII

Some of the most positive aspects of the Valid Metrics Framework are that it:

  • Provides a mechanism to link activities to outputs to outcomes
  • Tracks through the familiar sales funnel
  • Helps create a focus on outcomes.

One of the applications of the Valid Metrics Framework was for use with social media programs. Two potential issues were surfaced by early adopters of the framework in social programs. The intermediary effect, which in traditional public relations is the impact on the media, seemed at odds with the social world of direct interaction between consumers and brands, and consumers with each other. And use of the marketing sales funnel, while familiar, was only relevant in a percentage of social media use cases and perhaps not the best way to model common uses of social media like customer relations and building relationships with stakeholder groups. Also, thought leaders like Forrester Research and McKinsey & Company had noted the traditional communications funnel was not necessarily funnel-shaped in social media. They described the discovery process that occurs when investigating companies and brands that often cause the consideration set to expand rather than be reduced, and the fact that a lot of engagement around brands happens post-conversion event. For all of these reasons our task was develop and recommend a better framework and approach.

Models and Frameworks

When we use the word model, we are referring to a representation of a system, in this case social media. In the original Valid Metrics Framework, the model used was the traditional sales funnel. A framework adds additional dimensions to the model and is operationalized with metrics. In the Valid Metrics Framework the additional dimensions are the phases – activities, intermediary effects and target audience effects. We looked at both of these aspects individually and collectively when considering alternative approaches.

We studied and evaluated about fifteen different social media and communications models. A couple of common patterns emerged. Several of the models, including Forrester’s Customer Lifecycle and McKinsey’s Customer Journey showed a post-purchase engagement/experience step. We judged this important to include in our recommended approach. And we considered the importance of Engagement and Influence, as two key concepts in social media marketing and measurement, and decided to try to make these two elements explicit in our model as well.

Suggested Social Media Metrics Model

The model we developed is derivative of the categories chosen by The Conclave (Note: Richard Bagnall and I also participate in The Conclave) to organize social media metrics and definitions. We took a slightly different perspective on the front end of the model and reordered the back-end to create this model for our new framework. The descriptions of the stages use the definitions from the smmstandrards.org work wherever possible.

New Model

You will note Engagement includes both interactions with owned social channels as well as earned social conversation of people ‘talking about you’ in social channels. The definition of Influence is clear and concise, the result of a lot of discussion and prevailing clear thinking. The concept of Impact includes the outcomes of social initiatives as well as the Value those initiatives created. (I usually advise to always attempt to measure impact, and attribute value when it is feasible and makes sense.) Advocacy includes a very helpful definition and conditions that exist with advocacy.

Creating the Framework

To create the framework, we explored various ways to address the ‘phases’ of the Valid Metrics Framework. Two ideas stood out:

  • Use a simple structure that captures social media metrics from three key perspectives – programmatic-level, channel-specific and business. Programmatic metrics are those directly tied to social media objectives. Channel-specific metrics are just that, the metrics that are unique to specific social channels – tweets, RTs, Followers, Likes, Talking About This, Pins, Re-Pins. Business metrics are used to show the business impact of the campaign or initiative.
  • Use Paid, Owned, and Earned media metrics for integrated programs containing these elements. Borrowing the definitions from Forrester, Paid are social channels you pay to leverage (e.g. promoted tweets, display ads), Owned are channels you own and control (e.g. website, Facebook page) and Earned is where customers become the channel (e.g. WOM, viral)

There are certainly other ways to think about this (e.g. Business Performance Management) and we intend to possibly add others based on industry feedback and suggestions.

The AMEC Social Media Valid Framework

Currently we have developed both versions with sample metrics (taken from the smmstandards.org work where applicable). We are calling them The AMEC Social Media Valid Framework. Here is the version with Program, Channel and Business Metrics shown.

Valid Metrics Framwork

Where Do We Go From Here?

Look for the completed frameworks on the AMEC website shortly. We encourage you to adopt the frameworks for use by your company or clients. If you like them and find them useful, please help promote them widely. And please provide your feedback on the proposed framework on this blog, or through the social channel of your choice. We’re listening and looking forward to the dialogue.

Where is Your Organization on the Social Media Listening Maturity Model?

23 Jul

Quite often I am asked to consult with a company on their social media listening strategy. Their initial question more times than not is about the listening platform they should use. But it is increasingly common for the questions to be more sophisticated and the ambition behind them to be much greater. Companies with experience in social listening know that it is all too easy to focus on rudimentary analysis of brand mentions and topics, Followers and Likes and never get to the truly actionable insights that lead to marketing or business actions. Experience in listening is an important element here but you also need a path to follow. I thought a maturity model approach to social media listening could provide a possible path to consider and would provide a construct that could be used in consulting with a company on their social listening strategy.

Maturity models are sort of hot – there seems to be a proliferation in the last two years or so. One that I find particularly insightful and helpful when thinking about social listening is Forrester’s Social Maturity Model.  Two really important points the folks at Forrester make is that listening is not the goal, social intelligence is, and that social intelligence informs actions taken by marketing or some other area of the business. Action being the operative word here. Social intelligence is a closely related topic to social business, and if social business is more your thing the Dachis Group has an interesting social business maturity model.  Big data more your bag? Check out IBM’s big data governance model. After looking at the models out there, I could not find one specific enough to social media listening so I took a stab at creating one.

Social Media Listening Maturity Model 

There are five stages in the Social Media Listening Maturity Model, beginning with reactive alerts and ending with social intelligence. Let’s take a brief look at each stage and some of the overarching differences or changes one sees with social listening maturity.

Reactive Alerts – Many companies or brands begin by establishing a reactive alert system that notifies them whenever their brand is mentioned or is mentioned with specific keywords. Think Google Alerts. Companies in this stage may only periodically check social media channels to see what may have changed or is new since the last check-in.

Monitoring Social Media – At the next stage, the company has begun active monitoring of all ‘owned’ social embassies. They also are monitoring social media conversations, often focused on trying to detect any ‘bad’ news, mentions or conversations.

Companies in these first two stages generally have a reactive stance toward social media, viewing it as another way to find out about news and circumstances that may harm or otherwise impact the organization. It is common for companies in these stages to use one or more of the various free tools available to gather web and social media data.

Social Listening – The third stage is most likely where the largest percentage of companies reside today. Companies in stage three are listening to social conversations about their company, brands and products. They are tracking mentions of competitors and calculating share of conversation. Many also track issues and topics that are important to their brands/products/company. At this stage many begin to put additional emphasis on ‘who’ is talking (source) not just what is being said (post). Most companies in the social listening phase have transitioned from free tools to paid platforms.

Companies in the first three stages often suffer from having too much data and not enough insights. They are up to their necks in ‘big data’ but lack the experience and expertise to analyze the data and reduce it down to crisp, actionable insights supported by the data. They look for the Insight button on the tools they use but increasingly realize insights are the product of human analysts, not tools or data.

Strategic Listening – The transition to strategic listening brings with it a bias toward ‘listening with a purpose’. I first heard this turn of phrase from my friends at Radian6 and use it often. Listening with a purpose is just that – listening to specific sets of conversations with a specific goal or objective in mind. Often in insight work, the goal or objective may take the form of a hypothesis we are trying to test. Here are a few examples of listening with a purpose:

  • Listening for conversations of consumers in a particular phase of the buying decision process
  • Listening to customers whose subscriptions or policies are about to expire that are expressing thoughts of changing vendors
  • Identifying, tracking and building relationships with key influencers
  • Listening for consumer reactions to new packaging or product features
  • Mining the emotional content of specific stakeholder groups to determine potential risk around a sensitive issue.

During this phase, an Enterprise listening strategy is often developed and implemented. Some also begin to integrate data from sources beyond social media – search, web analytics and customer data for example.

Social Intelligence – Forrester defines social intelligence as the process of turning social media data into actionable marketing and business strategy. Social intelligence therefore is not about the best times to tweet or whether or not a twitter party would be an effective tactic, it is about informing strategic decisions that impact the company’s success. For me, three concepts are crucial:

  1. Action – social intelligence is designed to drive true actions.
  2. Integration – although the definition focuses on social media data and insights, the fact is that true insights often require more than just social data. Integrating data from multiple data sources – consumer survey, behavioral tracking, social posts, search analytics, advertising data, customer records, scan/sales data – allows for greater understanding and richer insights. Integration of multiple data types often requires multiple tools and platforms to aggregate and analyze the data.
  3. Sharing – For social intelligence to truly take root within an organization, the data and insights should involve cross-disciplinary groups that can look at the data from different perspectives and collectively arrive at better insights than any one group could in a vacuum. The insights then need to be systematically shared broadly across the organization so they may be acted upon in a manner that will create the most impact. Social intelligence can be a catalyst to the silos within an organization tumbling down.

Since the social listening and social intelligence ‘markets’ are relatively immature, this model will continue to evolve and be refined.

Where is your company today on the social media listening maturity model?

Time to Get Real About Social Media Audience Reporting

12 Jun

Though almost everyone would agree that social media is about engagement and not eyeballs, too much of digital and social media measurement is focused on audience size. How many Followers do we have? How can we get a million Likes? How many unique visitors did we have to our site this month? And unfortunately, audience size estimates in social media grossly overstate the actual relevant audience. We seem fixated and oriented toward ‘how many’, while our focus should be on ‘who’ and specifically, ‘who within our target audience’. Generally speaking, the advertising industry has led the way with audience measures and is ahead of where the public relations and social media camps are with respect to level of sophistication.

In television advertising, the concept of Target Rating Points is a refinement of Gross Rating Points where you only measure and get ‘credit’ for the percentage of the gross audience that meets your target audience criteria. In an effort to keep refining the audience data available, Nielsen has evolved from diary-based data to electronic data to software at the set-top box level that allows operators to monitor channels choices and changes. In audio-based media, Arbitron’s Portable People Meter recognizes today’s mobile world and begins to address cross-platform measurement. It is also interesting to reflect on the U.S. Congressional involvement in television audience ratings accuracy (or lack thereof as it were) that led to the formation of what is now known as the Media Rating Council in the early 1960’s. The time has come for social media audience research to greatly increase in sophistication, accuracy and relevance.

When we think about social media audience size measures today, the emphasis is on Opportunities To See (OTS), although almost never by this name. We might call them Impressions or Reach, but what we really mean is how many people had the potential to see this content item. There are two overarching issues here:

  • Opportunities to see are not the same as actually seeing
  • The metrics count all possible members of the audience, regardless of whether or not they are part of the targeted audience or can even buy the product or service.

OTS is also a prevalent metric in the public relations industry which has always focused on stating the highest possible audience measures. In traditional media we know the probability of any one person in the audience actually seeing the article in question is a fraction of the total audience – a reasonable estimate is 10% or less. So OTS greatly overstates the actually number of people who saw a given article. To compound the audience overstatement, we have the practice of using audience multipliers to ‘credit’ earned media for either a perceived credibility advantage over advertising or to account for pass-along circulation (see this IPR white paper for more on multipliers). Thankfully the practice of applying multipliers (and its evil cousin AVEs) is out of favor and rapidly on a path toward extinction.

In social media one can make the case the audience metrics situation is actually exacerbated in that the probability of any one follower seeing any one tweet, for example, is most likely an order of magnitude less than in earned media – my guesstimate is 1% or less. Before you call BS on this guesstimate, play around with a few Twitter factoids – the recent Pew Research study suggesting only 8% of Twitter users use it daily, the perishable nature of most individual’s twitter streams, and the fact that a reasonably high percentage of Followers of a brand are bots, and the reality is that only a small fraction of twitter followers actually see tweets, let alone find it interesting enough to share or comment on. And, of course, not all Facebook Likes see every post you make either. Riffing on the old, ‘if a tree falls in the forest…’, if you tweet into the twitterverse and no one sees it does it make an impact?

Evolving from ‘opportunities to see’ to ‘relevant audience’ measures.

Most social media campaigns have a specific target audience in mind, often described with demographics (Female, age 18 – 34), psychographics (who worry about feeding their family healthy food on a budget) and behavioral (access deal and coupon sites regularly) dimensions. Yet when it comes to reporting and measurement we take credit for the entire audience (total OTS) rather than the percentage of the audience that meets our targeting criteria. Trying to promote lingerie to 22 – 29 year old ladies? No worries, count all your Twitter Followers and all the visitors to your website – the men, the young and the old – everybody counts. Trying to sell camo clothing to male hunters? No worries, everybody counts – male, female, hunters, non-hunters and PETA members, too. Of course this all seems a little silly and strange and I suppose it would be if it wasn’t the way most social audience reporting is done today. It is unusual to see someone in social media, or PR for that matter, report only the relevant audience opportunities to see. Why is this? I believe there are three primary reasons:

  1. Legacy – the PR industry has historically reported gross potential audience size rather than the relevant audience size. When social media came around, this same orientation toward gross audience measurement was used.
  2. Data – there is a lack of consistent social media demographic and psychographic audience data available and it often resides in channel silos rather than cross-platforms. And often the audience data from one platform (e.g. ComScore) does not match the data available from another platform (e.g. Compete).
  3. Standards – there are no standards for social media audience metrics and no codified best practices for audience measurement.

Where do we go from here?

First, we need a change in mindset of how we think about audiences. From ‘how many people theoretically had the potential to see our content’ to ‘how many of the people we were targeting actually saw our content’. Big audience numbers are irrelevant. Relevant audience numbers are big.

Next, as the demand for audience data that contains demographic, psychographic and behavioral data grows, it is reasonable to assume one or more of the large media data companies might start to aggregate and make the data available. Privacy concerns, cookies and other issues are also in play here.

And last but not least, industry standards for social media audience and engagement metrics and definitions are necessary for transparency and replicability that will increase credibility of social media measurement and reporting. 2012 will go down as the year that serious cross-industry progress on social media metrics standards began and gained momentum. There has already been a lot of progress (See this post from Katie Paine), and this week in Dublin at the 4th AMEC European Summit on Measurement the theme is around attempting to define standards for social media metrics and measurement. To tune into the debate as it occurs in Dublin, monitor #SMMStandards and #AMEC2012.

What  are your thoughts on the need for social media metrics standards and the use of target rather than gross audience size estimates?

Measurement 2020 and Other Fantasies

23 Sep

At the 3rd European Summit on Measurement held in Lisbon in June 2011, standardization, education, ROI and measurement ubiquity emerged as the key themes in response to a call to set the Measurement Agenda 2020.  Delegates to the conference voted on 12 priorities they thought were most important to focus on in the period leading up to 2020.  The top four vote-getters became the Measurement Agenda 2020:

  1. How to measure the return on investment of public relations (89%)
  2. Create and adopt global standards for social media measurement (83%)
  3. Measurement of PR campaigns and programs needs to become an intrinsic part of the PR toolkit (73%)
  4. Institute a client education program such that clients insist on measurement of outputs, outcomes and business results from PR programs (61%)

For a very nice overview of the Lisbon session and the Barcelona Principles that came before, read this post from Dr. David Rockland of Ketchum who chaired the Barcelona and Lisbon sessions.  David pretty much said it all on these sessions, so I’ll just add a couple of comments and share a few thoughts on what I believe the future of measurement 2020 could be.

The rallying cry coming out of Barcelona has been focused and loud – death to AVEs!  Will there be a similar thematic coming out of Lisbon and what might it be?  My money is on standardization, borne out of cross-industry cooperation.  As David points out in his post, and in the words of AMEC Chairman Mike Daniels, “The Summit identified some significant challenges for the PR profession to address by 2020.  However, what we also accomplished in Lisbon beyond setting the priorities was to harness the commitment and energy of the industry to agree what we need to do together.”  The current cooperation and collaboration between industry groups – AMEC, Institute for Public Relations, PRSA and the Council of PR Firms is unprecedented in my time in this industry and is focused on tangible outcomes.  Cross-organization committees are already at work developing standard metrics for social media measurement for example.  The spirit of cooperation is uplifting.  While the outward thematic appears to be standardization, cooperation is the enabling force.  

I was also struck by the symmetry of the call to end AVEs in Barcelona and the call to codify ways to measure ROI in Lisbon.  One follows the other.  In my opinion the primary reason AVEs exist is because PR practitioners feel pressure to prove the value of what they do, and quite often they are asked to describe the impact in financial terms.  AVEs are perceived as a path of least resistance way to express financial value.  Except, as we all know, AVEs don’t really have anything to do with the impact public relations creates.  They are a misguided proxy for financial value.  Hence the need for research-based methods to determine true return on investment.

All of the priorities coming out of Lisbon are excellent goals for the industry.  And like David Rockland, I believe they will be achieved, and be achieved before 2020.  Here are three other items on my wish list for Measurement 2020:

Word of Mouth/Word of Mouse Integration: For those of us focused in social media and other digital technologies, we can’t allow our digital lens to color what is fundamentally an analog world.  Research studies suggest the majority of word of mouth happens in real life.  From an influence perspective, I don’t think too many would argue that word of mouth from a trusted friend or family member is more powerful than word of mouse from someone you follow on Twitter.  Digital cross-platform research is difficult enough, but when one huge platform is ‘real life’, we have significant challenges in measurement.  WOMMA and others have made early attempts to define measurement approaches for offline WOM, but much work remains.  We need ways to assess its impact and then we need to think about ways to attribute value to that impact.  Mobile is a wild card here as it becomes the preferred platform for online activity.  The need to triangulate online, mobile and ‘real life’ measurement presents significant challenges today, and may still by 2020.

Cookie Wars: We all know the measurement versus privacy showdown is coming, right?  The first shots have already been fired.  The collection of source-level personal data, enabled by cookies, is crucial to measurement and insights but has the potential for misuse or unintended disclosure.  Some sophisticated consumers have had their fill of cookies.  Although the broader issue might be framed as social sharing versus privacy control, how it plays out will have a direct impact on digital analytics and measurement.

Integrated Measurement across the Paid Earned Shared Owned (PESO) Spectrum: Measurement has increasingly become integrated.  It began with integrated traditional (Earned) and social media (Shared) measurement and then progressed rapidly to Earned, Owned and Shared, which is where most integrated measurement programs are today.  Many leading-edge integrated programs today also include advertising or Paid media.  By 2020, integrated measurement across the PESO spectrum will most likely be the norm and not the exception.  A key enabling element here in my view is some base level of agreement on how each area should be measured and standard metrics for each.  It will take significant cooperation between industry groups, vendors, agencies and major customers/clients for cross-discipline standardization to move forward effectively.  We are at the beginning of this movement in 2011.  By 2020, it will be fascinating to look back and see how all this plays out.

When looking ahead to 2020, I am reminded of a measurement discussion pulled together by PRWeek a couple of years ago.  Many of the Measurati attended.  In response to a question of where measurement will be in five years, David Rockland replied (paraphrasing here), ‘Who knows?  Five years ago who would have guessed we would all be focused on how to measure social media?’  So, there is a certain fantasy element to discussing 2020 challenges in measurement.  What are your measurement fantasies?

AVEs Don’t Describe the Value of Media Coverage, They Sensationalize It.

26 Jun

Saturday, Wall Street Journal columnist Carl Bialik, The Numbers Guy, addressed the subject of advertising value equivalency (AVE).  This is perhaps the first example of a mainstream media publication shining a light of the controversial practice of AVEs.  (You can read the story here.)

The primary reason advertising value equivalents exist are because they are perceived to be a way to attribute value to programs that would otherwise be difficult to value directly.  They are a path of least resistance approach to return on investment calculations, but not a valid one.  Let’s take a deeper dive into the three specific examples in the WSJ story, ask the tough questions and discuss more valid ways to think about value attribution and ROI.

American Airlines  

You can enjoy both questionable valuation techniques and hyperbole in this article.  American Airlines stands to “make boatloads of cash” and “the airline company could gain as much as $95.9 million of exposure”.  Of really, let’s take a closer look.

The most incredible part of this financial calculation is the financial calculation itself.  The calculation is apparently based on sign placement within the arena and presumably the ‘impressions’ the brand will receive when people attending the venue see the signage and when TV cameras catch the signs when showing the scoreboard or during the action.  This is a very passive form of advertising that should have as its objective either creating top of mind awareness or perhaps creating more brand affinity.  Rather than using an advertising equivalency model that has no validity, a true measurement of the value created by naming rights would ask a series of questions designed to determine the actual, tangible (or even intangible) impact on the business:

  • Revenue: Can incremental revenue generation in the form of higher passenger miles be directly attributed to the exposure created by the naming rights?  Is it possible that incremental revenue would actually be realized on a game by game basis, or would any positive impact be realized over a longer time horizon?  Have new customers been created as a direct result of the exposure generated by the naming rights?
  • Brand: Can the increased exposure lead to people perceiving the brand differently and can the difference translate into higher transactional revenues generated or increased brand loyalty?

So where exactly are the ‘boatloads of cash’ American Airlines made?  Are they hitting the income statement in the form of incremental revenue or enhanced brand loyalty (repeat business)?  Are they residing on the balance sheet in terms of brand goodwill?  Given that American’s parent company AMR lost $11.5B dollars in the first decade of the 21st century, its last profitable year was 2007 and they are projected to lose money in 2011 and 2012, they could use the cash.  Perhaps they could use it to fund a ’bags fly free’ program or for enhancing their Advantage program to create more brand loyalty.  I would strongly suspect American’s shareholders would prefer a do-over on the investments made on naming rights to the ‘boatloads of cash’ they are now enjoying from the investment.

Couple Won’t Cash In on Kiss

15 minutes of fame is rarely worth $10 million.  In this case, the celebrity agent is suggesting the news value of the coverage generated by the kiss is somehow equivalent to advertising value and assigns what appears to be an arbitrary and ridiculously high value to it.  (He later admits he just made the number up.)  Just how was the couple going to monetize their 15 minutes of fame?   Yes, they turned down a few talk show opportunities and perhaps the National Enquirer would have thrown a few dollars their way for an exclusive, but the assertion that any major brand would have paid them to endorse their product is wildly speculative.  I would guess that if you did a survey after the event, a small number of people would remember seeing the coverage, and a very small percentage of the people who did see it would have recalled Scott Jones’ name.  So perhaps Mr. Jones walked away from tens of thousands of potential dollars in the short-term, but nowhere near the sensationalized estimate of $10 million.  15 minutes of fame might be worth 10 thousand dollars, but certainly not $10 million.

Obama Enjoys a Guinness

So Guinness is a winner and received $20 million worth of “free publicity”?  What was the outcome of the publicity?  Again, in order to determine the value of the “free publicity” (this term is despised in the PR industry by the way), Guinness would have to be able to measure incremental revenues directly attributable to the publicity generated.  Did sales of Guinness increase as a result?  Were new customers created?  Did existing customers feel compelled to drink even more?  What was the value of the incremental sales?  These are much more difficult questions to answer but are the correct ones to ask in order to measure the publicity.  Not by focusing on the mythical value of the coverage as measured by flawed advertising equivalency, but measuring the outcome or what happened as a result of the publicity.  The assertion that President’s Obama’s image was softened and will help keep him in the public’s favor is highly dubious thinking.  Perhaps it helps him in Boston, but in the grand scheme of things, this is a Presidential image non-event.

Beginning last Summer in Barcelona,  the public relations industry has come together to publicly state advertising value equivalency is not a valid measure of public relations.  The so-called Barcelona Principles are explicit against AVEs and also call for a focus on measuring outcomes and not (just) outputs.  While it will take some time for the PR industry to totally leave AVEs behind, there is a lot of momentum right now to make this happen sooner rather than later.  No serious measurement effort can use advertising value equivalency to attribute value and be credible.  

Follow

Get every new post delivered to your Inbox.

Join 171 other followers