Measurement 2020 and Other Fantasies

At the 3^rd European Summit on Measurement held in Lisbon in June 2011, standardization, education, ROI and measurement ubiquity emerged as the key themes in response to a call to set the Measurement Agenda 2020.  Delegates to the conference voted on 12 priorities they thought were most important to focus on in the period leading up to 2020.  The top four vote-getters became the Measurement Agenda 2020:

1. How to measure the return on investment of public relations (89%)
2. Create and adopt global standards for social media measurement (83%)
3. Measurement of PR campaigns and programs needs to become an intrinsic part of the PR toolkit (73%)
4. Institute a client education program such that clients insist on measurement of outputs, outcomes and business results from PR programs (61%)

For a very nice overview of the Lisbon session and the Barcelona Principles that came before, read this post from Dr. David Rockland of Ketchum who chaired the Barcelona and Lisbon sessions.  David pretty much said it all on these sessions, so I’ll just add a couple of comments and share a few thoughts on what I believe the future of measurement 2020 could be.

The rallying cry coming out of Barcelona has been focused and loud – death to AVEs!  Will there be a similar thematic coming out of Lisbon and what might it be?  My money is on standardization, borne out of cross-industry cooperation.  As David points out in his post, and in the words of AMEC Chairman Mike Daniels, “The Summit identified some significant challenges for the PR profession to address by 2020.  However, what we also accomplished in Lisbon beyond setting the priorities was to harness the commitment and energy of the industry to agree what we need to do together.”  The current cooperation and collaboration between industry groups – AMEC, Institute for Public Relations, PRSA and the Council of PR Firms is unprecedented in my time in this industry and is focused on tangible outcomes.  Cross-organization committees are already at work developing standard metrics for social media measurement for example.  The spirit of cooperation is uplifting.  While the outward thematic appears to be standardization, cooperation is the enabling force.  

I was also struck by the symmetry of the call to end AVEs in Barcelona and the call to codify ways to measure ROI in Lisbon.  One follows the other.  In my opinion the primary reason AVEs exist is because PR practitioners feel pressure to prove the value of what they do, and quite often they are asked to describe the impact in financial terms.  AVEs are perceived as a path of least resistance way to express financial value.  Except, as we all know, AVEs don’t really have anything to do with the impact public relations creates.  They are a misguided proxy for financial value.  Hence the need for research-based methods to determine true return on investment.

All of the priorities coming out of Lisbon are excellent goals for the industry.  And like David Rockland, I believe they will be achieved, and be achieved before 2020.  Here are three other items on my wish list for Measurement 2020:

Word of Mouth/Word of Mouse Integration: For those of us focused in social media and other digital technologies, we can’t allow our digital lens to color what is fundamentally an analog world.  Research studies suggest the majority of word of mouth happens in real life.  From an influence perspective, I don’t think too many would argue that word of mouth from a trusted friend or family member is more powerful than word of mouse from someone you follow on Twitter.  Digital cross-platform research is difficult enough, but when one huge platform is ‘real life’, we have significant challenges in measurement.  WOMMA and others have made early attempts to define measurement approaches for offline WOM, but much work remains.  We need ways to assess its impact and then we need to think about ways to attribute value to that impact.  Mobile is a wild card here as it becomes the preferred platform for online activity.  The need to triangulate online, mobile and ‘real life’ measurement presents significant challenges today, and may still by 2020.

Cookie Wars: We all know the measurement versus privacy showdown is coming, right?  The first shots have already been fired.  The collection of source-level personal data, enabled by cookies, is crucial to measurement and insights but has the potential for misuse or unintended disclosure.  Some sophisticated consumers have had their fill of cookies.  Although the broader issue might be framed as social sharing versus privacy control, how it plays out will have a direct impact on digital analytics and measurement.

Integrated Measurement across the Paid Earned Shared Owned (PESO) Spectrum: Measurement has increasingly become integrated.  It began with integrated traditional (Earned) and social media (Shared) measurement and then progressed rapidly to Earned, Owned and Shared, which is where most integrated measurement programs are today.  Many leading-edge integrated programs today also include advertising or Paid media.  By 2020, integrated measurement across the PESO spectrum will most likely be the norm and not the exception.  A key enabling element here in my view is some base level of agreement on how each area should be measured and standard metrics for each.  It will take significant cooperation between industry groups, vendors, agencies and major customers/clients for cross-discipline standardization to move forward effectively.  We are at the beginning of this movement in 2011.  By 2020, it will be fascinating to look back and see how all this plays out.

When looking ahead to 2020, I am reminded of a measurement discussion pulled together by PRWeek a couple of years ago.  Many of the Measurati attended.  In response to a question of where measurement will be in five years, David Rockland replied (paraphrasing here), ‘Who knows?  Five years ago who would have guessed we would all be focused on how to measure social media?’  So, there is a certain fantasy element to discussing 2020 challenges in measurement.  What are your measurement fantasies?

Public Relations Measurement 2010: Five Things to Forget & Five Things to Learn

(This post is a re-purposing of a speech I gave to the FPRA/PRSA-Orlando on July 23, 2009.  You can download the slides here.)

Public relations measurement is at a crossroads.  Old techniques are no longer sufficient.  Old metrics are no longer applicable.  Old thinking must be replaced by new.  The need for accountability, and to prove the value of PR and social media programs, has never been greater.

As we look to the next year, here are five things to forget and five things to learn about public relations measurement in 2010.

Things to Forget in 2010

1. Media Relations Focus

A focus on media relations fails to capture several important aspects of PR – brand, reputation, crisis, employee communication and DTC to name a few.  Also, the importance of traditional media is declining.  Numerous studies have shown people don’t trust what they read in the media, they trust each other.  I believe it was Hauser and Katz who coined the term ‘you are what you measure’ in 1998.  If measurement is focused on media relations that is how the public relations function will be judged.

__________________________________________________________

2. Outputs

The need to put PR results in a business context has never been greater.  We need to be able to address the question – what are we doing to help drive the business?  If you are focused on output metrics like impressions or message delivery, you will always have a hard time explaining business impact.  Instead, we need to focus on outcomes and answer the question – what happened as a result of our program or coverage?  Understanding outputs has primary benefit as a diagnostic tool rather than a ‘scorecard’.

__________________________________________________________

3. Impressions (and Multipliers!)

The most common PR metric today is Impressions.  While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek.  Impressions also (greatly) overstate actual relevant audience.  Generally only a fraction of any particular magazine or newspaper’s circulation meets your target audience demographics.  And impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content.  To compound the problems, many PR practitioners use a multiplier on impression numbers to account for pass-along readership or a mythical credibility advantage PR has over other communication tools.  The simple fact is there is no factual basis (e.g. research proof) that multipliers should be used in any case.

__________________________________________________________

4. Ad Equivalency (AVEs)
There are many reasons why using ad equivalency as a proxy for PR value is not advisable.  Here are five good reasons they should be avoided:

• AVE calculations vary and there are no standards.  Tonality, article length, competitive mentions and other factors are handled differently.
• AVE results can be misleading.  AVEs may be trending up while metrics like message communication, share of favorable positioning and share of positive press are falling.
• AVEs reduce PR to just the media dimension by only assigning a value in this area.
• AVEs only apply to traditional media.  What is the AVE of a positive conversation about your company on a leading blog?
• How much is it worth for a troubled company to not appear in the Wall Street Journal?  AVEs cannot address this.

__________________________________________________________

5. Return on (Engagement/Influence/etc.)
Not a day goes by on Twitter without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks.  Wrong and wrong.  Most of these folks either don’t understand ROI or don’t know how to obtain the data necessary to calculate it.   There is also a lot of confusion between creating value and ROI.  Generating awareness creates value, for example, but may not immediately result in demonstrable ROI.

__________________________________________________________

Things to Learn in 2010

1. Total Value of PR

The majority of current PR measurement efforts focus on marketing/sales and output metrics.  The Total Value Cube is a way to visualize and think about all the potential value your PR and social media efforts deliver.  Beyond marketing to include brand and reputation, beyond outputs to include engagement, influence and action, and beyond revenue generation to include cost savings and cost avoidance.

__________________________________________________________

2. A New Model for Measurement
Many public relations practitioners regularly get their Outputs confused with their Outtakes or Outcomes.  Outtakes is not often used in the U.S. – it seems much more prevalent in Europe.  The overall terminology is confusing and is defined in different ways by different practitioners.  Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.

What we need is a metrics taxonomy that is easier to understand and explain.  I like this one.

Exposure – to what degree have we created exposure to content and message?

Engagement – who, how and where are people interacting/engaging with our content?

Influence – the degree to which exposure and engagement have influenced perceptions and attitudes

Action – as a result of the PR/social media effort, what actions if any has the target taken?”

__________________________________________________________

3. Three Zones of Measurement

From the left, companies or brands control, own or manage websites  – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with.  This zone is measured primarily by web analytics.  In the middle are the actual social networks and conversations between individuals.   In this zone we are interested in data sets that cannot be gathered solely using web analytics packages.  How often is the brand being mentioned in conversation?  What is the sentiment of the comments?  How often is the brand being recommended and by whom?  Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone.  The third zone represents all the real-world, offline transactions that may be of interest.  Did someone visit the store or attend or event?  Did they buy a product?  Did they recommend the brand or product to a friend over coffee?  Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.

Your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones.  The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.

__________________________________________________________

4. New Metrics

_________________________________________________________

5. The Difference Between Impact/Value and ROI
ROI is a form of value/impact, but not all value takes form of ROI.  ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.  ROI lives on the income statement in business terms.

Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product.  Value creation occurs over time, not at a point in time.  Value creation is process-oriented.  Value lives on the balance sheet.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

Integrated Approaches to Social Media & PR Measurement Will Yield More Actionable Insights

With more public relations programming occurring online than ever before, the measurement world seems increasingly seduced by readily available and increasingly sophisticated web analytics.  Indeed, with respect to social media in particular, it seems the majority of thought leadership in measurement today is being driven by web analytics gurus and not measurement gurus (there are a few exceptions of course).  No real problem here since web analytics add real value in understanding online behavior.  They do a good job of understanding the what, the who and the where.  And the cost is often quite low to obtain the data.

A concern with a singular emphasis on web analytics is they do not provide any real insights into the whys driving the behavior.  What are people thinking when they interact with our content?  Are we influencing the way they perceive the brand or company?  What are the reasons they buy or not buy a product?   These insights not only help us better measure our results, they help inform the development of better and more effective future programming.

I don’t see this as an either/or proposition.  The best answer is both.  And add in content analysis too as a diagnostic tool.  A holistic, integrated social media/PR measurement approach that utilizes web analytics, content analysis and primary audience research adds measurement richness and provides valuable formative insights.  All research/measurement is good.  More is better.  Holistic is best.

Media Content Analysis: Are You Trying To Improve Or Just Keeping Score?

One of the maddening aspects of working in PR measurement is the emphasis on using the results to just keep score rather than using the data as a diagnostic tool to determine what is working and what is not.  In other words, too much What, not enough Why.  I have observed practitioners involved with media analysis generally have an orientation toward one camp or the other.  I’m not sure if one’s measurement orientation is genetic or socialized but it is there. 

Certainly the culture of the company or organization may reward one orientation over the other.  In my experience many senior leaders of companies or organizations seem to reward score keeping over diagnostics.  Perhaps this is natural for them, thinking someone else will worry about the Whys. 

So how can you determine your measurement orientation?  Here are a few statements to help guide you: 

1. If the first question you ask when the quarterly measurement report arrives is, “How many hits did we get this month?”, you’re a scorekeeper.
2. If you ask, “How many unique visitors to our blog did we get this month”, you’re just keeping score.
3. If you notice an explosion of positive comments about your brand in online forums and ask. “I wonder what is causing the explosion of positive comments, who is commenting, and how many are re-commenters”, you are diagnostically oriented.  A scorekeeper would immediately be most interested in the total number of comments and unique visitors.
4. If you are a huge advocate of indexing all public relations results to a single number on a 1 – 100 scale (a la Microsoft and others), you are just keeping score.
5. If you are perfectly content with only measuring outputs, or Exposure as I prefer to say, and don’t care so much about measuring outcomes (Influence), you most likely are a scorekeeper at heart. 

There is a little scorekeeper in all of us.  But, the highest and best use of media content analysis is as a diagnostic tool used to continually fine-tune and improve your public relations programs.

Thanks for reading.  -Don B