Public Relations Measurement 2010: Five Things to Forget & Five Things to Learn
(This post is a re-purposing of a speech I gave to the FPRA/PRSA-Orlando on July 23, 2009. You can download the slides here.)
Public relations measurement is at a crossroads. Old techniques are no longer sufficient. Old metrics are no longer applicable. Old thinking must be replaced by new. The need for accountability, and to prove the value of PR and social media programs, has never been greater.
As we look to the next year, here are five things to forget and five things to learn about public relations measurement in 2010.
Things to Forget in 2010
1. Media Relations Focus
A focus on media relations fails to capture several important aspects of PR – brand, reputation, crisis, employee communication and DTC to name a few. Also, the importance of traditional media is declining. Numerous studies have shown people don’t trust what they read in the media, they trust each other. I believe it was Hauser and Katz who coined the term ‘you are what you measure’ in 1998. If measurement is focused on media relations that is how the public relations function will be judged.
__________________________________________________________
2. Outputs
The need to put PR results in a business context has never been greater. We need to be able to address the question – what are we doing to help drive the business? If you are focused on output metrics like impressions or message delivery, you will always have a hard time explaining business impact. Instead, we need to focus on outcomes and answer the question – what happened as a result of our program or coverage? Understanding outputs has primary benefit as a diagnostic tool rather than a ‘scorecard’.
__________________________________________________________
3. Impressions (and Multipliers!)
The most common PR metric today is Impressions. While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek. Impressions also (greatly) overstate actual relevant audience. Generally only a fraction of any particular magazine or newspaper’s circulation meets your target audience demographics. And impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content. To compound the problems, many PR practitioners use a multiplier on impression numbers to account for pass-along readership or a mythical credibility advantage PR has over other communication tools. The simple fact is there is no factual basis (e.g. research proof) that multipliers should be used in any case.
__________________________________________________________
4. Ad Equivalency (AVEs)
There are many reasons why using ad equivalency as a proxy for PR value is not advisable. Here are five good reasons they should be avoided:
- AVE calculations vary and there are no standards. Tonality, article length, competitive mentions and other factors are handled differently.
- AVE results can be misleading. AVEs may be trending up while metrics like message communication, share of favorable positioning and share of positive press are falling.
- AVEs reduce PR to just the media dimension by only assigning a value in this area.
- AVEs only apply to traditional media. What is the AVE of a positive conversation about your company on a leading blog?
- How much is it worth for a troubled company to not appear in the Wall Street Journal? AVEs cannot address this.
__________________________________________________________
5. Return on (Engagement/Influence/etc.)
Not a day goes by on Twitter without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks. Wrong and wrong. Most of these folks either don’t understand ROI or don’t know how to obtain the data necessary to calculate it. There is also a lot of confusion between creating value and ROI. Generating awareness creates value, for example, but may not immediately result in demonstrable ROI.
__________________________________________________________
Things to Learn in 2010
1. Total Value of PR
The majority of current PR measurement efforts focus on marketing/sales and output metrics. The Total Value Cube is a way to visualize and think about all the potential value your PR and social media efforts deliver. Beyond marketing to include brand and reputation, beyond outputs to include engagement, influence and action, and beyond revenue generation to include cost savings and cost avoidance.
__________________________________________________________
2. A New Model for Measurement
Many public relations practitioners regularly get their Outputs confused with their Outtakes or Outcomes. Outtakes is not often used in the U.S. – it seems much more prevalent in Europe. The overall terminology is confusing and is defined in different ways by different practitioners. Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them. In short, the terms are too much ‘inside baseball’.
What we need is a metrics taxonomy that is easier to understand and explain. I like this one.
Exposure – to what degree have we created exposure to content and message?
Engagement – who, how and where are people interacting/engaging with our content?
Influence – the degree to which exposure and engagement have influenced perceptions and attitudes
Action – as a result of the PR/social media effort, what actions if any has the target taken?”
__________________________________________________________
3. Three Zones of Measurement
From the left, companies or brands control, own or manage websites – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with. This zone is measured primarily by web analytics. In the middle are the actual social networks and conversations between individuals. In this zone we are interested in data sets that cannot be gathered solely using web analytics packages. How often is the brand being mentioned in conversation? What is the sentiment of the comments? How often is the brand being recommended and by whom? Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone. The third zone represents all the real-world, offline transactions that may be of interest. Did someone visit the store or attend or event? Did they buy a product? Did they recommend the brand or product to a friend over coffee? Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.
Your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones. The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.
__________________________________________________________
4. New Metrics
_________________________________________________________
5. The Difference Between Impact/Value and ROI
ROI is a form of value/impact, but not all value takes form of ROI. ROI is a financial metric – percentage of dollars returned for a given investment/cost. The dollars may be revenue generated, dollars saved or spending avoided. ROI is transactional. ROI lives on the income statement in business terms.
Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product. Value creation occurs over time, not at a point in time. Value creation is process-oriented. Value lives on the balance sheet.
Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.
Don – this was a great post. Thanks a lot for writing it. It wouldn’t necessarily have any place in this post, but I think it’s worth mentioning that many of the metrics in your “new metrics” section cannot be obtained through anything other than primary research – either focus groups or survey research. I wonder how this post plays into the overall research stereotypes of “expensive” and “time consuming.” Most companies, particularly thinking about social media here, have a now now now attitude. Obviously, we know that skipping through stages in the evolution of a program typically leads to trouble. By adopting some of your best practices above, will we be forced to fight the same old research battles again? For that matter, did we ever win them? Thanks again for the post!
@chuckhemann
Excellent post. Coming from a qualitative research background, I understand how important it is to understand the quality of engagement and not just the quantity. Tone/sentiment and topics are best understood from conversations created organically. While this is a difficult and time consuming process, they should still be a part of any comprehensive PR measurement. Regardless, well said in moving the issue in the right direction.
Thank you.
Nazim
Hey Chuck,
Thanks for stopping by and for your thought-provoking comment. Let me tackle your last Q first – No, the battles were seldom won, many losses and a few draws. I think the issue with the perceived high cost of measurement is really a numerator and not a denominator issue. That is, the perceived value is not high enough for the given cost. Progress would be made by working on the perceived value issue through education.
You are correct about some of the new metrics requiring primary qualitative or quantitative research. Other metrics will require traditional content analysis (e.g. twitter sentiment analysis) or web analytics. The best measurement strategy going forward will be to integrate all of the above plus perhaps click-tracking, CRM and/or sales data to develop a holistic picture of impact and ROI.
Hi Nazim,
Thanks for your comment – I fully agree with you. Please see my response to Chuck (above) regarding holistic approaches (which may well include qualitative analysis). If you think about it, Who, What, Where, and How we can determine with web analytics and online tools for the most part. Primary research, qualitative and quantitative, is necessary to understand Why people think and behave the way they do.
Great stuff! Wish everyone, including the thought leaders of the industry, would follow your advice
Thanks for your comment and kind words, Katie.
What can I say, Don? You are so far advanced in your thinking… Thanks very much for your continuing and continuous thought leadership.
Sean,
Thanks so much for your all too kind words. It means a lot coming from you.
-Don B
Don – you hit this one straight down the middle and long. I especially like the delineation in the three zones of measurement. Many PR people I talk to, including big agency people, are getting into social media, but haven’t connected with SEO content for Websites, releases, articles, blog posts. Likewise, we can’t leave out the offline part of the equation. You have imposed order on an unsettled landscape. jrb
Once again, a great commentary from Don on measurement. You’re one of the few who keep us honest. Good stuff!
Hi Jim,
Thanks very much for your comment. I am probably remiss in not mentioning search more explicitly. PR/SM’s contribution to organic search positioning is an area that deserves more thought – and measurement. Thanks again.
Hi Jim,
I love your post! I especially like the three measurement zones and your visuals. Do you have these in a powerpoint on slideshare anyplace?
Hi Beth,
Thanks for stopping by and for your kind words. The ppt is available for download on the PRSA Orlando site: http://www.prsaorlando.org/ Look for Acumentics Research in the upper right quadrant. -Don B
Love this post and will grab the PPT. Love to hear someone (finally) challenging Multipliers (media’s greatest magic trick!). At Traackr we work on measuring Influence. Our roadmap is heading to tying that Influence to Action. It’s a great challenge, but one that will drive online PR metrics far into the future…
Hi Derek,
Thanks for your comment. Since my model attempts to tie engagement to influence and influence to action, I’ll be interested in your work at Traackr. Great minds think alike
For more info on why multipliers should not be used, here’s a link to a white paper on the subject I co-authored with Mark Weiner of Prime Research.
http://www.instituteforpr.org/research_single/dispelling_the_myth/
This is so very true. ROI is only one part of this. But, at the end of the day, everything you’ve discussed comes down to some form of measureable value: hence, ROI. Unfortunately, like so many other things, too many “experts” put strict definitions around it that only relate to their specific perspective. The ultimate impact on the Balance Sheet is absolutely ROI. The challenge is proving it. You’ve suggested some terrific approaches to getting us there.
My one last comment (something I’ll steal from @kdpaine), we need to remove the term “Media” from Social Media Measurement. It also puts a box around everything we discuss because it focuses everything around Media concepts. There is so much more the Social Web has to offer and I believe that once we start thinking about its value in broader terms across the enterprise, overall measurement will actually become more definable.
Hi Steve,
Thanks for your comment. I think the issue to some degree with ROI is confusion about the difference between value or impact and ROI. Creating awareness or enhancing relevance creates value, but the value is non-financial until it impacts a sale or saves money in some way (e.g. employee retention). Value creation occurs in many ways over a longer period of time. ROI is transactional and occurs over a finite period of time. Its all good.
I’m with you on the term social MEDIA. Unfortunate choice of word. It does cause many to view social networking through a media lens. I prefer ’social networks’ but no one asked;)
Hey Don, I understand and fully agree! As far as the naming convention is concerned, even Social Networks limits it because it doesn’t account for so many of the variables variables. I’m kind of liking “Social Web”. But, like you, nobody has asked!
This is some of the best thinking I’ve seen on this topic. As someone who teaches public relations research and measurement, I’m going to incorporate your model in my classroom presentation. Thanks for bringing clarity to the difference between impact/value/ROI.
Mary Ann
Hi Mary Ann,
Thanks very much for your comment and kind words. As a teacher myself, having someone say they would teach this material is one of the best compliments I have ever received. I really appreciate it. -Don B
Wow. Very well said and excellent post. So glad you mentioned web analytics to measure web site effectiveness–I spent some time working at a company all about conversions for e-commerce companies, so when I moved into PR measurement, it seemed like a real ‘duh’ move: Measure online PR’s effectiveness via analytics, have online campaigns that drive traffic to the site and measure it there. However, I very rarely see in practice. In fact, I’m appalled by the number of PR practitioners who know nothing about analytics–not even that they exist!
I also think measuring web traffic can be another method to measure the effectiveness of social media efforts. By using benchmarks and comparative metrics, if traffic to your site increases as social media participation increases, than it’s one way to tell your efforts are working in some capacity. (Not the only one of course)
Hi Kelly,
Thanks for stopping by and for your kind words. Your comment is spot on. I find many PR measurement folks have been a little slow to embrace web analytics as one critical set of tools for online PR measurement. That is changing I believe.
Using customized links to drive consumers to specific websites or micro sites is a very good PR tactic that allows you to measure the website metrics with analytics tools as you suggest. By using different links for different social networks you can also isolate the source.
It is also important to consider other measurement methodologies like digital content analysis to measure conversations within social nets and primary audience research to understand consumer thoughts, feelings and behaviors. Triangulating between web analytics, content analysis and click-tracking, and primary research will provide the holistic view and data sets necessary for rigorous measurement of online efforts.
Thanks again. -Don B
Hey Don, great summary and thoughts!
Can only agree with you and some of the other comments: Too few PR professionals are really understanding and hence supporting measurement. And interestingly AVE most times actually is pushed through by the Marketing department as the PR folks don’t have any similar “one-number-says-it-all” metrics.
The tricky part is that already in the ‘old’, traditional media world, most Communicators didn’t invest time or money into sincere measurement. With the rise of Social Media – they now suddenly have so much more on their plates to do, but unfortunately most times with the same (or as consequence of the recession even reduced) resources. So unfortunately any comprehensive measurement approach is then neglected as it’s neither a standard nor easy to apply…
I’m full of hopes that one day PR practicioners will get a better understanding of measurement. But for now, I’m seeing that many professionals are actually rowing back-wards with the attitude “let’s not do anything we can measure in an easy fashion”.
Cheers,
Lars
Hi Lars,
Thanks for your comment and insights. Earlier I described the fight between Good (measurement and accountability) and Evil (budget pressures) and wondered how 2009 would play out along those two dimensions. My sense is that Evil is winning. Hope I’m wrong.
With social networks, one of the measurement risks is an over-reliance on easily obtainable web analytics data. It takes more (web analytics + content analysis + audience behavior and attitude tracking) it order to implement a comprehensive measurement approach.
-Don B
Hi:
Great PR insights here. Finally, finally people are beginning to understand the value of good publicity.
Don,
So much here to like.
I work with small and emerging businesses where, particularly now, the measurement model looks something like…. “What does it cost and how much will we make if we do it?” There exists little tolerance for esoteric metrics schemes.
What you have done is provide a nice, clean measurement model (really like the Three Zones of Measurement) that, even if not accepted as doable by a given client, will at least be understood as…. “oh, so that’s how we figure out if this social media stuff is working.” .
I am grateful for your insights and will move forward with your metrics in the forefront.
Jim,
A belated, Thank You!, for your comment. I appreciate it. -Don B
Hi Don,
I often struggle with finding a measurement tool that reflects what is going on in the real world. I deal mostly with small businesses who constantly need reminding that PR is not sales. I understand they worry about their investment in me, but I’ve run out of ways to say what public relations is and is not. I perform well and get them results, but in their eyes, if the phone doesn’t ring 3 more times a day than before, I’ve failed. My mom, who was my business partner, used to say, though usually not to the client’s face, “you see this? It is a pen, not a magic wand. I am not selling (insert product name here), I am telling people it is for sale, why they should buy it and what it will do for them.”
Your insight and specific description gives me a more defined and concrete way of explaining to my clients what they can expect from me – even if I have to tell them every week or so.
I’m incorporating your research and metrics (citing you as the developer) within my proposals to show how a prospect should be measuring and reading results. It’s still an uphill battle, but the more I keep referring back to this and you, the sooner people, clients will understand what it is that we do and won’t see us as the unchecked villain.
Bravo and Thank You!!
Roberta