(This post is a re-purposing of a speech I gave to the FPRA/PRSA-Orlando on July 23, 2009. You can download the slides here.)
Public relations measurement is at a crossroads. Old techniques are no longer sufficient. Old metrics are no longer applicable. Old thinking must be replaced by new. The need for accountability, and to prove the value of PR and social media programs, has never been greater.
As we look to the next year, here are five things to forget and five things to learn about public relations measurement in 2010.
Things to Forget in 2010
1. Media Relations Focus
A focus on media relations fails to capture several important aspects of PR – brand, reputation, crisis, employee communication and DTC to name a few. Also, the importance of traditional media is declining. Numerous studies have shown people don’t trust what they read in the media, they trust each other. I believe it was Hauser and Katz who coined the term ‘you are what you measure’ in 1998. If measurement is focused on media relations that is how the public relations function will be judged.
__________________________________________________________
2. Outputs
The need to put PR results in a business context has never been greater. We need to be able to address the question – what are we doing to help drive the business? If you are focused on output metrics like impressions or message delivery, you will always have a hard time explaining business impact. Instead, we need to focus on outcomes and answer the question – what happened as a result of our program or coverage? Understanding outputs has primary benefit as a diagnostic tool rather than a ‘scorecard’.
__________________________________________________________
3. Impressions (and Multipliers!)
The most common PR metric today is Impressions. While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek. Impressions also (greatly) overstate actual relevant audience. Generally only a fraction of any particular magazine or newspaper’s circulation meets your target audience demographics. And impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content. To compound the problems, many PR practitioners use a multiplier on impression numbers to account for pass-along readership or a mythical credibility advantage PR has over other communication tools. The simple fact is there is no factual basis (e.g. research proof) that multipliers should be used in any case.
__________________________________________________________
4. Ad Equivalency (AVEs)
There are many reasons why using ad equivalency as a proxy for PR value is not advisable. Here are five good reasons they should be avoided:
- AVE calculations vary and there are no standards. Tonality, article length, competitive mentions and other factors are handled differently.
- AVE results can be misleading. AVEs may be trending up while metrics like message communication, share of favorable positioning and share of positive press are falling.
- AVEs reduce PR to just the media dimension by only assigning a value in this area.
- AVEs only apply to traditional media. What is the AVE of a positive conversation about your company on a leading blog?
- How much is it worth for a troubled company to not appear in the Wall Street Journal? AVEs cannot address this.
__________________________________________________________
5. Return on (Engagement/Influence/etc.)
Not a day goes by on Twitter without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks. Wrong and wrong. Most of these folks either don’t understand ROI or don’t know how to obtain the data necessary to calculate it. There is also a lot of confusion between creating value and ROI. Generating awareness creates value, for example, but may not immediately result in demonstrable ROI.
__________________________________________________________
Things to Learn in 2010
1. Total Value of PR
The majority of current PR measurement efforts focus on marketing/sales and output metrics. The Total Value Cube is a way to visualize and think about all the potential value your PR and social media efforts deliver. Beyond marketing to include brand and reputation, beyond outputs to include engagement, influence and action, and beyond revenue generation to include cost savings and cost avoidance.
__________________________________________________________
2. A New Model for Measurement
Many public relations practitioners regularly get their Outputs confused with their Outtakes or Outcomes. Outtakes is not often used in the U.S. – it seems much more prevalent in Europe. The overall terminology is confusing and is defined in different ways by different practitioners. Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them. In short, the terms are too much ‘inside baseball’.
What we need is a metrics taxonomy that is easier to understand and explain. I like this one.
Exposure – to what degree have we created exposure to content and message?
Engagement – who, how and where are people interacting/engaging with our content?
Influence – the degree to which exposure and engagement have influenced perceptions and attitudes
Action – as a result of the PR/social media effort, what actions if any has the target taken?”
__________________________________________________________
3. Three Zones of Measurement
From the left, companies or brands control, own or manage websites – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with. This zone is measured primarily by web analytics. In the middle are the actual social networks and conversations between individuals. In this zone we are interested in data sets that cannot be gathered solely using web analytics packages. How often is the brand being mentioned in conversation? What is the sentiment of the comments? How often is the brand being recommended and by whom? Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone. The third zone represents all the real-world, offline transactions that may be of interest. Did someone visit the store or attend or event? Did they buy a product? Did they recommend the brand or product to a friend over coffee? Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.
Your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones. The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.
__________________________________________________________
4. New Metrics
_________________________________________________________
5. The Difference Between Impact/Value and ROI
ROI is a form of value/impact, but not all value takes form of ROI. ROI is a financial metric – percentage of dollars returned for a given investment/cost. The dollars may be revenue generated, dollars saved or spending avoided. ROI is transactional. ROI lives on the income statement in business terms.
Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product. Value creation occurs over time, not at a point in time. Value creation is process-oriented. Value lives on the balance sheet.
Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.
metricsman.wordpress.com 
Don – this was a great post. Thanks a lot for writing it. It wouldn’t necessarily have any place in this post, but I think it’s worth mentioning that many of the metrics in your “new metrics” section cannot be obtained through anything other than primary research – either focus groups or survey research. I wonder how this post plays into the overall research stereotypes of “expensive” and “time consuming.” Most companies, particularly thinking about social media here, have a now now now attitude. Obviously, we know that skipping through stages in the evolution of a program typically leads to trouble. By adopting some of your best practices above, will we be forced to fight the same old research battles again? For that matter, did we ever win them? Thanks again for the post!
@chuckhemann
Excellent post. Coming from a qualitative research background, I understand how important it is to understand the quality of engagement and not just the quantity. Tone/sentiment and topics are best understood from conversations created organically. While this is a difficult and time consuming process, they should still be a part of any comprehensive PR measurement. Regardless, well said in moving the issue in the right direction.
Thank you.
Nazim
Hey Chuck,
Thanks for stopping by and for your thought-provoking comment. Let me tackle your last Q first – No, the battles were seldom won, many losses and a few draws. I think the issue with the perceived high cost of measurement is really a numerator and not a denominator issue. That is, the perceived value is not high enough for the given cost. Progress would be made by working on the perceived value issue through education.
You are correct about some of the new metrics requiring primary qualitative or quantitative research. Other metrics will require traditional content analysis (e.g. twitter sentiment analysis) or web analytics. The best measurement strategy going forward will be to integrate all of the above plus perhaps click-tracking, CRM and/or sales data to develop a holistic picture of impact and ROI.
Hi Nazim,
Thanks for your comment – I fully agree with you. Please see my response to Chuck (above) regarding holistic approaches (which may well include qualitative analysis). If you think about it, Who, What, Where, and How we can determine with web analytics and online tools for the most part. Primary research, qualitative and quantitative, is necessary to understand Why people think and behave the way they do.
Great stuff! Wish everyone, including the thought leaders of the industry, would follow your advice 🙂
Thanks for your comment and kind words, Katie.
What can I say, Don? You are so far advanced in your thinking… Thanks very much for your continuing and continuous thought leadership.
Sean,
Thanks so much for your all too kind words. It means a lot coming from you.
-Don B
Don – you hit this one straight down the middle and long. I especially like the delineation in the three zones of measurement. Many PR people I talk to, including big agency people, are getting into social media, but haven’t connected with SEO content for Websites, releases, articles, blog posts. Likewise, we can’t leave out the offline part of the equation. You have imposed order on an unsettled landscape. jrb
Once again, a great commentary from Don on measurement. You’re one of the few who keep us honest. Good stuff!
Hi Jim,
Thanks very much for your comment. I am probably remiss in not mentioning search more explicitly. PR/SM’s contribution to organic search positioning is an area that deserves more thought – and measurement. Thanks again.
Hi Jim,
I love your post! I especially like the three measurement zones and your visuals. Do you have these in a powerpoint on slideshare anyplace?
Hi Beth,
Thanks for stopping by and for your kind words. The ppt is available for download on the PRSA Orlando site: http://www.prsaorlando.org/ Look for Acumentics Research in the upper right quadrant. -Don B
Love this post and will grab the PPT. Love to hear someone (finally) challenging Multipliers (media’s greatest magic trick!). At Traackr we work on measuring Influence. Our roadmap is heading to tying that Influence to Action. It’s a great challenge, but one that will drive online PR metrics far into the future…
Hi Derek,
Thanks for your comment. Since my model attempts to tie engagement to influence and influence to action, I’ll be interested in your work at Traackr. Great minds think alike 😉
For more info on why multipliers should not be used, here’s a link to a white paper on the subject I co-authored with Mark Weiner of Prime Research.
http://www.instituteforpr.org/research_single/dispelling_the_myth/
This is so very true. ROI is only one part of this. But, at the end of the day, everything you’ve discussed comes down to some form of measureable value: hence, ROI. Unfortunately, like so many other things, too many “experts” put strict definitions around it that only relate to their specific perspective. The ultimate impact on the Balance Sheet is absolutely ROI. The challenge is proving it. You’ve suggested some terrific approaches to getting us there.
My one last comment (something I’ll steal from @kdpaine), we need to remove the term “Media” from Social Media Measurement. It also puts a box around everything we discuss because it focuses everything around Media concepts. There is so much more the Social Web has to offer and I believe that once we start thinking about its value in broader terms across the enterprise, overall measurement will actually become more definable.
Hi Steve,
Thanks for your comment. I think the issue to some degree with ROI is confusion about the difference between value or impact and ROI. Creating awareness or enhancing relevance creates value, but the value is non-financial until it impacts a sale or saves money in some way (e.g. employee retention). Value creation occurs in many ways over a longer period of time. ROI is transactional and occurs over a finite period of time. Its all good.
I’m with you on the term social MEDIA. Unfortunate choice of word. It does cause many to view social networking through a media lens. I prefer ‘social networks’ but no one asked;)
Hey Don, I understand and fully agree! As far as the naming convention is concerned, even Social Networks limits it because it doesn’t account for so many of the variables variables. I’m kind of liking “Social Web”. But, like you, nobody has asked!
This is some of the best thinking I’ve seen on this topic. As someone who teaches public relations research and measurement, I’m going to incorporate your model in my classroom presentation. Thanks for bringing clarity to the difference between impact/value/ROI.
Mary Ann
Hi Mary Ann,
Thanks very much for your comment and kind words. As a teacher myself, having someone say they would teach this material is one of the best compliments I have ever received. I really appreciate it. -Don B
Wow. Very well said and excellent post. So glad you mentioned web analytics to measure web site effectiveness–I spent some time working at a company all about conversions for e-commerce companies, so when I moved into PR measurement, it seemed like a real ‘duh’ move: Measure online PR’s effectiveness via analytics, have online campaigns that drive traffic to the site and measure it there. However, I very rarely see in practice. In fact, I’m appalled by the number of PR practitioners who know nothing about analytics–not even that they exist!
I also think measuring web traffic can be another method to measure the effectiveness of social media efforts. By using benchmarks and comparative metrics, if traffic to your site increases as social media participation increases, than it’s one way to tell your efforts are working in some capacity. (Not the only one of course)
Hi Kelly,
Thanks for stopping by and for your kind words. Your comment is spot on. I find many PR measurement folks have been a little slow to embrace web analytics as one critical set of tools for online PR measurement. That is changing I believe.
Using customized links to drive consumers to specific websites or micro sites is a very good PR tactic that allows you to measure the website metrics with analytics tools as you suggest. By using different links for different social networks you can also isolate the source.
It is also important to consider other measurement methodologies like digital content analysis to measure conversations within social nets and primary audience research to understand consumer thoughts, feelings and behaviors. Triangulating between web analytics, content analysis and click-tracking, and primary research will provide the holistic view and data sets necessary for rigorous measurement of online efforts.
Thanks again. -Don B
Hey Don, great summary and thoughts!
Can only agree with you and some of the other comments: Too few PR professionals are really understanding and hence supporting measurement. And interestingly AVE most times actually is pushed through by the Marketing department as the PR folks don’t have any similar “one-number-says-it-all” metrics.
The tricky part is that already in the ‘old’, traditional media world, most Communicators didn’t invest time or money into sincere measurement. With the rise of Social Media – they now suddenly have so much more on their plates to do, but unfortunately most times with the same (or as consequence of the recession even reduced) resources. So unfortunately any comprehensive measurement approach is then neglected as it’s neither a standard nor easy to apply…
I’m full of hopes that one day PR practicioners will get a better understanding of measurement. But for now, I’m seeing that many professionals are actually rowing back-wards with the attitude “let’s not do anything we can measure in an easy fashion”.
Cheers,
Lars
Hi Lars,
Thanks for your comment and insights. Earlier I described the fight between Good (measurement and accountability) and Evil (budget pressures) and wondered how 2009 would play out along those two dimensions. My sense is that Evil is winning. Hope I’m wrong.
With social networks, one of the measurement risks is an over-reliance on easily obtainable web analytics data. It takes more (web analytics + content analysis + audience behavior and attitude tracking) it order to implement a comprehensive measurement approach.
-Don B
Hi:
Great PR insights here. Finally, finally people are beginning to understand the value of good publicity.
Don,
So much here to like.
I work with small and emerging businesses where, particularly now, the measurement model looks something like…. “What does it cost and how much will we make if we do it?” There exists little tolerance for esoteric metrics schemes.
What you have done is provide a nice, clean measurement model (really like the Three Zones of Measurement) that, even if not accepted as doable by a given client, will at least be understood as…. “oh, so that’s how we figure out if this social media stuff is working.” .
I am grateful for your insights and will move forward with your metrics in the forefront.
Jim,
A belated, Thank You!, for your comment. I appreciate it. -Don B
Hi Don,
I often struggle with finding a measurement tool that reflects what is going on in the real world. I deal mostly with small businesses who constantly need reminding that PR is not sales. I understand they worry about their investment in me, but I’ve run out of ways to say what public relations is and is not. I perform well and get them results, but in their eyes, if the phone doesn’t ring 3 more times a day than before, I’ve failed. My mom, who was my business partner, used to say, though usually not to the client’s face, “you see this? It is a pen, not a magic wand. I am not selling (insert product name here), I am telling people it is for sale, why they should buy it and what it will do for them.”
Your insight and specific description gives me a more defined and concrete way of explaining to my clients what they can expect from me – even if I have to tell them every week or so.
I’m incorporating your research and metrics (citing you as the developer) within my proposals to show how a prospect should be measuring and reading results. It’s still an uphill battle, but the more I keep referring back to this and you, the sooner people, clients will understand what it is that we do and won’t see us as the unchecked villain.
Bravo and Thank You!!
Roberta
Terrific article!
This is by far the BEST piece I have ever read on measuring and communicating the value of public relations.
I especially like the points you make about the process of going from exposure to engagement to influence and then to action. It is incumbent upon us as professionals to help our clients develop and understanding of a communication model.
I also like the fact that you are encouraging PR professionals to avoid using “Ad Equivalency” and “ROI.”
The focus needs to be on the goals, objectives and value proposition instead!
Dick Pirozzolo, APR
Wellesley, MA
I disagree about ad equivalency.
As a vp of marketing in 3 public companies I have found this is an excellent way to measure message delivery and make some trade-offs between both budgets.
If you cannot measure it you cannot improve it.
Ad equivalency works – I’ve used it with Edelman and other leading firms.
good luck to you.
Hi Ralph,
Thanks for your comment. Nice to see some disagreement on the blog. Helps with healthy discussion. Hope you don’t if I disagree right back.
– Message delivery/inclusion has nothing to do with the standard way in which AVEs are calculated. It is not therefore a good way to measure message delivery. Content analysis can help with that.
– Setting aside the argument that PR and advertising work together and not necessarily as zero-sum alternatives, looking at CPM figures for advertising and public relations is a better way to compare than AVEs.
– Fully agree with your comment that if you can’t measure it you can’t improve it. There are many ways to measure that provide diagnostic information – what’s working and what’s not/needs fixing. AVEs, however, do not provide any diagnostic information.
– You say ad equivalency ‘works’. Not sure what that means really. IMO, it does not work as a way to properly measure the impact and value created by public relations. It doesn’t work for me.
Good luck to you too.
Poor Ralph — no courage, eh? Only a first name, no link…
AVE is a scourge. It reduces complexity to kindergarten level and is inaccurate as typically practiced. It makes no allowance for way too much public relations activity, as @Don replies above.
Sean,
That’s @Donbart to you…Thanks for stopping by and your comment
That would be a PR insights here. Using customized links to drive consumers to specific websites or micro sites is a very good przeprowadzki warszawa PR tactic that allows you to measure the website metrics with analytics tools as you suggest. By using different links for different social networks you can also isolate the source.
Hi there, Great article and super metrics! I am looking for some help in finding a good metric, or study or an example of how lack of awarness can negatively impact the business. Usually we look at the situation when an awarness activity is already agreed and ongoing, and we face the challange to measure it. Now, how can I build a case, with metrics to show how the business will continue to suffer if there are no communication activities, awarness is not place, etc. Any advice?
I am curious to read a PR report that incorporates these recommended metrics. My biggest client, a tourist destination, insists on Impressions and Ad Values, which are, at best, time consuming. Anyone have a report they are willing to share?
hai, i’,m rara from indonesi. i want to know more about exposure in measuring ROI of online PR. could you please explain detail of this element? im waiting for your answer soon. thanks alot 🙂
Good reference for public relations measurements.
The challenge is that how to equip these public relations practitioners with simple strategy and tactics as to convince the top management on the hidden values of public relations. How to secure their position as a strategic position to senior managers in organization where public relations is considered as a cost centre activity not a long-term investment.
I am the founder of Public Relations Week Indonesia – a ten-year educational campaign in public relations development as a strategic management function.
http://www.prweek-indonesia.com
I am also in charge as the Program Co-ordinator of Master of Management in Communication – the only management based education in the graduate level in which we combine the management disciplines and public relations management as a concentration of our curriculum
http://www.mmcommtrisakti.com
We also share our expertise, trends in both academic and professional nature in form of seminar, workshop and in house training under the name of EGA briefings, Your Partner in Change, since 2001.
Definitely some great insight towards PR measurement. Highly relevant to both practitioners and PR students aiming to break into the PR work field.
&1489;&1488;&1497;&1504;&1496;&1512;&1504;&1496; &1489;&1504;&1497;&1497;&1514; &1488;&1514;&1512; – &1502;&1510;&1497;&1506;&1497;&1501; &1489;&1504;&1497;&1497;&1514; &1488;&1514;&1512;&1497; &1514;&1491;&1502;&1497;&1514; &1489;&1502;&1495;&1497;&1512;&1497;&1501; &1513;&1490;&1501; &1492;&1506;&1505;&1511; &1513;&1500;&1498; &1497;&1499;&1493;&1500; &1500;&1506;&1502;&1493;&1491; &1489;&1492;&1501;
Would you please explain how to design a web for engaging with consumers?
Thanks before..
Wonderful thoughts on PR evaluation. It has indeed made the case of PR relevance to business simplified.
I will surely include it in my training programmes in my country- Nigeria.
Nice work.