Proving the Value of Public Relations

Question:  You are about to go into a meeting with the C-suite members of your company to report on the success of your PR campaign.  Would you rather be able to say:

A.)  The campaign is working well, we have generated over 12 million impressions    

or

B.)  The campaign is working well, we have reached 91% of our target audience an average of 4.2 times.  

OK, so it would have been best to be able to report a direct causal link between PR activities and sales with an estimated ROI of 250%, but, between the two examples above, most of us would rather have reach and frequency data than just impressions.  Reach is typically expressed as a percentage of the audience and therefore requires the practitioner to know the total audience size and how many of the audience were ‘reached’ with a given combination of articles.  You also need to understand media consumption dynamics.  In the advertising world this sort of analysis is common and expected.

The power of reach is that it provides a much more realistic estimate of possible impact than impressions.  For example, if you are targeting women and get a hit in the New York Times, most PR pros today would lay claim to nearly 5 million impressions.  Some, citing pass-along readership and/or a mythical PR credibility advantage, might even inflate this number by a factor of three and claim almost 15 million impressions (For additional discussion on the use of multipliers, please see this IPR White Paper) .  In fact, you have reached a little less than 2 million women with the New York Times hit.  Reach paints a more realistic picture.

In the UK, Metrica purports to have reach and frequency data gleaned from a primary study of 12,000 consumers.  They use it as a standard metric in their bespoke media content analysis offering.  So why don’t any of the U.S.-based media content analysis firms offer reach and frequency metrics?  The database development would be expensive but uptake most likely would be strong. 

While both impressions and reach only represent opportunities to see (OTC), at least the reach approach requires the potential reader to be someone who could actually buy your product or service.  Baby steps of progress.  Next, perhaps we’ll start to measure people who actually saw an article and took the time to read it.  Maybe we’ll use the word du jour and call these ‘authentic impressions’.  

As always, thanks for reading.  -Don B  

For additional viewpoints on the Output/Outtakes/Outcomes debate, see:

Jim Nail’s post here and

Dr. Tom Watson’s post here

In October I attended the 5th. annual Summit on Measurement sponsored by the IPR.  One of the speakers showed a chart listing public relations Outputs and Outcomes.  He listed press releases as an Output rather than Impressions, Number of Hits, Message Pick-up or any of the other metrics correctly referred to as Outputs (See the Dictionary of Public Relations Measurement and Research here).  Why bring this minor transgression up?  Because this is not an isolated occurrence.  Many public relations practitioners, even senior people, regularly have to pause for a moment to make sure they don’t get their Outputs confused with their Outtakes or Outcomes.  Outtakes is not often used in the U.S., it seems much more prevalent in Europe.  The terminology is confusing and is defined in different ways by different practitioners.  Further compounding the confusion is the fact the audiences we present our results or requests to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.   

What we need is a metrics taxonomy that is easier to understand and explain.  Perhaps simple and descriptive enough that we could skip the need for explanation altogether.   I propose the following three terms:

• Exposure - to what degree have we created exposure to materials and message?
• Influence - the degree to which exposure has influenced perceptions and attitudes
• Action - as a result of the public relations effort, what actions if any has the target taken?

The E-I-A construct is easy to understand and does a reasonable job of describing what we are trying to accomplish in public relations.  Here is a graphic that brings it to life a bit.

EIA graphic

There are lot of possible answers to this problem.  EIA may be one of them.  It would be great to hear whether or not you share the view that Outputs/Outtakes/Outcomes is problematic and what solutions you might offer in response.

As always, thanks for reading.  -Don B  

If you would like to read the paper from Hauser and Katz (Metrics: You Are What You Measure) referred to in my last post, you can download a copy here.

Thanks to Constantin Basturea for pointing this out.  -DB

“You Are What You Measure”…I believe the first time I heard this phrase was from the mouth of Katie Delahaye Paine.  I’m quite sure Katie didn’t invent this phrase (it may have been Hauser and Katz in their excellent April 1998 paper entitled, Metrics: You Are What You Measure!), but I certainly have heard Katie use it repeatedly over the years we have known each other. 

There is a corollary to this phrase that might go a little like this: If all you measure is media relations (primarily clip tonnage), that is how the PR profession will be valued.  I have heard or seen this from a number of measurement savvy people recently - Dr. Jim Grunig at the IPR Measurement Summit and Julia Hood in her October 2 column in PRWeek to name two.  So the Catch 22 is this - while we all might agree that attempting to measure public relations is a positive and to be encouraged, what we are measuring and how we are presenting the results is marginalizing our profession - clip book by clip book. 

If public relations is to fully assume a seat at the executive table, we must address this Catch 22 in two fundamental ways:

1. We must do a better job of measuring and communicating all the value (tangible and intangible) delivered by public relations above and beyond media relations/clip counts - executive counsel, investor relations, employee communications, brand building, reputation enhancement, crisis communications…
2. We must connect the dots to show how public relations, including media relations, is helping to achieve desired business outcomes. 

So, as Hauser and Katz suggest in their paper, “Many metrics seem right and are easy to measure, but have subtle, counter-productive consequences.”  This is what I believe is happening today with our clip-happy mentality. 

Hauser and Katz go on, “Other metrics are more difficult to measure, but focus the enterprise on those decisions and actions that are critical to success.”  Exactly! 

Mark Weiner, CEO of Delahaye, and I recently co-authored a paper for the IPR entitiled:

Dispelling the Myth of PR Multipliers and Other Inflationary Audience Measures

Here is a brief summary of the paper:

PR multipliers are often used by PR professionals to factor circulation or audience figures when calculating impressions.  Multipliers are generally rationalized by users to take into account pass-along circulation and/or to assign a higher value to PR impressions than advertising impressions due to a perceived higher level of credibility.  The authors argue that the facts do not support the use of multipliers, and their use may actually hurt the credibility of the profession.

In speaking with colleagues about the use of multipliers, I often hear they would like to not use them, but clients (want/use/demand) them.  The PR agency perspective often is that ‘the previous agency used them, so if we don’t use them it looks like we’re not generating as much coverage.’

So what do you think?  Do multipliers hurt the credibility of the profession?  Are they too entrenched to displace?  Do they help us better compete for marketing dollars with advertising?

Thanks for reading, DB 

In a recent post following up on the Gap IV study, Sherrilynne of Strive PR poses the question in the headline above when referring to determining the true value of PR.  So what is the right answer?  Is it Outputs, Outcomes, or both? 

Of course the right answer is both.  As I pointed out in my post on the Gap IV study (Gap IV Study Points Out Measurement Gap), the majority of PR measurement firms today concentrate on media content analysis (outputs) while companies want to know how PR is contributing to business objectives (outcomes).  Hence the 'Gap'.  And the 'Gap' may be widening as pressure increases to prove the business benefit and return on investment for public relations.  While this suggests we need more focus and resources on Outcome measurement, it does not suggest it should be thought of as an either/or proposition. 

Measurement of PR Outputs provides valuable diagnostic information to help answer the question, "Is our PR campaign working?".  Are we receiving coverage in our target publications?  How favorable is the coverage?  How prominently is our brand/company being featured in the stories?  How often are our key messages being included, and with what interpretation?  All of this data should be analyzed, and discussed by the PR team with an eye toward improving PR efforts going forward.

Too often, PR outputs are merely used to keep score and not as a diagnostic tool.  So the answer is both - outputs and outcomes are valuable - but the orientation of how we use the output data could be improved.   

Thanks for reading, DB    

Results of the fourth annual Public Relations Generally Accepted Practices Study (GAP IV) were released today.  The study is published by the USC Annenberg Strategic Public Relations Center (SPRC), and is intended to provide the PR profession with data on evaluation, organization, budgeting, emerging trends, use of outside agencies, perceptions of the PR function and other important topics. 

The study has several interesting findings, but let's take a look at what the findings say about Evaluation/Measurement:

• Respondents spent only 4% of their total PR budgets on evaluation
• 'Ability to Quantify Results' (12%) ranked low on the reasons a company would choose to use an outside agency ('Additional arms & legs' (51%) and 'Complements our internal capabilities' (47%) were the two highest-ranked factors)
• 64% of all respondents (77% of F-500 respondents) report to the C-Suite in their organizations
• The differences in metrics used by organizations where PR reports to the C-Suite versus those used where PR reports to Marketing are dramatic.  C-Suite metrics tend to be more strategic and organizationally focused while metrics used by those reporting to Marketing were more media and sales-oriented
• 'Influence on corporate reputation' has been the highest scoring metric in each of the four GAP studies, yet as the authors point out, "there are currently no consistently reliable, generally-accepted, quantifiable methods for correlating PR activities with reputation." 
• PR is seen as contributing to the bottom line success of the company (5.30 v. top-ranked Finance at 5.59)

To amplify the fourth bullet above, in companies reporting to the C-Suite, it was significantly more likely the following metrics were used for evaluation:

1. Crisis avoidance/mitigation
2. Influence on corporate culture
3. Influence on corporate reputation

In companies where PR reports to Marketing, it was significantly more likely the following metrics were used:

1. Contribution to sales
2. Total circulation of clips
3. Total number of clips

To me, there are two key learnings from the study:

• Confirming my personal experience and observations (see my earlier post), PR increasingly is reporting to the C-Suite and is seen as playing a strategic role within organizations, and
• There is a tremendous gap between what most companies actually measure today and the new requirements to measure how PR is impacting the organization

The measurement gap refers to the fact that the measurement industry today is focused on media content analysis (outputs measurement) while organizations increasingly value PR for our contributions in moving the needle on reputation, culture or sales (outcomes).  We may have great data on number of impressions or % of articles containing key messages, but how are we proving the value of PR's contribution to the more strategic outcomes the C-Suite demands?  This is a great challenge to the measurement community - we are not necessarily measuring the right things (media clips), and lack agreed-upon metrics and approaches for the types of measurement our new environment demands.  We need to evaluate the tangible contributions of PR (e.g. sales) as well as the intangible (e.g. brand or reputation).  And we need to not only look at PR's contribution to sales, but to PR's contribution in reducing costs (e.g. regulation or litigation) and reducing risks (e.g. new product introductions, downsizing).  So many challenges - so little time.

 Thanks for reading - Don B.

Disclaimer: The author is a member of the Professional Advisory Committee for the GAP Study       

I wanted to acknowledge a factual error in my post of April 24 (Oh Canada - Is MRP the Answer?) regarding the use of a multiplier in the new MRP system championed by the CPRS.  The system does not currently use a multiplier as Rachel Douglas, a member of the CPRS Measurement Committee, points out in her comment to the post.  Thank you Rachel.  Good news and a good decision by the Committee.

In my defense, the presentation from the CPRS on the MRP I saw dated March 2, 2006 clearly called for the use of the multiplier.  I'm glad this was changed prior to the official launch on April 20.

I stand by the rest of my comments including the misuse of "ROI" in describing the MRP metric. 

- Don B 

On April 20, the Canadian Public Relations Society (CPRS), unveiled the media relations rating points (MRP) system, an attempt to come up with a single metric for media relations measurement.  (Learn more here).  Here are a few initial impressions. 

Use of PR Multiplier Hurts Credibility

The MRP uses a multiplier of 2.5 on circulation figures to arrive at Impressions.  Advocates for the use of multipliers argue one of two positions – either PR impressions are “worth more” than other (i.e. advertising) impressions due to editorial credibility, or they are accounting for pass-along readership in addition to base circulation.  Neither position is defensible or advisable.  The only credible, consistent way to report impressions is by using audited circulation or audience figures.  I know of no industry association or governing body (e.g. IPR, PRSA) that supports the use of a multiplier.  It is bad business for the PR industry.

MRP is NOT ROI

Guess the CPRS Measurement Committee could not resist using the magic measurement acronym – ROI – to describe the MRP.  This is spin.  The MRP uses a cost per Impression metric, which is preferable to just Impressions and does allow comparison to advertising, but it does not give ROI.  ROI has become one of the most misused terms in public relations.  In order to calculate a true ROI, you compare the dollar value of what is created (sales, perception changes, etc.) to the cost of doing it.  Cost per impression is just that, a cost-oriented metric, not value-oriented.  It does not give ROI.

Are All MRPs the Same?

The fact that one selects five qualitative factors from a longer list to include in the MRP for a given campaign  seems to mean that for five different campaigns all reporting MRPs, you could have five slightly different ways of scoring articles for the index.  Not sure how meaningful this is, but it does seem to raise possible issues around consistency of application of the metric.  I also found it a little odd that no attempt was made to weight the various factors according to their ability to impact the readers as determined by primary research. 

Where’s the Proof?

For any single metric to be compelling, you would like to know how it correlates to desired business outcomes.  Does the metric show statistical correlation to some desired ‘downstream’ behavior – sales, likelihood to buy or recommend the stock, brand preference or prescription volume, for example?  Hard to tell if any of this sort of research has been undertaken by the MRP sponsors.  I did not find any evidence of it.  

It’s always easier to criticize than create, so I applaud the efforts of the CPRS and the MRP Committee.  This was a massive undertaking and they accomplished a lot to get to this point.  The MRP is not a magic bullet, and may be fatally flawed.  But who knows – it may take off in Canada and prove to be a valuable tool.  Should be fun to watch!

- Don B