With Primary Research Secondary, PR ROI Efforts Will Remain Elusive

8 Dec

A couple of months ago I was in a meeting with a client and their digital media agency.  The guy from the digital agency made an offhand comment I thought was interesting and hits on a key issue in public relations research and measurement.  He said he had always considered the number of posts a proxy for awareness, and the number of comments a proxy for engagement.  It struck me this notion of measurement by proxy is actually the norm rather than the exception.  Often, PR outputs are used to assess campaign performance rather than trying to determine whether or not the campaign actually touched and moved the targets in the ways we intended.  The continued industry reliance on the use of advertising value equivalents (AVEs) is perhaps the most common misuse of the measurement by proxy notion.  AVEs are an attempt to determine ROI by proxy.

We need more measurement by fact, not by proxy.  We need to move beyond just measuring what the PR result is (a hit, an OTS, a delivered message) to measuring what the PR result actually accomplished – did we reach the intended targets; did they engage with the content; did we cause them to rethink existing perceptions; did we change their attitudes; did we increase the likelihood they would consider the brand?  In order to move toward measurement by fact, a much greater emphasis on primary audience research is necessary.  As an industry, we would be well-served by a higher commitment to primary research.   Research allows us to shift the conversation from measurement by implication (i.e. x% of the coverage contained a key message therefore the campaign was a success) to measurement by fact (e.g. 10% of the target audience saw the PR campaign and 15% of these individuals plan to purchase the product in the next six months).  Rather than using content links or the comments/posts ratio as a proxy for brand engagement, wouldn’t it be better to do primary research designed to measure how engaged and attached online consumers actually are to the brand and why?  Importantly, efforts to determine the ROI of public relations are greatly enabled by primary research.  The better we actually understand whether or not PR has influenced the consumer and what actions, if any, they take as a result, the better we can assess the true ROI of a campaign.

There are three inhibitors to the expanded use of primary research in public relations measurement – education, lack of vendor focus, and perceived high costs.  Many PR practitioners lack the education and training to feel confident they know where and when to use primary research in their measurement efforts.  From a research vendor perspective, there are research silos of media content analysis firms and primary research firms.  While many firms offer both, their focus usually is in one area or the other.  And very few firms are actively attempting to use quantitative research together with content analysis to more holistically measure public relations.  (Echo Research is one company doing work in this area and I am sure there are a few others.)  Yes, primary research is generally more expensive than content analysis.  However I have seen content analysis systems costs in excess of $500K per year – you can do a lot of primary research for a fraction of that.  In 2009 if the PR industry spends about the same amount on primary research as we do in media measurement, we’ll be making great progress toward measurement by fact.

8 Responses to “With Primary Research Secondary, PR ROI Efforts Will Remain Elusive”

  1. Ben Proctor December 9, 2008 at 2:51 am #

    Dare I suggest that the PR industry is reluctant to invest in primary research because it might show that a lot of PR effort is, in fact, wasted money?

  2. Larry Irons December 9, 2008 at 10:16 am #

    Don, I’m not convinced we can ever determine why people are engaged with a brand through quantitative analysis. Their extent of engagement and how they engage are a different matter though, and on that I totally agree that most measures are proxies.

  3. metricsman December 9, 2008 at 10:55 am #

    Hi Larry,
    Thanks for your comment. I don’t disagree that Engagement is a very difficult concept to measure. It is a little abstract and contains an emotional element that is difficult to describe and quantify. In-depth interviews may better allow for the probing necessary to get to underlying feelings and emotions. However, being able to quantify, for example, how likely the average consumer is to recommend the brand to a friend or colleague provides deeper understanding than relying on number of links or comments/post ratio as a way to quantify engagement. I think we can agree primary research is not the total answer, but does provide better answers than just relying on measurement by proxy. -Don B

  4. Richard Bagnall December 14, 2008 at 5:36 am #

    Hi Don, As usual, you have hit the nail on the head with your explanation of the situation.

    Primary research is only going to be more important as the media world explodes and audiences fragment still further. We need primary research to understand where our audiences have gone – both in traditional and digital media, and we need more primary research to understand at the end of the campaign what our effect has been.

    As more press clipping agencies and social media monitoring companies start to offer basic metrics (although I would prefer to call it basic ‘categorisation’ only) the threat to meaningful analysis is going to grow and there is an even greater need for an education programme in the PR industry to understand the difference. This should be done by trade bodies like the IPR, CIPR and AMEC, but also by thought leaders in the industry like you – so I applaud this post.

    As you know, I do have a vested interest in this working for Metrica (www.metrica.net) where in conjunction with our media analysis services, we help our clients with integrated pre campaign audience identification through our UKPulse research and also with post campaign comprehensive metrics. All best, Richard.

  5. Mark Weiner December 14, 2008 at 3:17 pm #


    As always, you’ve provided a provocative perspective. Here what you’ve provoked in my mind: we in PR — and PR measurement in particular — tend to think that researchers in other fields have a much easier time or do a much more thorough job in planning and evaluating programs through research. I think it might surprise many of us to learn that the situation you’re describing –imprecise measures, measurement by proxy, wasted resources based on faulty research — is as at least as common in advertising and marketing as it is in PR. And since advertising research budgets are so much larger than PR’s, it can only be assumed that many more dollars are wasted (even if the percentage of waste is the same, which in my opinion, would underestimate the levels of waste in most market research).

    At the same time, I believe that a case can be made for “approximately right” being better than “totally wrong.” As long as we in PR (and those in marketing) understand the degree to which we are correct and factor that into our thinking and our documentation, we are in a better place than if there were no measurement at all (including even the dreaded ad value, which in my opinion and when properly qualified, can be a useful gauge for those with nothing better).

    I continue to enjoy the discussion so please let’s continue.

  6. metricsman December 15, 2008 at 4:44 pm #

    Hi Richard,
    Thanks for your comment – as always you’ve added to the scope of the discussion. You are spot on with the call for more education. It most likely will have to occur within the industry as you suggest as colleges and universities do not address research and measurement rigorously within the undergraduate curriculum. For my students, my course is the only one they receive addressing the subject during their four year program. Thanks again, Don B

  7. metricsman December 15, 2008 at 4:50 pm #

    Thanks very much for your comment. My experience with advertising and marketing research and measurement mirrors yours, although I have seen good commitment to the research function in many brand departments.

    Your insight about being ‘approximately right’ is right on. There is way too much ‘feel good’ marketing and measurement unsupported by any real data. I could even agreed with your radical statement on ad values 🙂 if practitioners were using them as a diagnostic tool rather than as a score keeping mechanism.

    Thanks for engaging, Don B

  8. Hampus Landelius January 23, 2009 at 8:08 am #

    Hello Don

    Thank you for a interesting article!

    I’m a student at Stockholm University Sweden and are just now in the process of writing my master thesis in marketing. I find ROI measurements in social networks/media very interesting and I will make this a subject for my research for this term. I´m however struggling to find some hard facts regarding measurements like RSS, feedback etc. Do you know any good source of information regarding this?

    Best regards Hampus

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: