Archive | Ad Value Equivalents RSS feed for this section

Five Social Media Measurement Questions I Hope (NOT) To See in 2014

2 Jan

I get asked lots of great questions about social media measurement. Following are five not so great ones I hope not to hear in 2014. 

How do you measure social media?

I get this question quite often and I enjoy it each time because if provides me the opportunity to make an important point about measurement and be a little snarky at the same time. Good stuff! When I get this question, my answer is always the same; “I don’t measure ‘social media’, I measure what you are trying to accomplish with social media.” This may seem like I’m playing semantic games, but the distinction is very important. Measurement is fundamentally about performance against objectives. So, we measure our performance against the objectives established in the social media plan. A lot of what passes for measurement in social media is really data collection – tracking Followers or Likes, blog traffic or consumer engagement on Facebook. Unless you have measurable objectives and targets in each of these areas, you are collecting data not measuring. What do you want to happen as a result of your social media campaign or initiative? Measure that.

QMarksHow much is a Like worth?

This question doesn’t come up quite as often as in 2012, but it is still asked and, unfortunately, answered largely based on flawed logic and/or research design. You may recall the first two ‘research’ studies attempting to answer this question came up with widely disparate values – somewhere around $3.14 in one case and over 100 dollars in the other. This alone should raise major red flags. Setting the flawed research aside, trying to assign a value to a Like happens because people are desperate to assign financial value to social media and determine ROI. Those are noble things to do, but we need to focus on the other end of the customer journey – have we created engagement, has the engagement changed opinions, attitudes, beliefs or behavior, and how those changes translate to Impact. Unless you understand the Impact created by your social media program you really can’t attribute value properly. I would argue that Likes, which can be bought or gamified, really have no inherent value.

Can I use a banner ad cost to calculate social media AVE?

This question is somewhat related to the ‘Like worth’ question in that it reflects a desire to quickly and easily assign financial value, when in fact assigning financial value is often hard and expensive. In this case, the questioner is attempting to take the highly flawed and discredited concept of Advertising Value Equivalency (AVE) and apply it to social media. Where this question typically comes up is in blogger relations where a company/brand/organization has worked with a blogger to earn ‘coverage’ on the blog and wants to assign a financial value to the post. They would like to say that the post is worth X, with X being the cost of a banner ad on the blog (setting aside, of course, that many blogs do not accept advertising). Equating cost with value is comparing apples to oranges. First, a better practice is not to assign value to each post, but to all the posts together in a campaign. Then instead of trying to say the campaign is worth say 14.35 ads, try to explain the actual impact the campaign has created on the target audience – e.g. increase in awareness, increase in purchase intent or, higher propensity to purchase more often. Once you understand the impact, decide if you have the data, time, expertise and budget to assign financial impact to the impact created.

Which social media listening tool do you recommend?

The correct answer to this question is, “it depends.” This is a bad question simply because there is no one ‘best’ social media listening tool for all circumstances and use cases. I believe you should always develop a set of platform requirements driven by the social listening stakeholders in your organization. Once these needs and requirements are understood, develop a custom RFI designed around the specific requirements you have identified. Have each of your potential platform partners respond to the RFI. Have the best respondents given you a platform demonstration according to a custom demonstration script you have developed. Pick the listening tool that best meets your unique requirements. The last three evaluations I have conducted for clients resulted in three different ‘winners’. There is no ‘best social listening tool, so find the tool that meets your requirements the best.

How many Impressions did we get with our latest social media campaign?

This is not a terrible question at all unless it is the only question asked or is perceived to be the key metric for measuring social media campaign performance. Too often, organizations use Impressions as their primary social media metric instead of engagement, influence or action-oriented metrics. Also, keep in mind Impressions represent an opportunity to see content, they are not the actual number of people who saw the content, that number is MUCH lower. Impressions always overstate the actual number of people who were exposed to your content and message.

If you plan to report on campaign impressions, please seriously consider only taking credit for those impressions that are directly against your target audience. If your target is 25 – 34 year old Males, you should only report on the impressions against this target group. Why take credit for 45 – 60 year old Female impressions when the product is not at all relevant to this audience?  Target audience impressions are really what you should be concerned about and what you should be reporting. Many people know and understand this but still persist in reporting all impressions because the number is usually much larger – meaningless but larger.

If you do report on Impressions, please consider using the emerging industry standard definitions developed by The Coalition. This will help ensure we define Impressions consistently and don’t confuse Reach with Impressions.

See things differently? Have your own pet peeve social media measurement questions to share? As always, thanks for reading. All the best in 2014!

@Donbart

Image credit: amasterpics123 / 123RF Stock Photo

Time to Get Real About Social Media Audience Reporting

12 Jun

Though almost everyone would agree that social media is about engagement and not eyeballs, too much of digital and social media measurement is focused on audience size. How many Followers do we have? How can we get a million Likes? How many unique visitors did we have to our site this month? And unfortunately, audience size estimates in social media grossly overstate the actual relevant audience. We seem fixated and oriented toward ‘how many’, while our focus should be on ‘who’ and specifically, ‘who within our target audience’. Generally speaking, the advertising industry has led the way with audience measures and is ahead of where the public relations and social media camps are with respect to level of sophistication.

In television advertising, the concept of Target Rating Points is a refinement of Gross Rating Points where you only measure and get ‘credit’ for the percentage of the gross audience that meets your target audience criteria. In an effort to keep refining the audience data available, Nielsen has evolved from diary-based data to electronic data to software at the set-top box level that allows operators to monitor channels choices and changes. In audio-based media, Arbitron’s Portable People Meter recognizes today’s mobile world and begins to address cross-platform measurement. It is also interesting to reflect on the U.S. Congressional involvement in television audience ratings accuracy (or lack thereof as it were) that led to the formation of what is now known as the Media Rating Council in the early 1960’s. The time has come for social media audience research to greatly increase in sophistication, accuracy and relevance.

When we think about social media audience size measures today, the emphasis is on Opportunities To See (OTS), although almost never by this name. We might call them Impressions or Reach, but what we really mean is how many people had the potential to see this content item. There are two overarching issues here:

  • Opportunities to see are not the same as actually seeing
  • The metrics count all possible members of the audience, regardless of whether or not they are part of the targeted audience or can even buy the product or service.

OTS is also a prevalent metric in the public relations industry which has always focused on stating the highest possible audience measures. In traditional media we know the probability of any one person in the audience actually seeing the article in question is a fraction of the total audience – a reasonable estimate is 10% or less. So OTS greatly overstates the actually number of people who saw a given article. To compound the audience overstatement, we have the practice of using audience multipliers to ‘credit’ earned media for either a perceived credibility advantage over advertising or to account for pass-along circulation (see this IPR white paper for more on multipliers). Thankfully the practice of applying multipliers (and its evil cousin AVEs) is out of favor and rapidly on a path toward extinction.

In social media one can make the case the audience metrics situation is actually exacerbated in that the probability of any one follower seeing any one tweet, for example, is most likely an order of magnitude less than in earned media – my guesstimate is 1% or less. Before you call BS on this guesstimate, play around with a few Twitter factoids – the recent Pew Research study suggesting only 8% of Twitter users use it daily, the perishable nature of most individual’s twitter streams, and the fact that a reasonably high percentage of Followers of a brand are bots, and the reality is that only a small fraction of twitter followers actually see tweets, let alone find it interesting enough to share or comment on. And, of course, not all Facebook Likes see every post you make either. Riffing on the old, ‘if a tree falls in the forest…’, if you tweet into the twitterverse and no one sees it does it make an impact?

Evolving from ‘opportunities to see’ to ‘relevant audience’ measures.

Most social media campaigns have a specific target audience in mind, often described with demographics (Female, age 18 – 34), psychographics (who worry about feeding their family healthy food on a budget) and behavioral (access deal and coupon sites regularly) dimensions. Yet when it comes to reporting and measurement we take credit for the entire audience (total OTS) rather than the percentage of the audience that meets our targeting criteria. Trying to promote lingerie to 22 – 29 year old ladies? No worries, count all your Twitter Followers and all the visitors to your website – the men, the young and the old – everybody counts. Trying to sell camo clothing to male hunters? No worries, everybody counts – male, female, hunters, non-hunters and PETA members, too. Of course this all seems a little silly and strange and I suppose it would be if it wasn’t the way most social audience reporting is done today. It is unusual to see someone in social media, or PR for that matter, report only the relevant audience opportunities to see. Why is this? I believe there are three primary reasons:

  1. Legacy – the PR industry has historically reported gross potential audience size rather than the relevant audience size. When social media came around, this same orientation toward gross audience measurement was used.
  2. Data – there is a lack of consistent social media demographic and psychographic audience data available and it often resides in channel silos rather than cross-platforms. And often the audience data from one platform (e.g. ComScore) does not match the data available from another platform (e.g. Compete).
  3. Standards – there are no standards for social media audience metrics and no codified best practices for audience measurement.

Where do we go from here?

First, we need a change in mindset of how we think about audiences. From ‘how many people theoretically had the potential to see our content’ to ‘how many of the people we were targeting actually saw our content’. Big audience numbers are irrelevant. Relevant audience numbers are big.

Next, as the demand for audience data that contains demographic, psychographic and behavioral data grows, it is reasonable to assume one or more of the large media data companies might start to aggregate and make the data available. Privacy concerns, cookies and other issues are also in play here.

And last but not least, industry standards for social media audience and engagement metrics and definitions are necessary for transparency and replicability that will increase credibility of social media measurement and reporting. 2012 will go down as the year that serious cross-industry progress on social media metrics standards began and gained momentum. There has already been a lot of progress (See this post from Katie Paine), and this week in Dublin at the 4th AMEC European Summit on Measurement the theme is around attempting to define standards for social media metrics and measurement. To tune into the debate as it occurs in Dublin, monitor #SMMStandards and #AMEC2012.

What  are your thoughts on the need for social media metrics standards and the use of target rather than gross audience size estimates?

Measurement 2020 and Other Fantasies

23 Sep

At the 3rd European Summit on Measurement held in Lisbon in June 2011, standardization, education, ROI and measurement ubiquity emerged as the key themes in response to a call to set the Measurement Agenda 2020.  Delegates to the conference voted on 12 priorities they thought were most important to focus on in the period leading up to 2020.  The top four vote-getters became the Measurement Agenda 2020:

  1. How to measure the return on investment of public relations (89%)
  2. Create and adopt global standards for social media measurement (83%)
  3. Measurement of PR campaigns and programs needs to become an intrinsic part of the PR toolkit (73%)
  4. Institute a client education program such that clients insist on measurement of outputs, outcomes and business results from PR programs (61%)

For a very nice overview of the Lisbon session and the Barcelona Principles that came before, read this post from Dr. David Rockland of Ketchum who chaired the Barcelona and Lisbon sessions.  David pretty much said it all on these sessions, so I’ll just add a couple of comments and share a few thoughts on what I believe the future of measurement 2020 could be.

The rallying cry coming out of Barcelona has been focused and loud – death to AVEs!  Will there be a similar thematic coming out of Lisbon and what might it be?  My money is on standardization, borne out of cross-industry cooperation.  As David points out in his post, and in the words of AMEC Chairman Mike Daniels, “The Summit identified some significant challenges for the PR profession to address by 2020.  However, what we also accomplished in Lisbon beyond setting the priorities was to harness the commitment and energy of the industry to agree what we need to do together.”  The current cooperation and collaboration between industry groups – AMEC, Institute for Public Relations, PRSA and the Council of PR Firms is unprecedented in my time in this industry and is focused on tangible outcomes.  Cross-organization committees are already at work developing standard metrics for social media measurement for example.  The spirit of cooperation is uplifting.  While the outward thematic appears to be standardization, cooperation is the enabling force.  

I was also struck by the symmetry of the call to end AVEs in Barcelona and the call to codify ways to measure ROI in Lisbon.  One follows the other.  In my opinion the primary reason AVEs exist is because PR practitioners feel pressure to prove the value of what they do, and quite often they are asked to describe the impact in financial terms.  AVEs are perceived as a path of least resistance way to express financial value.  Except, as we all know, AVEs don’t really have anything to do with the impact public relations creates.  They are a misguided proxy for financial value.  Hence the need for research-based methods to determine true return on investment.

All of the priorities coming out of Lisbon are excellent goals for the industry.  And like David Rockland, I believe they will be achieved, and be achieved before 2020.  Here are three other items on my wish list for Measurement 2020:

Word of Mouth/Word of Mouse Integration: For those of us focused in social media and other digital technologies, we can’t allow our digital lens to color what is fundamentally an analog world.  Research studies suggest the majority of word of mouth happens in real life.  From an influence perspective, I don’t think too many would argue that word of mouth from a trusted friend or family member is more powerful than word of mouse from someone you follow on Twitter.  Digital cross-platform research is difficult enough, but when one huge platform is ‘real life’, we have significant challenges in measurement.  WOMMA and others have made early attempts to define measurement approaches for offline WOM, but much work remains.  We need ways to assess its impact and then we need to think about ways to attribute value to that impact.  Mobile is a wild card here as it becomes the preferred platform for online activity.  The need to triangulate online, mobile and ‘real life’ measurement presents significant challenges today, and may still by 2020.

Cookie Wars: We all know the measurement versus privacy showdown is coming, right?  The first shots have already been fired.  The collection of source-level personal data, enabled by cookies, is crucial to measurement and insights but has the potential for misuse or unintended disclosure.  Some sophisticated consumers have had their fill of cookies.  Although the broader issue might be framed as social sharing versus privacy control, how it plays out will have a direct impact on digital analytics and measurement.

Integrated Measurement across the Paid Earned Shared Owned (PESO) Spectrum: Measurement has increasingly become integrated.  It began with integrated traditional (Earned) and social media (Shared) measurement and then progressed rapidly to Earned, Owned and Shared, which is where most integrated measurement programs are today.  Many leading-edge integrated programs today also include advertising or Paid media.  By 2020, integrated measurement across the PESO spectrum will most likely be the norm and not the exception.  A key enabling element here in my view is some base level of agreement on how each area should be measured and standard metrics for each.  It will take significant cooperation between industry groups, vendors, agencies and major customers/clients for cross-discipline standardization to move forward effectively.  We are at the beginning of this movement in 2011.  By 2020, it will be fascinating to look back and see how all this plays out.

When looking ahead to 2020, I am reminded of a measurement discussion pulled together by PRWeek a couple of years ago.  Many of the Measurati attended.  In response to a question of where measurement will be in five years, David Rockland replied (paraphrasing here), ‘Who knows?  Five years ago who would have guessed we would all be focused on how to measure social media?’  So, there is a certain fantasy element to discussing 2020 challenges in measurement.  What are your measurement fantasies?

AVEs Don’t Describe the Value of Media Coverage, They Sensationalize It.

26 Jun

Saturday, Wall Street Journal columnist Carl Bialik, The Numbers Guy, addressed the subject of advertising value equivalency (AVE).  This is perhaps the first example of a mainstream media publication shining a light of the controversial practice of AVEs.  (You can read the story here.)

The primary reason advertising value equivalents exist are because they are perceived to be a way to attribute value to programs that would otherwise be difficult to value directly.  They are a path of least resistance approach to return on investment calculations, but not a valid one.  Let’s take a deeper dive into the three specific examples in the WSJ story, ask the tough questions and discuss more valid ways to think about value attribution and ROI.

American Airlines  

You can enjoy both questionable valuation techniques and hyperbole in this article.  American Airlines stands to “make boatloads of cash” and “the airline company could gain as much as $95.9 million of exposure”.  Of really, let’s take a closer look.

The most incredible part of this financial calculation is the financial calculation itself.  The calculation is apparently based on sign placement within the arena and presumably the ‘impressions’ the brand will receive when people attending the venue see the signage and when TV cameras catch the signs when showing the scoreboard or during the action.  This is a very passive form of advertising that should have as its objective either creating top of mind awareness or perhaps creating more brand affinity.  Rather than using an advertising equivalency model that has no validity, a true measurement of the value created by naming rights would ask a series of questions designed to determine the actual, tangible (or even intangible) impact on the business:

  • Revenue: Can incremental revenue generation in the form of higher passenger miles be directly attributed to the exposure created by the naming rights?  Is it possible that incremental revenue would actually be realized on a game by game basis, or would any positive impact be realized over a longer time horizon?  Have new customers been created as a direct result of the exposure generated by the naming rights?
  • Brand: Can the increased exposure lead to people perceiving the brand differently and can the difference translate into higher transactional revenues generated or increased brand loyalty?

So where exactly are the ‘boatloads of cash’ American Airlines made?  Are they hitting the income statement in the form of incremental revenue or enhanced brand loyalty (repeat business)?  Are they residing on the balance sheet in terms of brand goodwill?  Given that American’s parent company AMR lost $11.5B dollars in the first decade of the 21st century, its last profitable year was 2007 and they are projected to lose money in 2011 and 2012, they could use the cash.  Perhaps they could use it to fund a ’bags fly free’ program or for enhancing their Advantage program to create more brand loyalty.  I would strongly suspect American’s shareholders would prefer a do-over on the investments made on naming rights to the ‘boatloads of cash’ they are now enjoying from the investment.

Couple Won’t Cash In on Kiss

15 minutes of fame is rarely worth $10 million.  In this case, the celebrity agent is suggesting the news value of the coverage generated by the kiss is somehow equivalent to advertising value and assigns what appears to be an arbitrary and ridiculously high value to it.  (He later admits he just made the number up.)  Just how was the couple going to monetize their 15 minutes of fame?   Yes, they turned down a few talk show opportunities and perhaps the National Enquirer would have thrown a few dollars their way for an exclusive, but the assertion that any major brand would have paid them to endorse their product is wildly speculative.  I would guess that if you did a survey after the event, a small number of people would remember seeing the coverage, and a very small percentage of the people who did see it would have recalled Scott Jones’ name.  So perhaps Mr. Jones walked away from tens of thousands of potential dollars in the short-term, but nowhere near the sensationalized estimate of $10 million.  15 minutes of fame might be worth 10 thousand dollars, but certainly not $10 million.

Obama Enjoys a Guinness

So Guinness is a winner and received $20 million worth of “free publicity”?  What was the outcome of the publicity?  Again, in order to determine the value of the “free publicity” (this term is despised in the PR industry by the way), Guinness would have to be able to measure incremental revenues directly attributable to the publicity generated.  Did sales of Guinness increase as a result?  Were new customers created?  Did existing customers feel compelled to drink even more?  What was the value of the incremental sales?  These are much more difficult questions to answer but are the correct ones to ask in order to measure the publicity.  Not by focusing on the mythical value of the coverage as measured by flawed advertising equivalency, but measuring the outcome or what happened as a result of the publicity.  The assertion that President’s Obama’s image was softened and will help keep him in the public’s favor is highly dubious thinking.  Perhaps it helps him in Boston, but in the grand scheme of things, this is a Presidential image non-event.

Beginning last Summer in Barcelona,  the public relations industry has come together to publicly state advertising value equivalency is not a valid measure of public relations.  The so-called Barcelona Principles are explicit against AVEs and also call for a focus on measuring outcomes and not (just) outputs.  While it will take some time for the PR industry to totally leave AVEs behind, there is a lot of momentum right now to make this happen sooner rather than later.  No serious measurement effort can use advertising value equivalency to attribute value and be credible.  

AVEs are a Disease – Here’s a Little Vaccine

16 Apr

One of the truly insidious aspects of public relations measurement is the use of advertising value equivalency (AVEs) or media value to assign financial value to public relations outputs.  It is a highly flawed, path-of-least-resistance attempt to calculate return on investment (ROI) for public relations.   To make matters worse, the practice has clearly moved into social media measurement as well.  For example, research studies that attempt to monetize the value of a Facebook Fan/Liker by attributing a CPM value from the advertising world.  Online media impact rankings also utilize equivalent paid advertising costs to assign monetary value to online news and social media.  AVE is like a disease that has infected and spread throughout the public relations industry.

In June of 2010, the PR industry came together in Barcelona to draft the Barcelona Principles, a set of seven principles of good measurement intended to provide guideposts for the industry.  The principle that has generated the most conversation is this one:

Advertising Value Equivalency (AVE) is Not the Value of Public Relations

 While many of the Measurati have been preaching against AVEs for years, there now appears to be a critical mass of outrage that may kill the practice in the coming years.  Here are four compelling reasons why I believe we must make this happen – the sooner the better.

1. AVEs Do Not  Measure Outcomes

AVEs equate an article with the appearance cost of an advertisement.  It does not speak at all to the results or impact that the article may have on a reader.  Advertisers do not judge the success of advertising on how much the insertions cost.  Imagine an advertising manager being asked by his or her boss, “How are we doing in advertising this year?”, and them replying, “Great!  We have spent $500,000 so far!  The true value of public relations or social media is not the appearance cost, but what happened as a result of the PR or social media effort – the impact it has on brand, reputation and marketing.  You will note the Barcelona Principles also call for a focus on measuring outcomes and not (just) outputs.  What happened as a result of media coverage is inherently more interesting and valuable than how much coverage was obtained.

2. AVEs Reduce Public Relations to Media Relations

You are, or become, what you measure.  AVEs do not address the impact or value of several important aspects of public relations including strategic counsel, crisis communications, grassroots efforts, viral campaigns or public affairs.  In other words, AVEs reduce PR to just the media dimension by only assigning a value in this area.  If only AVEs are used to assess PR value, the results will understate the totality of value delivered by PR.  AVEs also cannot measure the value of keeping a client with potentially negative news out of the media, yet that may be the primary objective of the PR practitioner.

3. AVEs Fly in the Face of Integrated Measurement                

Good marketing, branding and reputation campaigns have always been integrated to varying degrees.  The digitization of our lives has accelerated integration.

Advertising and PR actually work together synergistically, yet AVEs treat them as cost alternatives.  Studies have shown ads that run in a climate of positive publicity actually receive lift from the PR.  Conversely, ads run in an environment of negative publicity will likely not be successful and/or may be perceived negatively by consumers.  We have seen exposure to brand advertising increases conversion rates in social channels. Integrated campaigns and programs require integrated measurement.  AVEs don’t play well in this world.  They are analog and segregated in a digital and integrated world.

4. AVEs Provide No Diagnostic Value

Too much measurement energy is focused on score-keeping and not diagnostics.  This is one reason why single-number metrics like the Klout score and others have great appeal to many.  However, measurement is fundamentally about assessing performance against objectives with sufficient detail and granularity to determine what is working and what is not.  AVEs fail miserably in this regard.  AVE results can actually be misleading and result in false positives.  AVEs may be trending up while important metrics like message communication, share of favorable positioning and share of voice are falling.  Unfortunately, AVEs provide neither a valid single-number score nor any diagnostic value.

Some have said the Barcelona Principles are the ‘end of AVEs’.  I would agree directionally with that statement with one minor addition, Barcelona was the ‘beginning of the end of AVEs’.  Awareness of the practice and recognition of its flaws are at an all-time high in our industry.  More education and evangelism are required.  Understanding concepts like impact, tangible value, intangible value and (true) return on investment help foster much more sophisticated conversation about the total value delivered by public relations and social media.  AVEs are a disease, education and knowledge are the vaccine.  AVEs won’t die easily.  The momentum generated by the Barcelona event has provided focus and intent.  It is up to all of us to make AVEs a thing of the past.

Social Media Measurement 2011: Five Things to Forget and Five Things to Learn

30 Dec

It has been said that social media came of age in 2010.  Not so for social media measurement.  But the mainstreaming of social media marketing brings with it a heightened call for accountability.  The need to prove the value of social media initiatives has never been greater.  So, perhaps 2011 will be the year that social media measurement matures and comes of age.

As we look to the next year, here are five things to forget and five things to learn about social media measurement in 2011.

Things to Forget in 2011

1. Impressions

The public relations industry has historically measured and reported success through the lens of quantity not quality.  The most common PR metric today is Impressions.  While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek.  Impressions also (greatly) overstate actual relevant audience.   Impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content.

For Twitter, many folks use the sum of all first generation followers as ‘impressions’ for a particular tweet.  The obvious problem here is that the probability that any one follower sees any one tweet is quite small.  I don’t have good data on this (please share if you do), but an educated guess might put the percentage at less than 5%.  Similarly for Facebook, use of impressions as a metric is also problematic.  Facebook impressions do not indicate unique reach and you don’t have any idea who, if anyone, actually viewed the content.

Number of Impressions is a flawed, unwashed masses metric for social media measurement.  Any time you are tempted to use the word ‘impressions’ in social media, think about ‘potential reach’ or ‘opportunities to see’ instead.  Or better yet, concentrate on Engagement and Influence.

____________________________________________________________

2. Vanity Metrics – Fans and Followers

Most social media measurement efforts place far too much emphasis on Fans/Likers and Followers.  For Twitter, the number of Followers is seen as a key metric, thought by many to relate to potential influence.  For Facebook it is the number of Fans/Likers many companies/brands attempt to maximize.  While these may be the vanity metrics of choice, they fall far short of being adequate for rigorous measurement.  The largest disconnect of course is these numbers really don’t describe potential audience size very well and they have nothing to do with interactions/engagement.

For Twitter, there is a growing amount of evidence (read the Million Follower Fallacy paper) that number of Followers really has little to do with Influence.  Number of Followers may be an indication of popularity but not influence.  Influence talks more to one’s ability to start conversations and spread ideas.  For Facebook, number of Fans bears little semblance to average daily audience size and tells you nothing about engagement of the community.  All Fans are not created equally.  Some are engaged, some never return.  Some are your best customers, others are there only to trash you.

Number of Fans and Followers are metrics you probably should include in your overall metrics set, but should be de-emphasized and not be a primary area of focus.

________________________________________________________

3. Standardization

Measurement standardization is always an interesting topic to debate.  On one side you have the folks who believe standards are absolutely necessary for measurement to proliferate, and on the other side you have the snowflake measurement disciples who believe each program is unique and therefore requires unique objectives/metrics.  I fall somewhere between the two extremes.

In June 2010 IPR, AMEC, PRSA, ICCO and The Global Alliance got together in Barcelona for a conference intended to create an atmosphere for measurement consistency/standardization around a codified set of principles of good measurement.  The Barcelona Principles as they have come to be called are basic statements of good measurement practice – focus on outcomes not outputs, don’t use AVEs, etc.  Absolutely nothing to disagree with in the Principles.  However, the heavy lifting of standardization comes at the metrics-level.  Subcommittees have been formed that are taking the Principles all the way down to the metrics level.  I have reviewed the work of the social media committee and believe there is a lot of good work being done.

But in 2011, I expect a lot of debate but not a lot of progress in creating social media measurement standardization.   One to watch is the Klout score for online influencers which is being integrated as metadata in social media listening and engagement platforms.  There are issues with the Klout score (read this post), and I question the type of ‘influence’ it is measuring – there is a big difference between motivating someone to action (e.g. retweeting your content) and motivating someone to purchase which is ultimately the type of influence many companies and brands are most interested in effecting.

__________________________________________________________

4. Ad or Media Equivalency

One of the truly insidious aspects of public relations measurement is the use of advertising or media equivalency (AVEs – advertising value equivalency) to assign financial value to public relations outputs.  It is a highly flawed, path of least resistance attempt to calculate return on investment (ROI) for public relations.  There are many reasons why using ad equivalency as a proxy for PR value is not advisable.

To make matters worse, the practice has clearly moved into social media measurement as well.  For example, research studies that monetize the value of a Facebook Fan/Liker by attributing an arbitrary $5 CPM value from the advertising world.  Online media impact rankings also utilize equivalent paid advertising value to assign monetary value to online news and social media.  The true value of social media is not how much an equivalent ad would have cost but in the impact it has on brand, reputation and marketing.

__________________________________________________________

5. Return on Engagement/Influence/etc.

Not a day goes by without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks.  A recent Google search for “Return on Engagement” returned 192,000 results.  “Return on Influence” returned 68,300.

Most of the folks who use these terms either don’t understand ROI or don’t know how to obtain the data necessary to calculate it.  Many confuse the notion of impact with ROI (addressed in Things to Learn).  Engagement creates impact for a brand or organization, but may or may not generate ROI in the short-term.  Creating influence – effecting someone’s attitudes, opinions and/or actions – creates impact but may or may not create ROI in the short-term.  It often is better to think about measuring impact first and then deciding whether or not you have the means and data necessary to attribute financial value.

__________________________________________________________

Things to Learn in 2011

1. Measurable Objectives

There are many issues and challenges in the field of social media measurement.  The easiest one to fix is for everybody to learn how to write measurable objectives.  Most objectives today are either not measurable as written or are strategies masquerading as objectives.  (For example, any sentence starting with an action buzzword like leverage is a strategy.)

‘Increase awareness of product X’ is not a measurable objective.  In order to be measurable, objectives must contain two essential elements:

  • Must indicate change in metric of interest – from X to Y
  • Must indicate a timeframe for the desired change – weeks, months, quarter, year, specific dates tied to a campaign (pre/post)

Therefore, properly stated, measurable objectives should look more like these:

  • Increase awareness of product X from 23% to 50% by year-end 2011
  • Increase RTs per 1000 Followers from 0.5% in Q1’11 to 10% by the end of Q2’11.

__________________________________________________________

2. Impact versus ROI

ROI is one of the most overused and misused term in social media measurement.  Many people say ‘ROI’ what they really just mean results or impact.  ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.

ROI is a form of impact, but not all impact takes the form of ROI.  Impact is created when people become aware of us, engage with our content or brand ambassadors, are influenced by engagement with content or other people, or take some action like recommending to a friend, writing a review or buying a product.  Impact ultimately creates value for an organization, but the value creation occurs over time, not at a point in time.  Value creation is process-oriented.  It has both tangible and intangible elements.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

Most social media initiatives today do not (or should not) have ROI as a primary objective.  Most social programs are designed to create impact, not ROI, in the short-term.  There is also the notion that many social media initiatives are in an investment phase, not a return phase of maturity.

__________________________________________________________

3. Hypothetical ROI Models

One important step in determining how a social media initiative creates ROI for an organization is to create a hypothetical model that articulates the cascading logic steps in the process, as well as the data needed and assumptions used.  The model is most useful in the planning stages of a program.  It helps address the proverbial question, “If I approve this budget, what is a reasonable expectation for the results we will achieve?”  Let’s take a look at a simple Twitter example:

Program: Five promoted tweets are sent with a special offer to purchase a product on an e-commerce site.

Hypothetical ROI Model:

  • (Data)                   Total potential unduplicated reach of the five tweets is 1,000,000 people
  • (Assume)            10% of the potential audience will actually see the tweet = 100,000 people
  • (Assume)            20% of the individuals who see the tweet find it relevant to them = 20,000 people
  • (Assume)            10% of those finding it relevant will visit the site = 2,000 people
  • (Assume)            10% of those visiting the site will convert and buy the product = 200 people
  • (Data)                   Incremental profit margin on each sale is $50
  • (Data)                   Total cost of the social media initiative is $2,400

ROI Calculation: (200 x $50) = $10,000 – $2,400 = $7,600/$2,400 = 3.17 x 100 = 317% ROI

Our model suggests this program will be successful and generate substantial ROI.  If in reviewing a model with someone who needs to approve a program, they conceptually buy into the model but challenge the assumptions, that is a positive step.  Negotiate different assumptions and rerun the numbers.  Hypothetical models help you think through the data requirements your research approach must address in order to actually measure the ROI of the program after implementation.

__________________________________________________________

4. Integrated Digital Measurement

The definition of public relations is fluid, and rapidly evolving to encompass a much broader and more integrated view of communications and how we connect, engage and build relationships with consumers and other stakeholders.  Digitization in all its forms has driven and accelerated this important change.  Communicators should now take a more content and consumer-centric view of the world, orchestrating all the consumer touch points available in our increasingly digital world.  At Fleishman Hillard, we capture this expanded scope and integration in a model we refer to as PESO – Paid/Earned/Shared/Owned.  Here is how we define the elements of our model:

Paid – refers to all forms of paid content that exists on third-party channels or venues.  This includes banner or display advertisements, pay-per-click programs, sponsorships and advertorials.

Earned – includes traditional media outreach as well as blogger relations/outreach where we attempt to influence and encourage third-party content providers to write about our clients and their products and services.

Shared – refers to social networks and technologies controlled by consumers along with online and offline WOM

Owned – includes all websites and web properties controlled by a company or brand including company or product websites, micro-sites, blogs, Facebook pages and Twitter channels.

The social media measurement Holy Grail in many ways is to be able to track behavior of individuals across platforms, online and offline, tethered and mobile, understanding how online behavior impacts offline behavior and vice-versa.  We also seek to understand how the PESO elements work together synergistically.  For example, how exposure to online advertising impacts conversions within social channels.  To address this, your measurement strategy should be to take a holistic, integrated approach using a variety of methodologies, tools and data.

_________________________________________________________

5. Attribution

If you are not already familiar with value attribution models, prepare to hear much more about them in 2011.  Value attribution models attempt to assign a financial value to specific campaigns and/or channels (e.g. advertising, search, direct, social) that are part of a larger marketing effort.  So rather than giving all the conversion credit to the last click in a chain or even the first click, the model attributes portions of the overall value across the relevant campaigns and/or channels.

A simple model might look at the following metrics for each channel:

  • Frequency – the number of exposures to a specific marketing channel or campaign
  • Duration – time on site for exposures referring to the conversion site
  • Recency – credit for exposures ranging from first click to last click, with last click typically receiving more credit.

Value attribution models require human analysis and expertise.  This factor is often cited in studies as the reason more companies do not pursue attribution modeling.

_________________________________________________________

Here’s wishing you and yours an exciting and prosperous 2011!

The Barcelona Principles: Leaders Speak

23 Jun

Well, the Second European Summit on Measurement held last week in Barcelona has come and gone, but its impact may be felt for some time to come.  The Summit was organized by the International Association for the Measurement and Evaluation of Communication (AMEC) and the Institute for Public Relations.  The most notable outcome of the Summit was the creation of the ‘Barcelona Declaration of Research Principles’.  The Principles were debated and voted upon by about 200 delegates representing 33 countries and five global PR and measurement organizations (AMEC, IPR, PRSA, ICCO, The Global Alliance).  David Rockland, Ph.D. chaired the debate.

Here are the ‘Barcelona Declaration of Research Principles’:

1. Goal setting and measurement are fundamental aspects of any PR programs.
2. Media measurement requires quantity and quality – cuttings in themselves are not enough.
3. Advertising Value Equivalents (AVEs) do not measure the value of PR and do not inform future activity.
4. Social media can and should be measured.
5. Measuring outcomes is preferred to measuring media results.
6. Business results can and should be measured where possible.
7. Transparency and Replicability are paramount to sound measurement.


I asked three of the leaders of the conference to comment on four questions regarding the Summit and what it may mean for the future of measurement.  The leaders are:

Barry Leggetter, FPRCA, FCIRR is Executive Director of AMEC  (barryleggetter@amecorg.com)

Pauline Draper-Watts, is Chairperson of the Institute for Public Relations, Commission for Public Relations Measurement and Evaluation  (pauline.s.draper@gmail.com)

David Rockland, PhD, is Partner/CEO, Ketchum Pleon Change and Managing Director, Global Research (David.Rockland@ketchum.com)

Here are their thoughts on the Summit:

Q1. For those not able to attend the Summit, how would you briefly describe what they missed?

BL: A milestone moment when delegates from 33 countries agreed to take program measurement more seriously, starting with the abandonment of AVE’s

DR: Missed a great opportunity to network with colleagues from 33 countries, hear some engaging speakers, and be part of a moment in time where the industry first adopted a set of measurement principles.

Q2. From your perspective, what are the two or three most significant outcomes of the Second European Summit on Measurement?

BL:

  • For AMEC to be successful in getting five global organisations on the same platform for the first time and talk from the same page about the need for the PR and media intelligence industry to act – not just talk – about improved methods of program measurement.
  • That the Summit achieved its own break-through status in receiving speaker support from senior level clients from global organisations such as FedEx Corporation, Yahoo, Royal Philips Electronics, Nissan, Telefónica, Banco Santander and others.

PD-W: The percentages (voting) in favor for each of the Barcelona Principles following the discussion

DR: To me a significant outcome was a gathering of the industry in a manner where ideas were shared, friendships and partnership extended, and we agreed as an industry to look ahead to how we can do what we do better and professionalize the practice of public relations.

Q3. How do you hope agencies, companies and organizations operationalize the seven principles?

BL: I introduced quality management processes when a Director of Porter Novelli in the UK in the 90’s – the first agency in the world to make this commitment. It became part of our agency’s way of working. I hope agencies, companies and organisations will similarly make the same commitment to the Barcelona Principles and introduce more stretching methods of program measurement on all programs.

PD-W: Integrating them into the culture and corporate language within the organization so that they are lived out in practice.

DR: My hope would be that first the principles are widely talked about and become SOP for what we do. Second, that the term AVE disappears along with the incredibly counter-productive debate around this subject that has distracted the industry from its own development. And third, that each organization adapts the Principles into their own words and practices; when we see them translated into the languages of the 33 countries represented at the Summit, we’ll know it worked.

Q4. Please complete this sentence: A year from now, we will know the Second European Summit on Measurement was successful …….

BL: …if when I judge my next PR Awards schemes PR consultancies and company in-house PR teams are putting more effort into the Program Measurement heading on the award entry as they are now doing to demonstrating creativity!

PD-W: … the Barcelona principles (modified to reflect the comments made at the Summit and submitted afterwards) are not only adopted but also put into practice throughout the industry.

DR: …if we don’t hear PR practitioners continue to complain we don’t have a “seat at the table” because we lack the metrics and measurement approaches other disciplines have.

Keeping it real…and transparent

  • I am a member of the Institute for Public Relations, Commission on Public Relations Measurement & Evaluation
  • My agency, Fleishman Hillard, is a member of AMEC
  • Ketchum is a sister Omnicom agency

What Is That Hit In The (insert major publication name here) Worth? Nothing, Unless it Creates Engagement.

7 May

A few months back someone posed a question in a Linked-In discussion group wondering how much the major hit in USA Today he had just got for a client was worth.  Obviously he is not the first PR practitioner to ask this question.  Before pondering the answer, there are several questions we should address first:

  • How many people in our target audience had an opportunity to see the placement?
  • How many actually saw it?
  • Of these, how many actually read the article?
  • Of those reading it, did it change their thinking in any way?
  • Did they forward it on to others?
  • Mention it in a phone conversation with a friend?
  • Visit a website?
  • Digg it.
  • Tweet it?
  • Blog about it?
  • Buy it?…

While one must have Exposure before Engagement, much like Awareness must precede Purchase Consideration, true value creation begins at the Engagement stage.  Using old school language, value occurs with Outcomes, not Outputs.  Seems simple enough yet the majority of PR professionals are still relying on output-oriented metrics like clip counts and ad value equivalents (AVEs) to judge success.  PR pros who are savvy about social media seem to be further evolved.  They understand that true value is not in the content (an output) per se, but in the level of engagement caused by the content.

Are you looking for value in all the right or wrong places?

AVEs (Advertising Value Equivalents) Revisited

24 Dec

Generally speaking, advertising delivers a lower ROI than public relations.  So why do we want to compare our results to those of an ad?  Because it is a path of least resistance to calculating ROI, flawed as it is.  Many people obviously believe a poor metric for ROI is better than none at all.

So, in the spirit of the 12 Days of Christmas, here are the 12 reasons why AVEs are a poor metric for public relations.  And one rationale for using them that makes at least a little sense.      

1. Advertisements and editorial articles are perceived differently by receivers/readers.  Editorial material benefits from the credibility of a third-party (the publication) by earning, not paying, its way into the magazine, newspaper or broadcast

2. AVEs equate an article with the appearance cost of an advertisement.  It does not speak at all to the results or impact that the article may have on a reader.  Advertisers do not judge the success of advertising on how much the insertions cost.  The true value of an ad or article is in what it does – the outcome or impact, not the cost of appearance.

3. AVEs do not address the value of several important aspects of public relations including strategic counsel, crisis communications, grassroots, viral campaigns or public affairs.  In other words, AVEs reduce PR to just the media dimension by only assigning a value in this area.  If only AVEs are used to assess PR value, the results will be much understated when considering the totality of value delivered by PR.

4. AVEs cannot measure the value of keeping a client with potentially negative news (e.g. layoff, scandal) out of the media, yet that may be the primary objective of the PR practitioner.  How much is it worth for a troubled company to notappear in the Wall Street Journal?  AVEs cannot address this.

5. Impression information for public relations is somewhat inconsistent.  Online impressions figures are not as reliable as print or broadcast, and are generally believed to be overstated  The fact that they are inflated skews AVE calculations to (pick one: somewhat, very, grossly) overstate the value of online media, often assigning unbelievable values to online articles compared to their print counterparts.  This hurts credibility and believability.

6. AVEs do not properly distinguish between hits/articles that appear in ‘high value’ columns or publications and articles in more general or generic publications.  The calculation is based on ad cost only.  The value of appearing in a Walt Mossberg column in the WSJ or on Oprah with your new book far exceeds the cost of an advertisement in the WSJ or on Oprah due to the implied or explicit endorsement with earned media.  Just look at what Oprah’s endorsement has done for Obama recently.

7. Advertising and PR actually work together synergistically, yet AVEs treat them essentially as equals or alternatives.  Ads that run in a climate of positive publicity actually receive lift from the PR.  Conversely, ads run in an environment of negative publicity will likely not be successful and/or may be perceived negatively by consumers/customers.

8. AVEs are generally calculated by mainly, or only, taking into account the physical size of the article, and then equating that to the cost/value of an advertisement of the same size.  Often, article valence is not even considered, so a predominantly negative article would add positively to the overall AVE calculation.  Others count the size of the entire article, even if only one paragraph directly addresses the company in question.

9. Some groups have devised their own ways to calculate AVEs.  PR articles are generally rated or scored as part of an algorithm used to calculate AVEs.  Factors considered might include brand prominence within the article, competitive mentions, overall article tonality and finally size/length of the article.  The problem here is there is no standard way to ‘score’ PR articles to implement an AVE system.

10. If you get a hit on the front page of a newspaper or a cover story in a magazine, there is no way to calculate an advertising equivalency since advertising space is never sold in these locations 

11. AVE results can be misleading.  AVEs may be trending up while important metrics like message communication, share of favorable positioning and share of positive press are falling.  Objects may appear larger than they really are.

12.  AVEs only apply to traditional media.  What is the AVE of a positive conversation about your company on a leading blog? 

If you are still searching for a rationale for using AVEs, there is one that has some merit IMHO.  That is the economic argument that advertising rates are established in a free market system.  Publishers can only charge what advertisers are willing to pay for a page in their publication.  Essentially the value of that page is established by this free market system.  Of course what appears on this page is not considered in the value determination.  I would suggest one refer to the value of a page determined in this way as the ‘Media Value’ rather than the Advertising Value.  By using the term Media Value, one eliminates the uncomfortable and unjustified comparison of an article generated by PR with an advertisement.  You are merely suggesting the page has a value and you use that value to determine how much the public relations content is therefore ‘worth’.  Media Value is still not a great way to assess the value generated by public relations for many of the reasons stated previously, but it seems to me to be less objectionable than AVEs.  

(Note: Much of this commentary about AVEs appeared in earlier posts and comments in this blog.  You can see them here)

As always, thanks for reading.  May the New Year bring you and yours happiness, health and prosperity.  -Don B

Quick update on AVEs

5 May

In my earlier post on Ad Value Equivalents (The Evils of AVEs), I somewhat tongue-in-cheek asked for help in coming up with a tenth reason why AVEs are a poor measure of public relations.  Well, I was re-reading a 2003 IPR article today written by Dr. Walter K. Lindenmann with Fraser Likely (Guidelines for Measuring the Effectiveness of PR Programs and Activities), and there was reason number ten staring me in the face. 

10. If you get a hit on the front page of a newspaper or a cover story in a magazine, there is no way to calculate an advertising equivalency since advertising space is never sold in these locations 

Love it, thanks Walt.  –Don B

Follow

Get every new post delivered to your Inbox.

Join 158 other followers