The Digitization of Research and Measurement

This post first appeared as an agency guest post on Jason Fall’s Social Media Explorer blog.  You can see it here.

The field of public relations has undergone two major revolutions in the past 15 years or so.  The advent of the Internet represents the first revolution.  This revolution primarily impacted the way content was created, distributed and consumed.  It also fundamentally changed the nature of communication – remember email became the first killer app of the Internet revolution.  The second revolution is social networks.  Again content creation was impacted, led by consumer generated content in multiple forms.  Perhaps more importantly, peer-to-peer communication between consumers, and two-way communication between consumers and brands/companies, have been enabled and are having a profound impact on the way companies are organized and behave.  The worlds of marketing and public relations have made an analog to digital conversion.  And with it, we are in the midst of the digitization research and measurement.

New Models, New Metrics

Communication models are a linear representation of how a communication process works and are important in providing a framework for evaluation and measurement.   The Outputs – Outtakes – Outcomes communication model often used in public relations today has two primary deficiencies in the era of digitization and social networks – clarity and relevance. 

• Clarity: The model is difficult for many to understand and apply.  Public relations practitioners regularly get Outputs confused with Outtakes or Outcomes.  Outtakes are not often used in the U.S. – they seem much more prevalent in Europe.  The overall taxonomy can be confusing and is defined in different ways by different practitioners or organizations.  Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.
• Relevance: The model was developed when communication was media-centric.  Digitization, consumer-generated content and social networks have shifted communication from a media-centric world to a content-centric world.  How receivers of communication engage and are influenced by content has fundamentally changed.

What is needed is a metrics taxonomy that is easier to explain, understand and apply.  Ideally one that is applicable for traditional and social media.  Here is the model we apply at Fleishman Hillard.

With the new model comes new metrics primarily driven by social media/networks.  Exposure  includes traditional metrics like Impressions and Message Delivery, and digital metrics like Search Rank, Twitter Reach and Average Daily Visitors.  Engagement includes traditional metrics like Readership, but adds new metrics like Subscriptions, Repeat Visitors and Follower Mention %.  Influence in the model refers to influence of the target audience, not who has influence in social networks.  Influence metrics range from increases in Brand Consideration to changes in attitudes and opinions to changes in online click behavior.  Action metrics can range from event attendance to voting for/against legislation to buying a product.

New Data, New Places

Public relations research and measurement has historically been driven by content analysis.  As content increasingly became available in digital form, the techniques of research and measurement didn’t change so much as the way content was aggregated and delivered for analysis.  Then web-based platforms became available from a variety of vendors to digitize and automate content analysis while the metrics being measured – article counts, impressions, message uptake and sentiment for example – basically remained constant with previous, more manual, methods.  Today, the digitization of research and measurement has broadened from this predominately singular focus to include data and interactions from three distinct regions or zones of research and measurement as shown in the figure below.

As company websites, e-Commerce sites and other forms of ‘owned’ media proliferated, web analytics software provided an explosion of data and new metrics like unique visitors, page views, click through rates, duration, referring sites and conversions become widely used and reported.  We became over-served with data and underserved with insight.

The exponential rise in popularity of social networks in the last five years raised the bar again and presented new challenges in digital research and measurement.  Now we were faced with measuring conversations and not just clicks.  Measuring engagement became more important than measuring eyeballs.  The frontier in social media measurement is evolving toward measuring both the conversations and behavior patterns occurring within social networks, and understanding and connecting the underlying influences and motivations for the online behavior.

The third area of interest is in all the real-world, offline interactions and transactions. Scan and other digital sales data is important to understanding, tracking and connecting online and offline behavior and actions.   Connecting mobile transactions, online and offline behavior and WOM is a significant challenge.

Although we have attempted to define three distinct ‘zones’ of digital research and measurement necessary to address the full spectrum of social media and marketing impact, a robust measurement strategy should take a holistic, integrated approach using methodologies, tools, data and metrics from all three zones.  The goal is to be able to track the behavior, interactions and transactions of individuals across all three zones, across multiple platforms and physical locations, understanding how online behavior impacts offline behavior and vice-versa.

New Scope, New Integration

Today at Fleishman Hillard, we recognize the very definition of public relations is rapidly evolving to encompass a much broader and more integrated view of communications and how we connect, engage and build relationships with consumers and other stakeholders on behalf of our clients.  Digitization in all its forms has driven and accelerated this important change.  While public relations has traditionally been oriented toward ‘earned media’ – gaining placements of client stories in print and broadcast media based on the strength of the story and quality of the pitch – today’s content-driven world demands much more.  The scope now must include all the consumer touch points available in our increasingly digital world.  We capture this new scope and integration in a model we refer to as PESO – Paid/Earned/Shared/Owned.  Our PESO model predates the similar Forrester model (Paid/Earned/Owned) and is different in an important way.  We created two categories, Earned and Shared, where the other model has one – Earned.  We believe this better comprehends strategies like blogger outreach and other proactive efforts undertaken by practitioners as ’Earned’,  distinct from efforts that may be passive or reactive.  Here is how we define the elements of our model:

Paid – refers to all forms of paid content that exists on third-party channels or venues.  This includes banner or display advertisements, pay-per-click programs, sponsorships and advertorials.

Earned – includes traditional media outreach as well as blogger relations/outreach where we attempt to influence and encourage third-party content providers to write about our clients and their products and services.

Shared – refers to social networks and technologies controlled by consumers along with online and offline WOM

Owned – includes all websites and web properties controlled by a company or brand including company or product websites, micro-sites, blogs, Facebook pages and Twitter channels.

The enhanced scope and integration represented by the PESO model drives a corresponding broadening and need for integration in digital research and measurement.  One can easily find themselves attempting to measure a highly integrated program that includes the awareness created with paid media, the relevance and information delivered via owned, the credibility delivered by earned media and measuring the conversations and interactions occurring in shared media.  Just from a metrics perspective, the PESO model requires a significant broadening in thinking as shown in the matrix below.

Digitization has changed what we need to research and measure, where we find data and how we perform analysis.  The future will bring more data, better tools and improved methodologies.  Sifting insights from the mounds of data will remain a major challenge.  The intersection of marketing, privacy concerns and research must be navigated.  The constant in all the change brought by digitization is who – human analysts and research.  Discovery and insight, like it was 15 years ago, remains fundamentally a human process.  It remains the analog constant in a world of digitization.

Measure the Puzzle Not the Pieces

A while back, I remember someone posting a question to a Linked-In discussion group along the lines of, ‘I just got my client a hit in USA Today.  How much is that worth?’.   More recently, ADWEEK ran an article entitled, Value of a Fan on Facebook: $3.60, citing an attempt by Vitrue to essentially assign a media value to a Facebook Fan.  (Sidebar: Is a Liker worth as much as a Fan?).  Setting aside an argument of the value attribution methodology used by Vitrue (I’m not a fan, or a liker), the fundamental issue I have with each example is the same, they are trying to measure the pieces and not the puzzle.

A media hit, a tweet, gaining a Fan/Liker, or obtaining a Follower are all pieces to a larger puzzle called a social media/business campaign, initiative, effort or program.   For simplicity, let’s refer to them as programs henceforth.  Programs have, or should have, objectives.  Done correctly, these objectives are measurable.  Good measurement practice suggests you assess performance against stated objectives.  Sure, it is also important to assess performance of program strategies and tactics – primarily as a diagnostic – but ultimately we must measure performance against objectives.  This is a base condition for accountability.

Gaining media coverage, sending tweets or getting others to tweet about you, creating Likers or gaining Followers should be thought of as strategies or perhaps tactics.  Objectives are what you want to happen as a result of the combination of strategies and tactics.  Programs are not made of single media hits, tweets, Likers or Followers.  They are longitudinal, holistic and integrated.  Successful programs might generate hundreds of media hits, scores of blog posts, and thousands of Likers or Followers.  Orchestrated correctly, all these strategies and tactics should help us achieve our overall program objectives.  The reality of the situation is any one discrete result of a campaign – a hit, Liker or Follower for example – usually has a very small overall impact.  The impact most likely would not be measurable, and if it was, it would not likely be meaningful.  They are just pieces of the overall program puzzle.

Let’s conclude with a simplistic Facebook program example.  Your tactic is to gain more Likers that meet a certain demographic profile.  Your strategies are to create an engaged brand community in Facebook, and to encourage online and offline WOM about the brand.  Your objective is to increase brand preference from 17% to 21% in the next 12 months.  Measure this objective, and if you want to do value attribution and calculate ROI, figure out how much each 1% increase in brand preference is worth in incremental sales.  That’s a puzzle worth solving.

Photo From liza31337

Relief from your Social Media ROI Angst

Return on Investment (ROI) is one of the most discussed and agonized-over topics in social business today.  However, much of the discussion around social media ROI has been simply confused, confusing or misguided.  There have been posts on large, well-known blogs (riff on ways to prepare potatoes) that are incredibly naive in their discussion of ROI.  In some circles there is endless philosophic debate bounded on one side by the Puritans who believe ROI is an old/incorrect way to think social business and on the other side by the Analyticans who seem to believe you can always determine ROI with a web analytics package.

The net-net of all this is a lot of frustration, even angst over how to think about ROI and social business.  We’re here to help.  Take two deep breaths and read these four points.  You’ll feel better soon.

Point Number One:  As a practical matter, the majority of social business efforts will not result in true ROI (in the short term).

In fact, I would guess far less than half will.  Maybe less than 10%.  But that doesn’t mean the social business effort was not successful, or did not create significant value for the brand or organization.  It simply means the primary objectives of most social business efforts are centered on concepts like community-building, engagement, listening, and participating in conversations.  It is difficult and expensive to attribute financial value to these areas.  To use the old saying – the ROI on these sorts of ROI efforts is not good.  Traditional public relations, branding and reputation programs suffer from some of the same challenges.  So when a study like the one published by e-Marketer* suggests ‘only’ 16% of social business programs are measuring ROI, while many are surprised it isn’t higher, it actually sounds a little too high to me.   I wonder how respondents were thinking about and defining ROI.

Point Number Two: Loose use of the term ROI is a major cause of angst.

ROI is not synonymous with results, KPIs or value.  ROI is not the only or perhaps even the most relevant way to define success in social business.  ROI is a financial metric.  ROI can only be measured in terms of revenue generated, cost savings or costs avoided.  It is transactional in nature.   There is ample evidence on twitter, blog posts and in blog comments that many people say ROI when they really mean results or value.

Many of the well-intentioned but misguided attempts to rename or reinvent what ROI means in social media – return on influence and return on engagement probably getting the most play – seem to be the result of an inability to distinguish value creation from ROI.

Point Number Three: Understand the difference between value and ROI.

Social media efforts may create financial impact (ROI) and/or non-financial impact. Engagement and Influence are examples of non-financial impact.  Other non-financial impacts like increased brand awareness or purchase consideration may eventually result in ROI at a point in the future when a financial event occurs.  Try to be explicit as to whether the social business program is designed to generate non-financial impact or true financial ROI, and make sure the people writing the checks understand the difference.  Show how the effort is linked to one or more business processes and how it will deliver value by helping to drive the desired business outcomes.

We know social networks have the ability to create value through customer engagement and community building.  However, ROI can only be measured by their ultimate impact on downstream metrics like sales, employee retention and customer loyalty/repeat purchase.  Many social business efforts are in an investment phase.  The value is largely intangible.  Some may eventually become transactional and result in true financial ROI.

Point Number Four:  ROI in social business has a time dimension.

Value may be created in the short-term and longer-term.  Social business campaigns utilizing channel-specific URLs and ecommerce landing pages are an example of easily measured short-term ROI (setting aside last-click attribution issues).  Longer-term value is much more difficult to quantify.  There are some similarities between social media and brand in this regard.  Success in each is a process and not an event.  You may have ongoing activities that sustain the brand/social business program and brand building events or campaigns that provide short-term spikes in awareness and engagement.  Managing and measuring your social business effort properly requires thinking about the value you are creating in the short and longer-term.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business process transaction that results in ROI.  Calculate ROI whenever you can, but also try to articulate the value your programs will be creating, and how this value aligns with, and contributes toward, achieving one or more desired business outcomes.

If you are not feeling better already, please leave a comment and let us know why.

*Mzinga and Babson Executive Education, Social Software in Business, September 8, 2009

Social Media ROI Part 1: Framework

Social Media ROI Framework

Here is a simple, five-step framework for developing a social media ROI measurement program.  Remember that not all social media initiatives will result in short-term ROI generation. It is also important to comprehend the results of programs that result in non-financial value or impact. (For a quick refresher on the difference between value and ROI read this).  Holistic measurement programs should be designed to track and measure non-financial impact as well as ROI.

1. Set measurable objectives aligned with business outcomes

Failure to begin with measurable objectives is probably the most common impediment to proper social media measurement.  A couple of award seasons ago I was a judge for a major PR campaign competition and was appalled by the low percentage of programs that actually contained measurable objectives – about 20% or so.  Your objectives should be aligned with one or more desired business outcomes.  Think through all the ways in which the social business effort will contribute toward driving the overall business forward.  Make sure the alignment is obvious and understood by all involved in program approval.

2. Link to and understand the requisite business process

In order to demonstrate ROI in social media it is necessary to link the results seen in social media with the relevant business processes they are addressing.  Social programs to date generally relate to one or more of the following business processes:

Business Process	Description
CRM	Crowd-sourced help, info, recommendations, customer relationships
Research	Competitive intelligence, insights, voice-of-customer, trends, feedback, reputation assessment, influencers
Marketing & Sales	Ideation, product promotion, hyper-local marketing, lead generation and closure, fund raising, testing, brand attributes
Communication/PR/IR	Stakeholder communication (internal and external)
Innovation/Product Development	Crowd-sourced ideas, problem/opportunity identification, collaboration

For example, in a B2B company, you might try to link social media efforts with the lead generation and closure process.

For a program aimed at employee engagement, you might link social media efforts to the employee recruitment and retention business process.  For an e-Commerce company you might be able to directly link to the sales process through unique URLs or click-tracking technologies.

Understanding which business processes are impacted by social networks, and how, is fundamental to understanding ROI.

3. Select communications model, research approach and key metrics

In addition to understanding the business process impacted by social programs, it is important to have a communications model to assess non-financial impact.  The accepted Outputs – Outtakes – Outcomes communication model is difficult and confusing for many to understand and apply.  Here is an alternative communication model that is somewhat more intuitive and in tune with social media measurement.

• Exposure – To what degree have we created exposure to content and message?
• Engagement – Who is interacting/engaging with our content?  How and where?
• Influence – The degree to which exposure and engagement have influenced perceptions and attitudes of the target audience.
• Action – As a result of the social media effort, what actions if any has the target taken?”

The metrics listed are a starting point.  You will note these metrics come from all three “zones of measurement” – web analytics, digital content analysis and primary audience research.

Possible ROI research approaches include:

• Correlation modeling and econometrics
• Staff cost reduction and/or cost elimination tracking
• Direct linkage via unique URLs and click-tracking
• Exposed/Not-exposed primary audience research
• Integrated cross-platform research – web analytics, content analysis, click tracking, primary research.

4. Gather and analyze data

Gather longitudinal data for the requisite business process and communications model metrics.  When attempting to show statistical correlations, the amount of data you include is important because it impacts the confidence level of the results.  Also, what metrics you attempt to correlate and how is crucial.  For example, we might try to correlate social media brand engagement and audience influence with metrics like likelihood to recommend to a friend, likelihood to seriously consider the product or likelihood to purchase the product in the next X months.  Select the metrics that are most applicable to the business process you are attempting to drive.

5. Calculate ROI and report results

We started with measurable objectives, aligned them with requisite business processes, determined our research model and approach and have gathered the data.  Now we can calculate ROI and describe the value of the social business initiative.  Results for key program metrics should be captured on a dashboard that may be shared with all program stakeholders are a regular basis.

(Note: Part two of this post will cover specific research approaches to determining social media ROI in more detail)

Public Relations Measurement 2010: Five Things to Forget & Five Things to Learn

(This post is a re-purposing of a speech I gave to the FPRA/PRSA-Orlando on July 23, 2009.  You can download the slides here.)

Public relations measurement is at a crossroads.  Old techniques are no longer sufficient.  Old metrics are no longer applicable.  Old thinking must be replaced by new.  The need for accountability, and to prove the value of PR and social media programs, has never been greater.

As we look to the next year, here are five things to forget and five things to learn about public relations measurement in 2010.

Things to Forget in 2010

1. Media Relations Focus

A focus on media relations fails to capture several important aspects of PR – brand, reputation, crisis, employee communication and DTC to name a few.  Also, the importance of traditional media is declining.  Numerous studies have shown people don’t trust what they read in the media, they trust each other.  I believe it was Hauser and Katz who coined the term ‘you are what you measure’ in 1998.  If measurement is focused on media relations that is how the public relations function will be judged.

__________________________________________________________

2. Outputs

The need to put PR results in a business context has never been greater.  We need to be able to address the question – what are we doing to help drive the business?  If you are focused on output metrics like impressions or message delivery, you will always have a hard time explaining business impact.  Instead, we need to focus on outcomes and answer the question – what happened as a result of our program or coverage?  Understanding outputs has primary benefit as a diagnostic tool rather than a ‘scorecard’.

__________________________________________________________

3. Impressions (and Multipliers!)

The most common PR metric today is Impressions.  While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek.  Impressions also (greatly) overstate actual relevant audience.  Generally only a fraction of any particular magazine or newspaper’s circulation meets your target audience demographics.  And impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content.  To compound the problems, many PR practitioners use a multiplier on impression numbers to account for pass-along readership or a mythical credibility advantage PR has over other communication tools.  The simple fact is there is no factual basis (e.g. research proof) that multipliers should be used in any case.

__________________________________________________________

4. Ad Equivalency (AVEs)
There are many reasons why using ad equivalency as a proxy for PR value is not advisable.  Here are five good reasons they should be avoided:

• AVE calculations vary and there are no standards.  Tonality, article length, competitive mentions and other factors are handled differently.
• AVE results can be misleading.  AVEs may be trending up while metrics like message communication, share of favorable positioning and share of positive press are falling.
• AVEs reduce PR to just the media dimension by only assigning a value in this area.
• AVEs only apply to traditional media.  What is the AVE of a positive conversation about your company on a leading blog?
• How much is it worth for a troubled company to not appear in the Wall Street Journal?  AVEs cannot address this.

__________________________________________________________

5. Return on (Engagement/Influence/etc.)
Not a day goes by on Twitter without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks.  Wrong and wrong.  Most of these folks either don’t understand ROI or don’t know how to obtain the data necessary to calculate it.   There is also a lot of confusion between creating value and ROI.  Generating awareness creates value, for example, but may not immediately result in demonstrable ROI.

__________________________________________________________

Things to Learn in 2010

1. Total Value of PR

The majority of current PR measurement efforts focus on marketing/sales and output metrics.  The Total Value Cube is a way to visualize and think about all the potential value your PR and social media efforts deliver.  Beyond marketing to include brand and reputation, beyond outputs to include engagement, influence and action, and beyond revenue generation to include cost savings and cost avoidance.

__________________________________________________________

2. A New Model for Measurement
Many public relations practitioners regularly get their Outputs confused with their Outtakes or Outcomes.  Outtakes is not often used in the U.S. – it seems much more prevalent in Europe.  The overall terminology is confusing and is defined in different ways by different practitioners.  Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.

What we need is a metrics taxonomy that is easier to understand and explain.  I like this one.

Exposure – to what degree have we created exposure to content and message?

Engagement – who, how and where are people interacting/engaging with our content?

Influence – the degree to which exposure and engagement have influenced perceptions and attitudes

Action – as a result of the PR/social media effort, what actions if any has the target taken?”

__________________________________________________________

3. Three Zones of Measurement

From the left, companies or brands control, own or manage websites  – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with.  This zone is measured primarily by web analytics.  In the middle are the actual social networks and conversations between individuals.   In this zone we are interested in data sets that cannot be gathered solely using web analytics packages.  How often is the brand being mentioned in conversation?  What is the sentiment of the comments?  How often is the brand being recommended and by whom?  Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone.  The third zone represents all the real-world, offline transactions that may be of interest.  Did someone visit the store or attend or event?  Did they buy a product?  Did they recommend the brand or product to a friend over coffee?  Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.

Your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones.  The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.

__________________________________________________________

4. New Metrics

_________________________________________________________

5. The Difference Between Impact/Value and ROI
ROI is a form of value/impact, but not all value takes form of ROI.  ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.  ROI lives on the income statement in business terms.

Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product.  Value creation occurs over time, not at a point in time.  Value creation is process-oriented.  Value lives on the balance sheet.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

Inflationary Twitter Audience Numbers Hurt Social Media Credibility

In yesterday’s New York Times, you may have read the article, Spinning the Web: P.R. in Silicon Valley, an interesting although not overly insightful piece.  From a social media measurement perspective, two items caught my eye.  The first, referring to Brian Solis, Principal of FutureWorks, about how he calculates social media audience figures:

“Instead of calculating the impressions an article gets by estimating a publication’s circulation and pass-along rate, Mr. Solis counts the number of people who tweeted about a company and their combined following, the number of retweets or clicks on links, as well as traffic from Facebook and other social networks.”

Toward the end of the article, we learn:

“By 6:30 p.m. on the day Wordnik went live, Brew’s staff calculated that 1.43 million people had seen tweets about it.”

Setting aside for a moment that the article and these sorts of audience metrics take a broadcast-oriented view of Twitter (Mr. Solis discusses the shortcomings of the NYT viewpoint here), the emerging view of audience measures for Twitter is to calculate the Followers of each person tweeting about the subject of interest, and then adding Follower numbers for each person retweeting the subject and so on.  The issue here, much as it is in traditional public relations, is that the audience figure that results from these sorts of calculations grossly overstates, by one or two orders of magnitude or more, the actual “audience” for these tweets.  It is a hypothetical number that assumes everyone that possibly could see a tweet has in fact seen it, and everyone who sees it is relevant to you/your brand.  This is fantasy of course.

On the issue of relevant audience, here is a quick example.  At the time I pulled these figures, the audited circulation of the New York Times was 4,974,000.  Most PR practitioners getting a ‘hit’ in the NYT would claim this as their audience.  However:

• If you were only trying to reach a C-Suite audience with your message, the actual audience reached would be 598,000 or 12% of the total circulation
• If you were trying to reach Women, your audience would be 1,937,000 or 39% of the total
• If you were trying to reach 25 – 54 year old Men, your potential audience would have been 2,930,000, or 59% of the total number.

There is a large difference between how many people theoretically can see a tweet, versus how many actually saw it/read it, versus how many of those seeing the tweet find it relevant to them, versus how many engaged with it by hitting a link or retweeting.  Part of my issue with this is the language we use to report the figures.  For the Brew staff to use these numbers to estimate 1.43 million people “had seen tweets about it” is wrong.  If they had said 1.43 million people had an opportunity to see the tweet, it would have been more realistic, although still greatly overstating actual relevant audience.

This problem of audience inflation has already been institutionalized in public relations.  The use of Impressions as an output metric does not mean a true impression in the branding sense, but rather an opportunity to see the content.  To make matters worse, many PR practitioners believe Impressions should be factored by either dubious pass-along readership figures and/or use of a multiplier to account for the mythical credibility advantage PR enjoys over impressions generated from advertising.  The simple fact is there is no research-supported, fact-based argument for using any adder or multiplier in public relations when calculating potential audience (here’s an IPR white paper on this subject I co-authored).

For Twitter and other social networks we lack demographics and data about tweet readership averages (i.e. what is the probability that any one tweet is actually read) that would allow for more precise audience estimates.  In the absence of data, believable assumptions should be used:

• Out of all the opportunities to see, how many actually read the tweet?  10%?
• Of those reading the tweet, how many find it relevant to them (or from the other perspective, how many of the readers are in your intended target audience)?  Maybe 10% again?

You can see how our audience estimate has already been reduced by a factor of 100.  This may well still be overstating the actual, relevant audience.  The issue here is that unrealistic and overstated audience figures have the potential to hurt credibility and call into question other data and metrics that may be more grounded in fact.  Actually the more meaningful metrics pertain to engagement or outcomes rather than exposure/outputs.  It is more meaningful that 40,000 visited the Wordnik website as a result of the campaign discussed in the NYT article than the overstated 1.43 million who were estimated to have seen the tweets.  40,000 is real.  1.43 million is fantasy.

What Is That Hit In The (insert major publication name here) Worth? Nothing, Unless it Creates Engagement.

A few months back someone posed a question in a Linked-In discussion group wondering how much the major hit in USA Today he had just got for a client was worth.  Obviously he is not the first PR practitioner to ask this question.  Before pondering the answer, there are several questions we should address first:

• How many people in our target audience had an opportunity to see the placement?
• How many actually saw it?
• Of these, how many actually read the article?
• Of those reading it, did it change their thinking in any way?
• Did they forward it on to others?
• Mention it in a phone conversation with a friend?
• Visit a website?
• Digg it.
• Tweet it?
• Blog about it?
• Buy it?…

While one must have Exposure before Engagement, much like Awareness must precede Purchase Consideration, true value creation begins at the Engagement stage.  Using old school language, value occurs with Outcomes, not Outputs.  Seems simple enough yet the majority of PR professionals are still relying on output-oriented metrics like clip counts and ad value equivalents (AVEs) to judge success.  PR pros who are savvy about social media seem to be further evolved.  They understand that true value is not in the content (an output) per se, but in the level of engagement caused by the content.

Are you looking for value in all the right or wrong places?

Capturing the Total Value of Public Relations

Since public relations is a broad profession and may cover a wide variety of disciplines – media relations, online engagement, crisis communications, public affairs, executive counseling, brand building, events, reputation management, employee communications and financial communications to name a few – it is difficult to conceptualize the totality of value public relations and communication delivers to the organization.  For the most part, public relations measurement has focused on attempts to measure media relations value and is not really addressing the other areas very well.  When you are attempting to quantify the full value and ROI of public relations, taking the broad view paints a much richer picture.

The PR Value Cube is a tops-down conceptual framework for capturing all the ways PR is contributing value to the organization.  PR contributes value in one of three major, interrelated areas (Y-axis):

Marketing – Sales and other marketing oriented programs and metrics (e.g. lead generation) fit within this category.   The vast majority of PR measurement efforts today fall within the Marketing category.

Brand – PR contributes to building brands.  Value contribution in this area is usually more anecdotal than measured.  Experiential PR and many social media campaigns are contributing more to brand than sales or any other area.

Reputation – One of the primary overarching purposes of PR is reputation enhancement and protection, yet our contribution here again is usually measured more by ‘gut metrics’ than analytics.

Within each major area we can examine value created through Engagement, Influence and Action (X-axis).

Engagement – to what degree has exposure to PR materials, activities and events created Engagement with the intended target audience?  Are they interacting with our content, creating links, forwarding to friends, talking about the brand, etc.

Influence – the degree to which Engagement has influenced perceptions and attitudes.  Likelihood to recommend the brand to a friend and brand consideration changes are two possible examples of Influence.

Action – as a result of the public relations effort, what actions if any has the target taken?  Did they visit the web site, tell a friend, buy the product, vote for our candidate, etc.

The value itself can take one of three forms (Z-axis):

• Revenue generation
• Cost Savings (e.g. employee recruiting costs decline due to strong company reputation)
• Cost Avoidance (e.g. avoiding recruiting costs because employee retention/loyalty has improved)

There is one more important consideration when thinking about the total value delivered by public relations and social media.  That is time.  PR creates value on a transactional, short-term basis (e.g. the value of 10,000 potential customers reading your article in today’s Wall Street Journal) and on a process-oriented, longer-term basis.  Brand and Reputation are both examples of longer-term value.  Both are process-oriented, and build and lose value over time, often measured in years.  The other time dimension value created by PR is what I have referred to previously as the residual value of PR.  That is the value of the created searchable and archived content created by the PR function.  The residual value may take the form of influencing organic search positioning.

That’s a lot of value for one profession!  In 2009, let’s hope CEOs, CMOs and other decision makers increasingly recognize the great value and superior ROI delivered by public relations.

A New Model for Social (and traditional) Media Measurement

In November 2007 I suggested the current Outputs, Outtakes, Outcomes model and taxonomy for public relations measurement was confusing and therefore often misunderstood and misapplied (Let’s put Outputs, Outtakes and Outcomes in the Outhouse).  At that time I suggested a simpler, more descriptive approach was in order and offered the following:

“What we need is a metrics taxonomy that is easier to understand and explain.  Perhaps simple and descriptive enough that we could skip the need for explanation altogether.   I propose the following three terms:
* Exposure – to what degree have we created exposure to materials and message?
* Influence – the degree to which exposure has influenced perceptions and attitudes
* Action – as a result of the public relations effort, what actions if any has the target taken?”

Since November, I have given a lot of thought to the E-I-A construct and how to improve upon it.   Some of the feedback to the model was the gap between Exposure and Influence was too great, and perhaps there should be an interim step called Understanding or Relevance.  There is also the social media dynamic to consider since the measurement model should be flexible enough to work for both traditional and social media.

What seems to fit best between Exposure and Influence, and adds richness to social media measurement, is the concept of Engagement.  Not only is it one of the hotter topics in social media, it is consistent with the desire to have more descriptive and easily understood metrics.   With Engagement we now have an category that nicely contains such emerging key metrics as view-thrus, duration spent with content, repeat commenters and comments/posts ratio.  It also works well for old school metrics like recall and retention.  Engagement is what helps set the stage for Influence to occur.  Engagement is necessary for communities to form.  Engagement is fundamental to brand.

Here’s a graphic that shows the new model and sample metrics that might be used at each stage:

Would love your feedback on this new Exposure — Engagement — Influence — Action model.

There are still a few challenges in adoption of the model, not the least of which is that there is no consistent definition of Engagement.  Current definitions range from the simple comments to post ratio used by BusinessWeek in their Reader Engagement Index, to the 8-term formula for Engagement offered by Eric T Peterson.  The next year should bring more clarity and consistency to our understanding and use of Engagement.  At least there is modest agreement on the specific metrics contained within the category of Engagement.

Thanks for reading, Don B 

Two Keys to Low Cost Measurement

Cost is one of the key inhibitors of public relations measurement becoming more prevalent.  It probably is THE key inhibitor, with ignorance/lack of education a close second.  Cracking the code on lowering total costs of measurement would go a long way toward making measurement the rule rather than the exception.  In order to understand how to lower costs, you first have to understand what the largest cost drivers are in most measurement programs today.  The primary cost drivers are content acquisition/aggregation and human analysis of articles.

Up to 40% of the total cost of a media content analysis program can simply be acquiring and aggregating the content to be measured.  Common content services like Factiva, Lexis Nexis, Bacons, eWatch and VMS are not cheap.  In order to cast a wide net, many PR professional feel they need multiple services to cover (almost) every possible outlet where coverage may occur.  Content costs can quickly get out of hand.

The other major cost driver is the need to have real humans analyze coverage.  And no, automating article analysis is not a truly viable option right now.  IMHO, the accuracy of such systems is not high enough to justify the potential cost savings.  Even with many content analysis operations being off-shored to low cost countries like India, the cost for analysis on a per article basis ranges from $1.50 – $3.00.  If you are garnering a lot of coverage, these costs can add up in a hurray.

The good news is that one can address both of these cost components by simply measuring a subset of your total coverage rather than every single article.  There are two ways to accomplish this – by taking a nth. sample of your total coverage, or, the approach I prefer, determining the relatively short list of publications/outlets that have the most influence on your targets and only measuring coverage within this smaller population.  By relatively short, think 100 total publications or less.  With a little work, you can probably get your list down to 50 outlets that really make a difference.  I worked with a F500 company that targeted 64 publications they felt really helped move their business.  That is 64 globally.

By confining your measurement program to the most important and influential outlets you hold down both content and analysis costs.  So why don’t more people pursue this easy fix?  I believe it goes back to the industry attitude that views measurement as a score keeping mechanism more so than a diagnostic tool. (read more here).  If our industry remains prisoners of the ‘tonnage’ model of coverage, then measurement cost reduction is very difficult.

To close on a positive note, the quality of the free content sources and tools is getting better and better.  Now that is supports archiving, Google News is a viable source to acquire content.  Google Analytics and BlogPulse from Nielsen Buzzmetrics provide some interesting blog and website metrics at an aggressive price point – free.

Thanks for reading.  -Don B