Archive | Return on Investment) ROI RSS feed for this section

Don’t Let the Tool Tail Wag the Measurement Dog

19 Jul

Social media listening and measurement tools are sexy.  Well, at least to those of us in research and measurement – it’s all relative right?  In the last three years or so there has been an explosion of social media tool vendors and platform choices.  Tools are sexy and important, but in the grand scheme of things are being overemphasized to some degree.  We are letting tools decide what we can measure without giving sufficient thought to what we should measure.  We are letting the tool tail wag the measurement dog.

There are several steps and decisions that should be addressed prior to selecting a tool or suite of tools.  Consider this diagram as a starting point to help you think through these interim considerations and decisions:

OBJECTIVES

Proper social media objectives should be measurable (indicate change in metric of interest and timeframe) and aligned with desired organizational outcomes.  Understanding the social media objectives will suggest broad parameters the measurement program, and ultimately the tool decision, must operate within.  For example, geographic coverage requirements, type of content to be considered and on-platform engagement capability may all be strongly suggested based on a review of social media objectives.

PROCESS

In addition to comprehending organizational or business outcomes, it is essential to understand the business process the social media program will address or drive.  If the program is marketing oriented, the sales funnel process (Awareness/Consideration/Preference/Sales/Loyalty) may be most appropriate.  For a brand-building campaign, the brand pyramid (Presence/Relevance/Performance/Advantage/Bonding) is what you want to measure your program impact against.  Other business processes that are commonly addressed by social media programs include customer service and support, CRM, corporate reputation and lead generation.

METRICS

Understanding the requisite business process the social media program is driving is crucial because each business process drives specific metrics.  For example, the sales funnel drives a specific metrics set:  percentage of unaided or aided awareness; percentage of the target audience who would consider the product/company; percentage who prefer the product/company; incremental sales revenues; percentage who would purchase the product again number or the number/amount of repeat purchases.  For B2B companies, the lead generation process would drive a different set of metrics: number of incoming leads; percentage/number of qualified leads; lead conversion rate; sales revenues generated.  In addition to the business process metric sets, there are other metrics areas like Exposure and Engagement we will want to address.  Reach/opportunities to see, share of positive discussion, comments/post ratio, number of @ mentions and RTs per 1000 followers are examples of ‘standard’ metrics that might be applicable for many social media programs.

Understanding how the social media program drives a specific business process is also important to our ability to describe the impact or, in some cases, return on investment the program has created.

DATA SETS

Each metric has data requirements, usually two pieces of data per metric – a numerator and a denominator.  Examine the set of metrics you have defined for your social media program.  Catalog all the specific pieces of data you need to compute the various metrics.  For example, the data needed to compute the basic sales funnel metrics and some ‘standard’ metrics might include:

  • Number of individuals in the target audience
  • Number of survey respondents
  • Number of respondents ‘aware’ of the product/company
  • Number of respondents who would consider/seriously consider purchasing the product/doing business with the company
  • Number of respondents purchasing the product
  • Amount of sales revenue directly attributable to the program
  • Number of purchasers who purchased again
  • Total branded mentions
  • Volume of positive and negative mentions
  • Number of posts
  • Number of comments
  • Number of RTs and @ mentions
  • Number of followers

TOOLS

Armed with an understanding of all the data needed to calculate the metrics required to measure the social media program, you will be able to assess which tools or classes of tools best deliver the data you need.  Pick the best three to five tools for further evaluation.  You most likely will find no one tool can deliver the complete data set you need.  It is common to need two or more tools, e.g. web analytics package and social content analysis platform, in order to fully meet data requirements.  Budgetary constraints may also limit your ability to capture the entire data set required.

By addressing the interim steps leading up to tool selection, you will be able to make a more informed tool decision.  You also will have a much better chance of measuring what you should measure rather than settling for what you can measure.  No tool before its time.  Let the big dogs run.

The Digitization of Research and Measurement

12 May

This post first appeared as an agency guest post on Jason Fall’s Social Media Explorer blog.  You can see it here.


The field of public relations has undergone two major revolutions in the past 15 years or so.  The advent of the Internet represents the first revolution.  This revolution primarily impacted the way content was created, distributed and consumed.  It also fundamentally changed the nature of communication – remember email became the first killer app of the Internet revolution.  The second revolution is social networks.  Again content creation was impacted, led by consumer generated content in multiple forms.  Perhaps more importantly, peer-to-peer communication between consumers, and two-way communication between consumers and brands/companies, have been enabled and are having a profound impact on the way companies are organized and behave.  The worlds of marketing and public relations have made an analog to digital conversion.  And with it, we are in the midst of the digitization research and measurement.

New Models, New Metrics

Communication models are a linear representation of how a communication process works and are important in providing a framework for evaluation and measurement.   The Outputs – Outtakes – Outcomes communication model often used in public relations today has two primary deficiencies in the era of digitization and social networks – clarity and relevance. 

  • Clarity: The model is difficult for many to understand and apply.  Public relations practitioners regularly get Outputs confused with Outtakes or Outcomes.  Outtakes are not often used in the U.S. – they seem much more prevalent in Europe.  The overall taxonomy can be confusing and is defined in different ways by different practitioners or organizations.  Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.
  • Relevance: The model was developed when communication was media-centric.  Digitization, consumer-generated content and social networks have shifted communication from a media-centric world to a content-centric world.  How receivers of communication engage and are influenced by content has fundamentally changed.

What is needed is a metrics taxonomy that is easier to explain, understand and apply.  Ideally one that is applicable for traditional and social media.  Here is the model we apply at Fleishman Hillard.

With the new model comes new metrics primarily driven by social media/networks.  Exposure  includes traditional metrics like Impressions and Message Delivery, and digital metrics like Search Rank, Twitter Reach and Average Daily Visitors.  Engagement includes traditional metrics like Readership, but adds new metrics like Subscriptions, Repeat Visitors and Follower Mention %.  Influence in the model refers to influence of the target audience, not who has influence in social networks.  Influence metrics range from increases in Brand Consideration to changes in attitudes and opinions to changes in online click behavior.  Action metrics can range from event attendance to voting for/against legislation to buying a product.

New Data, New Places

Public relations research and measurement has historically been driven by content analysis.  As content increasingly became available in digital form, the techniques of research and measurement didn’t change so much as the way content was aggregated and delivered for analysis.  Then web-based platforms became available from a variety of vendors to digitize and automate content analysis while the metrics being measured – article counts, impressions, message uptake and sentiment for example – basically remained constant with previous, more manual, methods.  Today, the digitization of research and measurement has broadened from this predominately singular focus to include data and interactions from three distinct regions or zones of research and measurement as shown in the figure below.

As company websites, e-Commerce sites and other forms of ‘owned’ media proliferated, web analytics software provided an explosion of data and new metrics like unique visitors, page views, click through rates, duration, referring sites and conversions become widely used and reported.  We became over-served with data and underserved with insight.

The exponential rise in popularity of social networks in the last five years raised the bar again and presented new challenges in digital research and measurement.  Now we were faced with measuring conversations and not just clicks.  Measuring engagement became more important than measuring eyeballs.  The frontier in social media measurement is evolving toward measuring both the conversations and behavior patterns occurring within social networks, and understanding and connecting the underlying influences and motivations for the online behavior.

The third area of interest is in all the real-world, offline interactions and transactions. Scan and other digital sales data is important to understanding, tracking and connecting online and offline behavior and actions.   Connecting mobile transactions, online and offline behavior and WOM is a significant challenge.

Although we have attempted to define three distinct ‘zones’ of digital research and measurement necessary to address the full spectrum of social media and marketing impact, a robust measurement strategy should take a holistic, integrated approach using methodologies, tools, data and metrics from all three zones.  The goal is to be able to track the behavior, interactions and transactions of individuals across all three zones, across multiple platforms and physical locations, understanding how online behavior impacts offline behavior and vice-versa.

New Scope, New Integration

Today at Fleishman Hillard, we recognize the very definition of public relations is rapidly evolving to encompass a much broader and more integrated view of communications and how we connect, engage and build relationships with consumers and other stakeholders on behalf of our clients.  Digitization in all its forms has driven and accelerated this important change.  While public relations has traditionally been oriented toward ‘earned media’ – gaining placements of client stories in print and broadcast media based on the strength of the story and quality of the pitch – today’s content-driven world demands much more.  The scope now must include all the consumer touch points available in our increasingly digital world.  We capture this new scope and integration in a model we refer to as PESO – Paid/Earned/Shared/Owned.  Our PESO model predates the similar Forrester model (Paid/Earned/Owned) and is different in an important way.  We created two categories, Earned and Shared, where the other model has one – Earned.  We believe this better comprehends strategies like blogger outreach and other proactive efforts undertaken by practitioners as ’Earned’,  distinct from efforts that may be passive or reactive.  Here is how we define the elements of our model:

Paid – refers to all forms of paid content that exists on third-party channels or venues.  This includes banner or display advertisements, pay-per-click programs, sponsorships and advertorials.

Earned – includes traditional media outreach as well as blogger relations/outreach where we attempt to influence and encourage third-party content providers to write about our clients and their products and services.

Shared – refers to social networks and technologies controlled by consumers along with online and offline WOM

Owned – includes all websites and web properties controlled by a company or brand including company or product websites, micro-sites, blogs, Facebook pages and Twitter channels.

The enhanced scope and integration represented by the PESO model drives a corresponding broadening and need for integration in digital research and measurement.  One can easily find themselves attempting to measure a highly integrated program that includes the awareness created with paid media, the relevance and information delivered via owned, the credibility delivered by earned media and measuring the conversations and interactions occurring in shared media.  Just from a metrics perspective, the PESO model requires a significant broadening in thinking as shown in the matrix below.

Digitization has changed what we need to research and measure, where we find data and how we perform analysis.  The future will bring more data, better tools and improved methodologies.  Sifting insights from the mounds of data will remain a major challenge.  The intersection of marketing, privacy concerns and research must be navigated.  The constant in all the change brought by digitization is who – human analysts and research.  Discovery and insight, like it was 15 years ago, remains fundamentally a human process.  It remains the analog constant in a world of digitization.

Measure the Puzzle Not the Pieces

1 May

A while back, I remember someone posting a question to a Linked-In discussion group along the lines of, ‘I just got my client a hit in USA Today.  How much is that worth?’.   More recently, ADWEEK ran an article entitled, Value of a Fan on Facebook: $3.60, citing an attempt by Vitrue to essentially assign a media value to a Facebook Fan.  (Sidebar: Is a Liker worth as much as a Fan?).  Setting aside an argument of the value attribution methodology used by Vitrue (I’m not a fan, or a liker), the fundamental issue I have with each example is the same, they are trying to measure the pieces and not the puzzle.

A media hit, a tweet, gaining a Fan/Liker, or obtaining a Follower are all pieces to a larger puzzle called a social media/business campaign, initiative, effort or program.   For simplicity, let’s refer to them as programs henceforth.  Programs have, or should have, objectives.  Done correctly, these objectives are measurable.  Good measurement practice suggests you assess performance against stated objectives.  Sure, it is also important to assess performance of program strategies and tactics – primarily as a diagnostic – but ultimately we must measure performance against objectives.  This is a base condition for accountability.

Gaining media coverage, sending tweets or getting others to tweet about you, creating Likers or gaining Followers should be thought of as strategies or perhaps tactics.  Objectives are what you want to happen as a result of the combination of strategies and tactics.  Programs are not made of single media hits, tweets, Likers or Followers.  They are longitudinal, holistic and integrated.  Successful programs might generate hundreds of media hits, scores of blog posts, and thousands of Likers or Followers.  Orchestrated correctly, all these strategies and tactics should help us achieve our overall program objectives.  The reality of the situation is any one discrete result of a campaign – a hit, Liker or Follower for example – usually has a very small overall impact.  The impact most likely would not be measurable, and if it was, it would not likely be meaningful.  They are just pieces of the overall program puzzle.

Let’s conclude with a simplistic Facebook program example.  Your tactic is to gain more Likers that meet a certain demographic profile.  Your strategies are to create an engaged brand community in Facebook, and to encourage online and offline WOM about the brand.  Your objective is to increase brand preference from 17% to 21% in the next 12 months.  Measure this objective, and if you want to do value attribution and calculate ROI, figure out how much each 1% increase in brand preference is worth in incremental sales.  That’s a puzzle worth solving.

Photo From liza31337

Relief from your Social Media ROI Angst

25 Feb

Return on Investment (ROI) is one of the most discussed and agonized-over topics in social business today.  However, much of the discussion around social media ROI has been simply confused, confusing or misguided.  There have been posts on large, well-known blogs (riff on ways to prepare potatoes) that are incredibly naive in their discussion of ROI.  In some circles there is endless philosophic debate bounded on one side by the Puritans who believe ROI is an old/incorrect way to think social business and on the other side by the Analyticans who seem to believe you can always determine ROI with a web analytics package.

The net-net of all this is a lot of frustration, even angst over how to think about ROI and social business.  We’re here to help.  Take two deep breaths and read these four points.  You’ll feel better soon.

Point Number One:  As a practical matter, the majority of social business efforts will not result in true ROI (in the short term).

In fact, I would guess far less than half will.  Maybe less than 10%.  But that doesn’t mean the social business effort was not successful, or did not create significant value for the brand or organization.  It simply means the primary objectives of most social business efforts are centered on concepts like community-building, engagement, listening, and participating in conversations.  It is difficult and expensive to attribute financial value to these areas.  To use the old saying – the ROI on these sorts of ROI efforts is not good.  Traditional public relations, branding and reputation programs suffer from some of the same challenges.  So when a study like the one published by e-Marketer* suggests ‘only’ 16% of social business programs are measuring ROI, while many are surprised it isn’t higher, it actually sounds a little too high to me.   I wonder how respondents were thinking about and defining ROI.

Point Number Two: Loose use of the term ROI is a major cause of angst.

ROI is not synonymous with results, KPIs or value.  ROI is not the only or perhaps even the most relevant way to define success in social business.  ROI is a financial metric.  ROI can only be measured in terms of revenue generated, cost savings or costs avoided.  It is transactional in nature.   There is ample evidence on twitter, blog posts and in blog comments that many people say ROI when they really mean results or value.

Many of the well-intentioned but misguided attempts to rename or reinvent what ROI means in social media – return on influence and return on engagement probably getting the most play – seem to be the result of an inability to distinguish value creation from ROI.

Point Number Three: Understand the difference between value and ROI.

Social media efforts may create financial impact (ROI) and/or non-financial impact. Engagement and Influence are examples of non-financial impact.  Other non-financial impacts like increased brand awareness or purchase consideration may eventually result in ROI at a point in the future when a financial event occurs.  Try to be explicit as to whether the social business program is designed to generate non-financial impact or true financial ROI, and make sure the people writing the checks understand the difference.  Show how the effort is linked to one or more business processes and how it will deliver value by helping to drive the desired business outcomes.

We know social networks have the ability to create value through customer engagement and community building.  However, ROI can only be measured by their ultimate impact on downstream metrics like sales, employee retention and customer loyalty/repeat purchase.  Many social business efforts are in an investment phase.  The value is largely intangible.  Some may eventually become transactional and result in true financial ROI.

Point Number Four:  ROI in social business has a time dimension.

Value may be created in the short-term and longer-term.  Social business campaigns utilizing channel-specific URLs and ecommerce landing pages are an example of easily measured short-term ROI (setting aside last-click attribution issues).  Longer-term value is much more difficult to quantify.  There are some similarities between social media and brand in this regard.  Success in each is a process and not an event.  You may have ongoing activities that sustain the brand/social business program and brand building events or campaigns that provide short-term spikes in awareness and engagement.  Managing and measuring your social business effort properly requires thinking about the value you are creating in the short and longer-term.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business process transaction that results in ROI.  Calculate ROI whenever you can, but also try to articulate the value your programs will be creating, and how this value aligns with, and contributes toward, achieving one or more desired business outcomes.

If you are not feeling better already, please leave a comment and let us know why.

*Mzinga and Babson Executive Education, Social Software in Business, September 8, 2009

Social Media ROI Part 2: Research Approaches

8 Oct

In Part 1, we attempted to define a framework for thinking about measuring the ROI of social media activities and programs.  In this post we’ll take a relatively high-level view of specific research approaches that are applicable to calculating social business return on investment.

One of the items I stress to my PR/Advertising research students is the need for a researcher to understand the industry/business in which they are operating, how the discipline (e.g. PR or advertising) works as well as how the specific program or initiative is designed to ‘work’.  Two baseline concepts of the ROI Framework presented in Part 1 are the need to establish measurable objectives, and that these objectives should tie to one or more relevant business processes.  Alignment is crucial here and must be addressed as part of the planning process not post-execution.

At the risk of throwing-out a less than fully fleshed-out idea (OK, I am, so lets improve it together), here is a table to help you think through possible ways to align social programs.  The table shows the business functions/departments, a few possible uses of social programs in each area, the applicable business process, a few sample metrics and the basis (revenue, cost savings, cost avoidance) for creating ROI.

Microsoft Word-1

ROI Research Approaches

Direct Linkage – This approach is most applicable to social media promotional or e-Commerce efforts.  It generally involves use of unique URLs tied to specific social networks that direct respondents to a company e-Commerce website to redeem coupons or purchase product.  Using this approach, Dell generated over $2MM in incremental sales on their outlet site primarily driven by offers to their over 625K Twitter followers.  Direct linkage approaches mitigate two potential problems with ROI determination – tying the offer directly to an action and isolating the impact of each marketing channel.  Web analytics should provide the data necessary to determine ROI.

Staff Cost Reduction – The CRM or customer service and support functions are one of the more interesting uses of social networks.  There is some early work (e.g. Forrester Research) showing how social programs may directly reduce staff necessary for customer service and support.  For example, when questions can be answered by other customers and not just by the company.  ROI determination involves demonstrating how social programs have reduced staffing costs and call center investment requirements.  ROI may also be generated by enhanced customer loyalty resulting in higher average transaction volumes or more frequent purchases.

Correlation Modeling & Econometrics – Correlation models use statistical techniques to show the relationship between two variables of interest.  For example, we may be interested in how changes in Net Promoter Score correlate with sales.  The primary challenge with a correlation model is isolating the impact of social media from all the other ways – WOM, advertising, promotions, public relations – the change in the variable of interest may have occurred.  The simplest approach is to collect data during times of low or no other communication activity.  If multiple communications channels are in use, econometric models that attempt to statistically isolate the impact of each communication variable should be used.  Econometric modeling is expensive (in the ballpark of $100K to develop a model) and is data sensitive. That is, a lot of data is generally required for the models to work properly.  One also needs a lot of data (generally model designers want two or three years worth of data to isolate effects like seasonality) in order to achieve sufficiently high confidence levels in the correlation.  Other challenges include data normalization and the estimation of the baseline level of sales which is defined as the sales that would occur in the absence of any promotion or marketing.  For retail econometric models with established brands, the baseline sales might be around 50% of the observed volume.

I prefer models that attempt to correlate PR/SM outputs to PR/SM outcomes, and then a second correlation involving PR/SM outcomes (e.g. purchase consideration or Net Promoter) with business outcomes like sales.  Here is a simplified overview of a possible modeling approach.

CorrelationModel

Econometric models have two important characteristics – they are predictive so once you develop a model, in the absence of changes in the assumptions,  it may be used for forecasting without the need to generate a new model, and it provides a way to address value attribution for non-financial indicators like exposure, engagement or influence.

Exposed/Not Exposed – This form of research attempts to identify those individuals within your target audience who were exposed to programs and content, and compare their purchase intent or purchase history with a control group of audience members who were not exposed to the program and content.  The ‘lift’ created within the exposed group is used to calculate ROI.  The research approach involves use of primary audience research to gather the data necessary to calculate ROI.  You would screen respondents for exposure to specific social programs (this is tricky from a research questionnaire perspective) using visual cues and/or descriptions, being as specific as possible.  Experience shows the percentage of the potential audience exposed to a given program may be fairly low.  Therefore you may need a large sample size to net enough ‘exposed’ respondents to have a statistically projectable sample.  This dynamic drives higher research costs of course.

Integrated, Cross-Platform Research – By utilizing a combination of web analytics, click-tracking, digital content analysis, sales/scan data and primary research it is possible to track behavior of individuals across websites and social networks.   Companies like Compete and ComScore are becoming more integrated in their offerings along these lines, combining online behavioral tracking with panel research.  Early efforts have focused on using a combination of click-tracking, primary research and sales scan data to track opinion, behavior, actions and transactions.  The effort undertaken by ComScore and Dunnhumby to measure MySpace advertising (see April 17 AdAge) is a great early example of the cross-platform approach to ROI determination.

We are still in the early stages of understanding ROI with social business programs.  I look forward to continuing the journey with you.  Thanks for reading!

(Please also see this article in this week’s IABC CW Bulletin for a discussion of social media return on investment – separating myth from methodology.)

Social Media ROI Part 1: Framework

6 Oct

Social Media ROI Framework

Here is a simple, five-step framework for developing a social media ROI measurement program.  Remember that not all social media initiatives will result in short-term ROI generation. It is also important to comprehend the results of programs that result in non-financial value or impact. (For a quick refresher on the difference between value and ROI read this).  Holistic measurement programs should be designed to track and measure non-financial impact as well as ROI.

1. Set measurable objectives aligned with business outcomes

Failure to begin with measurable objectives is probably the most common impediment to proper social media measurement.  A couple of award seasons ago I was a judge for a major PR campaign competition and was appalled by the low percentage of programs that actually contained measurable objectives – about 20% or so.  Your objectives should be aligned with one or more desired business outcomes.  Think through all the ways in which the social business effort will contribute toward driving the overall business forward.  Make sure the alignment is obvious and understood by all involved in program approval.

2. Link to and understand the requisite business process

In order to demonstrate ROI in social media it is necessary to link the results seen in social media with the relevant business processes they are addressing.  Social programs to date generally relate to one or more of the following business processes:

Business Process

Description

CRM Crowd-sourced help, info, recommendations, customer relationships
Research Competitive intelligence, insights, voice-of-customer, trends, feedback, reputation assessment, influencers
Marketing & Sales Ideation, product promotion, hyper-local marketing, lead generation and closure, fund raising, testing, brand attributes
Communication/PR/IR Stakeholder communication (internal and external)
Innovation/Product Development Crowd-sourced ideas, problem/opportunity identification, collaboration

For example, in a B2B company, you might try to link social media efforts with the lead generation and closure process.

New Model.pptx

For a program aimed at employee engagement, you might link social media efforts to the employee recruitment and retention business process.  For an e-Commerce company you might be able to directly link to the sales process through unique URLs or click-tracking technologies.

New Model.pptx-1

Understanding which business processes are impacted by social networks, and how, is fundamental to understanding ROI.

3. Select communications model, research approach and key metrics

In addition to understanding the business process impacted by social programs, it is important to have a communications model to assess non-financial impact.  The accepted Outputs – Outtakes – Outcomes communication model is difficult and confusing for many to understand and apply.  Here is an alternative communication model that is somewhat more intuitive and in tune with social media measurement.

  • Exposure – To what degree have we created exposure to content and message?
  • Engagement – Who is interacting/engaging with our content?  How and where?
  • Influence – The degree to which exposure and engagement have influenced perceptions and attitudes of the target audience.
  • Action – As a result of the social media effort, what actions if any has the target taken?”

Microsoft Word

The metrics listed are a starting point.  You will note these metrics come from all three “zones of measurement” – web analytics, digital content analysis and primary audience research.

Possible ROI research approaches include:

  • Correlation modeling and econometrics
  • Staff cost reduction and/or cost elimination tracking
  • Direct linkage via unique URLs and click-tracking
  • Exposed/Not-exposed primary audience research
  • Integrated cross-platform research – web analytics, content analysis, click tracking, primary research.

4. Gather and analyze data

Gather longitudinal data for the requisite business process and communications model metrics.  When attempting to show statistical correlations, the amount of data you include is important because it impacts the confidence level of the results.  Also, what metrics you attempt to correlate and how is crucial.  For example, we might try to correlate social media brand engagement and audience influence with metrics like likelihood to recommend to a friend, likelihood to seriously consider the product or likelihood to purchase the product in the next X months.  Select the metrics that are most applicable to the business process you are attempting to drive.

5. Calculate ROI and report results

We started with measurable objectives, aligned them with requisite business processes, determined our research model and approach and have gathered the data.  Now we can calculate ROI and describe the value of the social business initiative.  Results for key program metrics should be captured on a dashboard that may be shared with all program stakeholders are a regular basis.

(Note: Part two of this post will cover specific research approaches to determining social media ROI in more detail)

Public Relations Measurement 2010: Five Things to Forget & Five Things to Learn

29 Jul

(This post is a re-purposing of a speech I gave to the FPRA/PRSA-Orlando on July 23, 2009.  You can download the slides here.)

Public relations measurement is at a crossroads.  Old techniques are no longer sufficient.  Old metrics are no longer applicable.  Old thinking must be replaced by new.  The need for accountability, and to prove the value of PR and social media programs, has never been greater.

As we look to the next year, here are five things to forget and five things to learn about public relations measurement in 2010.

Things to Forget in 2010

1. Media Relations Focus

A focus on media relations fails to capture several important aspects of PR – brand, reputation, crisis, employee communication and DTC to name a few.  Also, the importance of traditional media is declining.  Numerous studies have shown people don’t trust what they read in the media, they trust each other.  I believe it was Hauser and Katz who coined the term ‘you are what you measure’ in 1998.  If measurement is focused on media relations that is how the public relations function will be judged.

__________________________________________________________

2. Outputs

The need to put PR results in a business context has never been greater.  We need to be able to address the question – what are we doing to help drive the business?  If you are focused on output metrics like impressions or message delivery, you will always have a hard time explaining business impact.  Instead, we need to focus on outcomes and answer the question – what happened as a result of our program or coverage?  Understanding outputs has primary benefit as a diagnostic tool rather than a ‘scorecard’.

__________________________________________________________

3. Impressions (and Multipliers!)

The most common PR metric today is Impressions.  While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek.  Impressions also (greatly) overstate actual relevant audience.  Generally only a fraction of any particular magazine or newspaper’s circulation meets your target audience demographics.  And impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content.  To compound the problems, many PR practitioners use a multiplier on impression numbers to account for pass-along readership or a mythical credibility advantage PR has over other communication tools.  The simple fact is there is no factual basis (e.g. research proof) that multipliers should be used in any case.

__________________________________________________________

4. Ad Equivalency (AVEs)
There are many reasons why using ad equivalency as a proxy for PR value is not advisable.  Here are five good reasons they should be avoided:

  • AVE calculations vary and there are no standards.  Tonality, article length, competitive mentions and other factors are handled differently.
  • AVE results can be misleading.  AVEs may be trending up while metrics like message communication, share of favorable positioning and share of positive press are falling.
  • AVEs reduce PR to just the media dimension by only assigning a value in this area.
  • AVEs only apply to traditional media.  What is the AVE of a positive conversation about your company on a leading blog?
  • How much is it worth for a troubled company to not appear in the Wall Street Journal?  AVEs cannot address this.

__________________________________________________________

5. Return on (Engagement/Influence/etc.)
Not a day goes by on Twitter without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks.  Wrong and wrong.  Most of these folks either don’t understand ROI or don’t know how to obtain the data necessary to calculate it.   There is also a lot of confusion between creating value and ROI.  Generating awareness creates value, for example, but may not immediately result in demonstrable ROI.

__________________________________________________________

Things to Learn in 2010

1. Total Value of PR

Microsoft PowerPoint

The majority of current PR measurement efforts focus on marketing/sales and output metrics.  The Total Value Cube is a way to visualize and think about all the potential value your PR and social media efforts deliver.  Beyond marketing to include brand and reputation, beyond outputs to include engagement, influence and action, and beyond revenue generation to include cost savings and cost avoidance.

__________________________________________________________

2. A New Model for Measurement
Many public relations practitioners regularly get their Outputs confused with their Outtakes or Outcomes.  Outtakes is not often used in the U.S. – it seems much more prevalent in Europe.  The overall terminology is confusing and is defined in different ways by different practitioners.  Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.

What we need is a metrics taxonomy that is easier to understand and explain.  I like this one.

Social Media Model.pptx

Exposure – to what degree have we created exposure to content and message?

Engagement – who, how and where are people interacting/engaging with our content?

Influence – the degree to which exposure and engagement have influenced perceptions and attitudes

Action – as a result of the PR/social media effort, what actions if any has the target taken?”

__________________________________________________________

3. Three Zones of Measurement

PRSA.FPRA.07.23.ppt-3

From the left, companies or brands control, own or manage websites  – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with.  This zone is measured primarily by web analytics.  In the middle are the actual social networks and conversations between individuals.   In this zone we are interested in data sets that cannot be gathered solely using web analytics packages.  How often is the brand being mentioned in conversation?  What is the sentiment of the comments?  How often is the brand being recommended and by whom?  Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone.  The third zone represents all the real-world, offline transactions that may be of interest.  Did someone visit the store or attend or event?  Did they buy a product?  Did they recommend the brand or product to a friend over coffee?  Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.

Your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones.  The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.

__________________________________________________________

4. New Metrics

PRSA.FPRA.07.23.ppt-2

_________________________________________________________

5. The Difference Between Impact/Value and ROI
ROI is a form of value/impact, but not all value takes form of ROI.  ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.  ROI lives on the income statement in business terms.

Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product.  Value creation occurs over time, not at a point in time.  Value creation is process-oriented.  Value lives on the balance sheet.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

The Difference Between Value and ROI

12 Jun

Social media and public relations programs create value and in some cases generate demonstrable ROI.  The two concepts are different in important ways.  They are related like the rectangle and the square.  Remember that silly distinction you learned in elementary school?  A square is a rectangle, but a rectangle is not a square.  ROI is a form of value, but not all value takes the form of ROI.

ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.  ROI lives on the income statement in business terms.

Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product.  Value creation occurs over time, not at a point in time.  Value creation is process-oriented.  Value lives on the balance sheet.

From a sales process perspective, the ultimate value of a social media program may be in Cash_registerincreasing the number of people who are likely to buy our products and services.  Other programs may be designed to improve or protect corporate reputation or to build and enhance brands.  Much of this value is said to be intangible.  It is goodwill that becomes tangible at the point in time a transaction occurs.   When buying decisions happen, your investments in marketing, brand and reputation work together.  They become tangible.  You can measure the ROI.

Many of the well-intentioned but misguided attempts to rename or reinvent what ROI means in social media – return on influence and return on engagement probably getting the most play – seem to be the result of an inability to distinguish value creation from ROI.  We know social networks hpiggy-websaveave the ability to create value through customer engagement and community building.  However, ROI can only be measured by their ultimate impact on downstream metrics like sales, employee retention and customer loyalty/repeat purchase.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

Trying to get, keep or increase your budget for social tools, people and programs?   Estimate ROI where you can, but also try to articulate the value your programs will be creating, and how this value aligns with, and contributes toward achieving one or more desired business outcomes.  Propose metrics to track and assess progress in exposure, engagement and audience influence.  This is a better conversation to have than, “Let me tell you about Return on Influence…”

Make sense?

Five Things You Should Know About Social Media ROI

8 Jun

In a January post of 2009 social media predictions I wrote:

2009 will be the year when the pendulum swings from experimentation to accountability.  2009 will raise the bar on all of use to demonstrate how social media and PR programs are helping to drive desired business outcomes.

Are you are seeing the accountability bar being raised this year?  In my corner of the world, the volume of conversation about social media ROI is high and accelerating. Unfortunately much of the conversation has been misinformed and misguided.  It seems like every week brings another post attempting to reinvent the acronym or the meaning – ROI really means Return on Influence, or Return on Engagement is the new ROI, and on and on.  There is another group of online Zen Masters who would have you believe social media ROI is old school thinking and not in tune with social media Zeitgeist.   In that case, I’ll take’ Old School’ for $100, please.

Here are five things about social media ROI you should know:

  1. Return on Investment is a financial metric.  It tells the percentage of financial return you generated for a given investment level.  The financial return is usually revenue, butdollar-sign.jpg (JPEG Image, 520x731 pixels) - Scaled (85%) may also be money you saved by making the investment or money you avoided spending in the future.   Notice the common thread here – its about money.
  2. Attempts to reinvent the acronym are counterproductive.   Return on Influence/ Engagement/Whatever; do not ever get to the basic money question.  Most of these attempts share two characteristics – they are confusing ‘return’ with impact/results (read Olivier Blanchard’s The BrandBuilder Blog for more on Impact/Return confusion), and/or they are making an argument that social media ROI is largely intangible, represented by relationships, engagement and community.   What they are really saying, perhaps unintentionally, is ROI is often difficult to determine and I really don’t understand it.  In my opinion, attempts to reinvent or circumvent ROI discussion in social media actually hurt credibility with the people writing the checks.  They expect an apples-to-apples – money in and money out – discussion.
  3. ROI in social media has a time dimension.  Value may be created in the short-term and longer-term.  Social media-specific promotions are an example of easily measuredsand.jpg (JPEG Image, 300x400 pixels) short-term ROI.  Longer-term value is much more difficult to quantify.  There are some similarities between social media and brand in this regard.  Success in each is a process and not an event.  You generally will have ongoing activities that sustain the brand/social media program and brand building events or campaigns that provide short-term spikes in awareness and engagement.  Contribution to organic search results is another example of longer-term value creation with branding and social media efforts.   Managing and measuring your social media effort properly requires thinking about the value you are creating in the short and longer-term.
  4. Linkage and correlations are important.  In order to demonstrate ROI in social media it is necessary to link the results seen in social media with the relevant business processes they are addressing.  For example, in a B2B company, you might try to link social media efforts with the lead generation and closure process.  For a program aimed at empchain-links.jpg (JPEG Image, 245x328 pixels)loyee engagement, you might link social media efforts to the employee recruitment and retention business process.  For an eCommerce company you might be able to directly link to the sales process through unique URLs or click-tracking technologies.  When attempting to show statistical relationships, correlations become important.  We might try to correlate social media brand engagement and audience influence with metrics like likelihood to recommend to a friend, likelihood to seriously consider the product or likelihood to purchase the product in the next X months.
  5. All ROI studies are custom.  The simple fact is you cannot buy an off-the-shelf solution to calculate the ROI of your social media effort.  All ROI studies are custom.  This is primarily a reflection of the unique objectives each company may have for their social media efforts.  Objectives are specific and contextual, and your ROI measurement efforts will need to be as well.   Attempts to develop ROI Calculators where you simply plug in several numbers and hit a button to calculate your ROI are not worth the time it takes to plug in the data.  They are a one-size fits none approach to ROI.  (Read this brilliant and very humorous post by The Brand Builder where he methodically skewers a recent ROI calculator attempt).

We are in the very early stages in our ability to measure the ROI of social media.  Not enough cycles yet.  Case studies are limited but growing.  The need to demonstrate a financial return on social media investment, if not here already, will be here shortly.  We have a lot of work to do.  Let’s get started.

Measuring Influence in Social Media

4 Jun

Every week there are multiple articles and posts on measuring Influence in social media.  The vast majority of these focus on assessing who are the ‘Influencers’ – those analysts, pundits, micro-celebrities and visionaries whose words and actions influence others in their online communities.  Influencers are an important element of your audience targeting strategy.  (Here’s a great post from Todd DeFren on audiences and influencers).

Influencers are a potential social media strategy, but we should measure social media objectives to determine whether or not programs are working as planned.  For that we have to turn to the other type of online influence, audience influence.

With audience influence, we want to understand what influence, if any, our social media efforts have had on audience opinions, attitudes and behaviors.  Here’s a social media measurement model that shows where audience Influence fits with the other major measurement stages, Exposure, Engagement and Action.

Social Media Model.pptx

  • Exposure – to what degree have we created exposure to content and message?
  • Engagement – who, how and where are people interacting/engaging with our content?
  • Influence – the degree to which exposure and engagement have influenced perceptions and attitudes
  • Action – as a result of the social media effort, what actions if any has the target taken?”

In the model, successful relationships with Influencers would be represented as an aspect of Engagement – i.e. Influencers have the ability to influence if and how consumers engage with brands.

To measure audience influence typically requires primary research to quantify attitudes and opinions and to assess the role, if any, social media efforts had in any attitudinal changes and subsequent behavior.  Once we understand how Exposure and Engagement are impacting Influence, and whether or not Influence is motivating Action, we are well on our way to the understanding and data necessary to demonstrate the true ROI generated by social media.

In summary, determining who has influence should be part of your audience targeting strategy, determining whether or not you are creating audience influence should be part of your measurement strategy.

See it a different way?

Follow

Get every new post delivered to your Inbox.

Join 169 other followers