Archive | October, 2009

Social Media ROI Part 2: Research Approaches

8 Oct

In Part 1, we attempted to define a framework for thinking about measuring the ROI of social media activities and programs.  In this post we’ll take a relatively high-level view of specific research approaches that are applicable to calculating social business return on investment.

One of the items I stress to my PR/Advertising research students is the need for a researcher to understand the industry/business in which they are operating, how the discipline (e.g. PR or advertising) works as well as how the specific program or initiative is designed to ‘work’.  Two baseline concepts of the ROI Framework presented in Part 1 are the need to establish measurable objectives, and that these objectives should tie to one or more relevant business processes.  Alignment is crucial here and must be addressed as part of the planning process not post-execution.

At the risk of throwing-out a less than fully fleshed-out idea (OK, I am, so lets improve it together), here is a table to help you think through possible ways to align social programs.  The table shows the business functions/departments, a few possible uses of social programs in each area, the applicable business process, a few sample metrics and the basis (revenue, cost savings, cost avoidance) for creating ROI.

Microsoft Word-1

ROI Research Approaches

Direct Linkage – This approach is most applicable to social media promotional or e-Commerce efforts.  It generally involves use of unique URLs tied to specific social networks that direct respondents to a company e-Commerce website to redeem coupons or purchase product.  Using this approach, Dell generated over $2MM in incremental sales on their outlet site primarily driven by offers to their over 625K Twitter followers.  Direct linkage approaches mitigate two potential problems with ROI determination – tying the offer directly to an action and isolating the impact of each marketing channel.  Web analytics should provide the data necessary to determine ROI.

Staff Cost Reduction – The CRM or customer service and support functions are one of the more interesting uses of social networks.  There is some early work (e.g. Forrester Research) showing how social programs may directly reduce staff necessary for customer service and support.  For example, when questions can be answered by other customers and not just by the company.  ROI determination involves demonstrating how social programs have reduced staffing costs and call center investment requirements.  ROI may also be generated by enhanced customer loyalty resulting in higher average transaction volumes or more frequent purchases.

Correlation Modeling & Econometrics – Correlation models use statistical techniques to show the relationship between two variables of interest.  For example, we may be interested in how changes in Net Promoter Score correlate with sales.  The primary challenge with a correlation model is isolating the impact of social media from all the other ways – WOM, advertising, promotions, public relations – the change in the variable of interest may have occurred.  The simplest approach is to collect data during times of low or no other communication activity.  If multiple communications channels are in use, econometric models that attempt to statistically isolate the impact of each communication variable should be used.  Econometric modeling is expensive (in the ballpark of $100K to develop a model) and is data sensitive. That is, a lot of data is generally required for the models to work properly.  One also needs a lot of data (generally model designers want two or three years worth of data to isolate effects like seasonality) in order to achieve sufficiently high confidence levels in the correlation.  Other challenges include data normalization and the estimation of the baseline level of sales which is defined as the sales that would occur in the absence of any promotion or marketing.  For retail econometric models with established brands, the baseline sales might be around 50% of the observed volume.

I prefer models that attempt to correlate PR/SM outputs to PR/SM outcomes, and then a second correlation involving PR/SM outcomes (e.g. purchase consideration or Net Promoter) with business outcomes like sales.  Here is a simplified overview of a possible modeling approach.


Econometric models have two important characteristics – they are predictive so once you develop a model, in the absence of changes in the assumptions,  it may be used for forecasting without the need to generate a new model, and it provides a way to address value attribution for non-financial indicators like exposure, engagement or influence.

Exposed/Not Exposed – This form of research attempts to identify those individuals within your target audience who were exposed to programs and content, and compare their purchase intent or purchase history with a control group of audience members who were not exposed to the program and content.  The ‘lift’ created within the exposed group is used to calculate ROI.  The research approach involves use of primary audience research to gather the data necessary to calculate ROI.  You would screen respondents for exposure to specific social programs (this is tricky from a research questionnaire perspective) using visual cues and/or descriptions, being as specific as possible.  Experience shows the percentage of the potential audience exposed to a given program may be fairly low.  Therefore you may need a large sample size to net enough ‘exposed’ respondents to have a statistically projectable sample.  This dynamic drives higher research costs of course.

Integrated, Cross-Platform Research – By utilizing a combination of web analytics, click-tracking, digital content analysis, sales/scan data and primary research it is possible to track behavior of individuals across websites and social networks.   Companies like Compete and ComScore are becoming more integrated in their offerings along these lines, combining online behavioral tracking with panel research.  Early efforts have focused on using a combination of click-tracking, primary research and sales scan data to track opinion, behavior, actions and transactions.  The effort undertaken by ComScore and Dunnhumby to measure MySpace advertising (see April 17 AdAge) is a great early example of the cross-platform approach to ROI determination.

We are still in the early stages of understanding ROI with social business programs.  I look forward to continuing the journey with you.  Thanks for reading!

(Please also see this article in this week’s IABC CW Bulletin for a discussion of social media return on investment – separating myth from methodology.)

Social Media ROI Part 1: Framework

6 Oct

Social Media ROI Framework

Here is a simple, five-step framework for developing a social media ROI measurement program.  Remember that not all social media initiatives will result in short-term ROI generation. It is also important to comprehend the results of programs that result in non-financial value or impact. (For a quick refresher on the difference between value and ROI read this).  Holistic measurement programs should be designed to track and measure non-financial impact as well as ROI.

1. Set measurable objectives aligned with business outcomes

Failure to begin with measurable objectives is probably the most common impediment to proper social media measurement.  A couple of award seasons ago I was a judge for a major PR campaign competition and was appalled by the low percentage of programs that actually contained measurable objectives – about 20% or so.  Your objectives should be aligned with one or more desired business outcomes.  Think through all the ways in which the social business effort will contribute toward driving the overall business forward.  Make sure the alignment is obvious and understood by all involved in program approval.

2. Link to and understand the requisite business process

In order to demonstrate ROI in social media it is necessary to link the results seen in social media with the relevant business processes they are addressing.  Social programs to date generally relate to one or more of the following business processes:

Business Process


CRM Crowd-sourced help, info, recommendations, customer relationships
Research Competitive intelligence, insights, voice-of-customer, trends, feedback, reputation assessment, influencers
Marketing & Sales Ideation, product promotion, hyper-local marketing, lead generation and closure, fund raising, testing, brand attributes
Communication/PR/IR Stakeholder communication (internal and external)
Innovation/Product Development Crowd-sourced ideas, problem/opportunity identification, collaboration

For example, in a B2B company, you might try to link social media efforts with the lead generation and closure process.

New Model.pptx

For a program aimed at employee engagement, you might link social media efforts to the employee recruitment and retention business process.  For an e-Commerce company you might be able to directly link to the sales process through unique URLs or click-tracking technologies.

New Model.pptx-1

Understanding which business processes are impacted by social networks, and how, is fundamental to understanding ROI.

3. Select communications model, research approach and key metrics

In addition to understanding the business process impacted by social programs, it is important to have a communications model to assess non-financial impact.  The accepted Outputs – Outtakes – Outcomes communication model is difficult and confusing for many to understand and apply.  Here is an alternative communication model that is somewhat more intuitive and in tune with social media measurement.

  • Exposure – To what degree have we created exposure to content and message?
  • Engagement – Who is interacting/engaging with our content?  How and where?
  • Influence – The degree to which exposure and engagement have influenced perceptions and attitudes of the target audience.
  • Action – As a result of the social media effort, what actions if any has the target taken?”

Microsoft Word

The metrics listed are a starting point.  You will note these metrics come from all three “zones of measurement” – web analytics, digital content analysis and primary audience research.

Possible ROI research approaches include:

  • Correlation modeling and econometrics
  • Staff cost reduction and/or cost elimination tracking
  • Direct linkage via unique URLs and click-tracking
  • Exposed/Not-exposed primary audience research
  • Integrated cross-platform research – web analytics, content analysis, click tracking, primary research.

4. Gather and analyze data

Gather longitudinal data for the requisite business process and communications model metrics.  When attempting to show statistical correlations, the amount of data you include is important because it impacts the confidence level of the results.  Also, what metrics you attempt to correlate and how is crucial.  For example, we might try to correlate social media brand engagement and audience influence with metrics like likelihood to recommend to a friend, likelihood to seriously consider the product or likelihood to purchase the product in the next X months.  Select the metrics that are most applicable to the business process you are attempting to drive.

5. Calculate ROI and report results

We started with measurable objectives, aligned them with requisite business processes, determined our research model and approach and have gathered the data.  Now we can calculate ROI and describe the value of the social business initiative.  Results for key program metrics should be captured on a dashboard that may be shared with all program stakeholders are a regular basis.

(Note: Part two of this post will cover specific research approaches to determining social media ROI in more detail)