Social Media ROI Part 2: Research Approaches

In Part 1, we attempted to define a framework for thinking about measuring the ROI of social media activities and programs.  In this post we’ll take a relatively high-level view of specific research approaches that are applicable to calculating social business return on investment.

One of the items I stress to my PR/Advertising research students is the need for a researcher to understand the industry/business in which they are operating, how the discipline (e.g. PR or advertising) works as well as how the specific program or initiative is designed to ‘work’.  Two baseline concepts of the ROI Framework presented in Part 1 are the need to establish measurable objectives, and that these objectives should tie to one or more relevant business processes.  Alignment is crucial here and must be addressed as part of the planning process not post-execution.

At the risk of throwing-out a less than fully fleshed-out idea (OK, I am, so lets improve it together), here is a table to help you think through possible ways to align social programs.  The table shows the business functions/departments, a few possible uses of social programs in each area, the applicable business process, a few sample metrics and the basis (revenue, cost savings, cost avoidance) for creating ROI.

ROI Research Approaches

Direct Linkage – This approach is most applicable to social media promotional or e-Commerce efforts.  It generally involves use of unique URLs tied to specific social networks that direct respondents to a company e-Commerce website to redeem coupons or purchase product.  Using this approach, Dell generated over $2MM in incremental sales on their outlet site primarily driven by offers to their over 625K Twitter followers.  Direct linkage approaches mitigate two potential problems with ROI determination – tying the offer directly to an action and isolating the impact of each marketing channel.  Web analytics should provide the data necessary to determine ROI.

Staff Cost Reduction – The CRM or customer service and support functions are one of the more interesting uses of social networks.  There is some early work (e.g. Forrester Research) showing how social programs may directly reduce staff necessary for customer service and support.  For example, when questions can be answered by other customers and not just by the company.  ROI determination involves demonstrating how social programs have reduced staffing costs and call center investment requirements.  ROI may also be generated by enhanced customer loyalty resulting in higher average transaction volumes or more frequent purchases.

Correlation Modeling & Econometrics – Correlation models use statistical techniques to show the relationship between two variables of interest.  For example, we may be interested in how changes in Net Promoter Score correlate with sales.  The primary challenge with a correlation model is isolating the impact of social media from all the other ways – WOM, advertising, promotions, public relations – the change in the variable of interest may have occurred.  The simplest approach is to collect data during times of low or no other communication activity.  If multiple communications channels are in use, econometric models that attempt to statistically isolate the impact of each communication variable should be used.  Econometric modeling is expensive (in the ballpark of $100K to develop a model) and is data sensitive. That is, a lot of data is generally required for the models to work properly.  One also needs a lot of data (generally model designers want two or three years worth of data to isolate effects like seasonality) in order to achieve sufficiently high confidence levels in the correlation.  Other challenges include data normalization and the estimation of the baseline level of sales which is defined as the sales that would occur in the absence of any promotion or marketing.  For retail econometric models with established brands, the baseline sales might be around 50% of the observed volume.

I prefer models that attempt to correlate PR/SM outputs to PR/SM outcomes, and then a second correlation involving PR/SM outcomes (e.g. purchase consideration or Net Promoter) with business outcomes like sales.  Here is a simplified overview of a possible modeling approach.

Econometric models have two important characteristics – they are predictive so once you develop a model, in the absence of changes in the assumptions,  it may be used for forecasting without the need to generate a new model, and it provides a way to address value attribution for non-financial indicators like exposure, engagement or influence.

Exposed/Not Exposed – This form of research attempts to identify those individuals within your target audience who were exposed to programs and content, and compare their purchase intent or purchase history with a control group of audience members who were not exposed to the program and content.  The ‘lift’ created within the exposed group is used to calculate ROI.  The research approach involves use of primary audience research to gather the data necessary to calculate ROI.  You would screen respondents for exposure to specific social programs (this is tricky from a research questionnaire perspective) using visual cues and/or descriptions, being as specific as possible.  Experience shows the percentage of the potential audience exposed to a given program may be fairly low.  Therefore you may need a large sample size to net enough ‘exposed’ respondents to have a statistically projectable sample.  This dynamic drives higher research costs of course.

Integrated, Cross-Platform Research – By utilizing a combination of web analytics, click-tracking, digital content analysis, sales/scan data and primary research it is possible to track behavior of individuals across websites and social networks.   Companies like Compete and ComScore are becoming more integrated in their offerings along these lines, combining online behavioral tracking with panel research.  Early efforts have focused on using a combination of click-tracking, primary research and sales scan data to track opinion, behavior, actions and transactions.  The effort undertaken by ComScore and Dunnhumby to measure MySpace advertising (see April 17 AdAge) is a great early example of the cross-platform approach to ROI determination.

We are still in the early stages of understanding ROI with social business programs.  I look forward to continuing the journey with you.  Thanks for reading!

(Please also see this article in this week’s IABC CW Bulletin for a discussion of social media return on investment – separating myth from methodology.)