Social Media ROI Part 2: Research Approaches

8 Oct

In Part 1, we attempted to define a framework for thinking about measuring the ROI of social media activities and programs.  In this post we’ll take a relatively high-level view of specific research approaches that are applicable to calculating social business return on investment.

One of the items I stress to my PR/Advertising research students is the need for a researcher to understand the industry/business in which they are operating, how the discipline (e.g. PR or advertising) works as well as how the specific program or initiative is designed to ‘work’.  Two baseline concepts of the ROI Framework presented in Part 1 are the need to establish measurable objectives, and that these objectives should tie to one or more relevant business processes.  Alignment is crucial here and must be addressed as part of the planning process not post-execution.

At the risk of throwing-out a less than fully fleshed-out idea (OK, I am, so lets improve it together), here is a table to help you think through possible ways to align social programs.  The table shows the business functions/departments, a few possible uses of social programs in each area, the applicable business process, a few sample metrics and the basis (revenue, cost savings, cost avoidance) for creating ROI.

Microsoft Word-1

ROI Research Approaches

Direct Linkage – This approach is most applicable to social media promotional or e-Commerce efforts.  It generally involves use of unique URLs tied to specific social networks that direct respondents to a company e-Commerce website to redeem coupons or purchase product.  Using this approach, Dell generated over $2MM in incremental sales on their outlet site primarily driven by offers to their over 625K Twitter followers.  Direct linkage approaches mitigate two potential problems with ROI determination – tying the offer directly to an action and isolating the impact of each marketing channel.  Web analytics should provide the data necessary to determine ROI.

Staff Cost Reduction – The CRM or customer service and support functions are one of the more interesting uses of social networks.  There is some early work (e.g. Forrester Research) showing how social programs may directly reduce staff necessary for customer service and support.  For example, when questions can be answered by other customers and not just by the company.  ROI determination involves demonstrating how social programs have reduced staffing costs and call center investment requirements.  ROI may also be generated by enhanced customer loyalty resulting in higher average transaction volumes or more frequent purchases.

Correlation Modeling & Econometrics – Correlation models use statistical techniques to show the relationship between two variables of interest.  For example, we may be interested in how changes in Net Promoter Score correlate with sales.  The primary challenge with a correlation model is isolating the impact of social media from all the other ways – WOM, advertising, promotions, public relations – the change in the variable of interest may have occurred.  The simplest approach is to collect data during times of low or no other communication activity.  If multiple communications channels are in use, econometric models that attempt to statistically isolate the impact of each communication variable should be used.  Econometric modeling is expensive (in the ballpark of $100K to develop a model) and is data sensitive. That is, a lot of data is generally required for the models to work properly.  One also needs a lot of data (generally model designers want two or three years worth of data to isolate effects like seasonality) in order to achieve sufficiently high confidence levels in the correlation.  Other challenges include data normalization and the estimation of the baseline level of sales which is defined as the sales that would occur in the absence of any promotion or marketing.  For retail econometric models with established brands, the baseline sales might be around 50% of the observed volume.

I prefer models that attempt to correlate PR/SM outputs to PR/SM outcomes, and then a second correlation involving PR/SM outcomes (e.g. purchase consideration or Net Promoter) with business outcomes like sales.  Here is a simplified overview of a possible modeling approach.

CorrelationModel

Econometric models have two important characteristics – they are predictive so once you develop a model, in the absence of changes in the assumptions,  it may be used for forecasting without the need to generate a new model, and it provides a way to address value attribution for non-financial indicators like exposure, engagement or influence.

Exposed/Not Exposed – This form of research attempts to identify those individuals within your target audience who were exposed to programs and content, and compare their purchase intent or purchase history with a control group of audience members who were not exposed to the program and content.  The ‘lift’ created within the exposed group is used to calculate ROI.  The research approach involves use of primary audience research to gather the data necessary to calculate ROI.  You would screen respondents for exposure to specific social programs (this is tricky from a research questionnaire perspective) using visual cues and/or descriptions, being as specific as possible.  Experience shows the percentage of the potential audience exposed to a given program may be fairly low.  Therefore you may need a large sample size to net enough ‘exposed’ respondents to have a statistically projectable sample.  This dynamic drives higher research costs of course.

Integrated, Cross-Platform Research – By utilizing a combination of web analytics, click-tracking, digital content analysis, sales/scan data and primary research it is possible to track behavior of individuals across websites and social networks.   Companies like Compete and ComScore are becoming more integrated in their offerings along these lines, combining online behavioral tracking with panel research.  Early efforts have focused on using a combination of click-tracking, primary research and sales scan data to track opinion, behavior, actions and transactions.  The effort undertaken by ComScore and Dunnhumby to measure MySpace advertising (see April 17 AdAge) is a great early example of the cross-platform approach to ROI determination.

We are still in the early stages of understanding ROI with social business programs.  I look forward to continuing the journey with you.  Thanks for reading!

(Please also see this article in this week’s IABC CW Bulletin for a discussion of social media return on investment – separating myth from methodology.)

11 Responses to “Social Media ROI Part 2: Research Approaches”

  1. Nick Lim October 16, 2009 at 9:57 am #

    Nice framework. The challenge with going from PR/SM -> PR/SM outcome; and then PR/SM -> business outcome is the indirect aspect and the possibility of confounding factors. In the PR/SM->outcome correlation, there could be an underlying confounding factor that is independent of the PR/SM effect, so it just leads to one more potential break in the generalizability of the finding.

    On another note, are you aware of any research linking the social network structure of a brands’ consumers to key business outcomes like sales? In essence, I’m trying to find a corollary to the smoking cessation finding – the higher the eigenvector centrality in the social network, the higher the likelihood that the person will quit smoking. See http://sonamine.wordpress.com/2009/06/08/smokers-quit-together-as-a-connected-cluster-in-a-social-network/

    thanks
    Nick

  2. metricsman October 16, 2009 at 10:10 am #

    Hi Nick,
    Thanks for stopping by and leaving a comment. You are absolutely right about confounding factors. Even with econometrics it still may be difficult to mitigate factors like colinearity of variables, etc. On your question, I have not seen any such research. I think what you are thinking about is very intriguing. You may have to do the research yourself! -Don B

  3. Nick Lim October 16, 2009 at 10:30 am #

    Indeed, the research would be intriguing, now the hard part – getting the data! Keep up the good work here.

    Nick

Trackbacks/Pingbacks

  1. The Importance of a Social Media ROI Diagnostic - October 22, 2009

    [...] metrics and benchmarks?  This should be broken down by business function or department.  See Metrics Man for a good example of what I [...]

  2. Irrelevant knowledge, media usage patterns, body posture and so much more… | Customer Engagement Unit - October 31, 2009

    [...] Social media ROI: research approaches [...]

  3. Video: Is Twitter a Boon or a Bane for Business? – Opposable Planets - November 30, 2009

    [...] follow up to this post I have a few referrals…  On measurement – I would head over to Metricsman and his great posts on measuring Social Media. On adopting the proper tone when using these tools I [...]

  4. Opposable Planets » Video: Is Twitter a Boon or a Bane for Business? - February 2, 2010

    [...] follow up to this post I have a few referrals…  On measurement – I would head over to Metricsman and his great posts on measuring Social Media. On adopting the proper tone when using these tools I [...]

  5. Social Media Kommunikation messen | Grenzpfosten - February 20, 2010

    [...] Ansatz fahren. Wie das gehen kann zeigt Don Bartholomew, der Metrics Man. Er nennt das Ganze Social Media ROI (auch der erste Teil ist sehr empfehlenswert) . Es besteht im Prinzip aus einem Raster von [...]

  6. Social Media Massnahmen messen » Social Media Führerschein - March 31, 2011

    [...] Metrics Man habe ich einen interessanten und übersichtlichen Ansatz gefunden den Social Media ROI [...]

  7. Customer Competencies, Co-Creation, and Brand Communities | Skilful Minds - April 29, 2013

    [...] producing cultural capital distinguishing their status relative to the community, aside from the ROI and business value gained by the company owning the brand. Making sure members are provided [...]

  8. New Framework for Social Media Measurement – Update & Debate | - June 21, 2013

    […] broad enough to comprehend the majority of use cases for social media. Marketing is just one of the use cases. To be fair, the use of ‘audience’ in the description of Impact is a direct lift from the […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 170 other followers

%d bloggers like this: