Let’s Play 20 Questions: Social Media Measurement Style

On August 6, I gave a webinar for Carma, co-sponsored by PRNews called, Social Media Measurement at a Crossroads. The webinar focused on the current state of social media measurement with an emphasis on efforts to develop social media metrics standards. You may download the presentation courtesy of Carma here. There were many good questions asked by the webinar participants. I thought it might be fun to capture 20 of the questions and share the answers I gave in response. And it might be cool if you disagree with an answer, to share your different opinion in the comments.

Q1. What level of social media measurement do you think should be taught at Undergraduate level in PR or Communications degree courses?

A1. Most schools only require one research class in undergraduate education. In this class, all forms of research including measurement are covered. I think all schools should have one general research and analytics course and another specifically for measurement. I would cover traditional and digital in both courses with an emphasis on digital techniques.

Q2. What needs to happen for businesses to be able to integrate Communications Performance Management with Business Performance Management?

A2. Did Philip Sheldrake ask you to ask this question? Well, the first thing that would have to happen is for companies to start demanding it. I’ve not seen much demand for this. Once demand builds, smart people will figure out how to make it happen. The AMEC Social Media Measurement Committee is going to take on the challenge of developing a balanced scorecard approach to the social media valid framework to see where that takes us.

Q3. Speaking about social business, are you suggesting social media becomes the strategic imperative with marketing, customer service, PR, employee engagement subordinate?  So these functions will be driven by SM specialists?

A3. No, not at all. I think what we’ll see if that social media permeates all of these functions and creates new capabilities and connections between groups and between customers and companies. It is up to PR or HR people to learn something about social media, SM specialists are not going to take over the world.

Q4. Are the proposed standard social media metrics valid for native ads as well?

A4. I have not thought much about this, but my initial reaction is that the metrics for native ads would be same. A promoted tweet would have the same engagement metrics as any other tweet, although one would certainly hope the performance on some of the metrics would be better.

Q5. What do you mean when you say triage social media content for customer service and support?

A5. This would refer to evaluating and routing social content to different entities or people within an organization (customer care versus technical support versus legal, for example) that are best able to understand and act on the feedback and/or respond to the post.

Q6. Don, what do you put more emphasis on these days, Likes and Follows or Shares and Comments?

A6. I believe the emphasis should be on the stronger indications of engagement, shares and comments, than on simple Likes and Follows.

Q7. How well-known and widely accepted are the Conclave standards in the social space as a whole?

A7. The first complete set of standards were published in early June, 2013. They are known by social media measurement insiders, but I think it is fair to say they are not yet widely known. We need to promote their existence and use.

Q8. How would you measure perception and attitudes through social media?

A8. Generally we would measure consumer conversations about a topic and then do some analysis to see if there are clusters of comments that represent different and distinct viewpoints, attitudes or opinions about the issue or topic. We might also want to do an audience segmentation analysis to see how these attitudes differ by stakeholder group.

Q9. Any specific comments geared towards non-profit organizations?

A9. The basics of measurement – write measurable goals, align goals with organizational KPIs, assess performance against targets –  are the same for for-profit and not-for-profit organizations. How value is created is the primary difference.

Q10. Any suggestions to measure business impact for B2B organizations? Is there a way to understand the impact of social for B2B organizations?  

A10. Most B2B companies have a focus on sales leads. Therefore demonstrating how social is helping create leads or improve lead closure rates is important. There are a lot of uses of social listening in B2B companies as well – how the company is positioned on key issues, who is talking about the company, how products and services are being discussed, etc.

Q11. What are your favorite tools to use in terms of actually measuring your programs/channels/campaigns? Do you identify the tools as you are defining the metrics (do we have the ability to measure X, Y, Z?) or do you select tools after you define your metrics (this is what we need to know, let’s find A, B, C, solutions to measure these things?)?

A11. Generally Google Analytics, a social listening platform (Radian6, Brandwatch, Netbase, Visible, etc.), channel analytics programs like Facebook Insights and also Excel. Ideally you should define metrics first, then the data required for each metric, then look at the tools best able to get the specific data you need.

Q12. What are the most common or most surprising questions you have gotten from CMOs or other key stakeholders regarding social media measurement?

A12. CMOs want to know how social media contributes value to marketing – if they are sales funnel oriented they want to know how social is helping drive the funnel for example. They are also interested if you are helping on front-end or downstream funnel metrics.

Q13. What advice do you have for small businesses for use of and measuring success of social media campaigns effectively (few resources).

A13. Start with the free tools (Hootsuite, Excel, Facebook Insights, Twitter Analytics, Google Analytics) and then work your way up to some of the paid social listening platforms. There is no ‘best’ platform to start with – it really depends on your needs and what you intend to do with the platform. Many companies start with measuring their own channels and evolve to listening to earned/ shared social conversations.

Q14. Which social media analytics do the C-suite find most valuable?

A14. The C-suite don’t really care about social media analytics so much, They care about how social media is helping drive the business metrics forward. That said, C-level folks are usually interesting in competitive benchmarking in social and positioning on key issues and topics that are important to the business. Anything pertaining to online reputation is also an area of interest for many.

Q15. How do you determine what are the correct things to measure?

A15. Measure what matters to the organization. Measurement is about performance against objectives so make sure your measurement program is aligned with business objectives. Don’t measurement ‘social media’, measure what you are trying to accomplish with social media.

Q16. How can someone who is interested in the movement toward standard metrics get involved helping to move the PR industry forward? In other words, how can someone get involved in the debate?

A16. I would suggest interacting directly on the smmstandards.org website. Volunteer to help. Leave suggestions. You could also get involved through one of the PR associations – IPR, PRSA or CoPRF.

Q17. What software would you recommend be used by PR firms to most cost effectively measure social media efforts for clients?

A17. A good social listening platform, Google Analytics, Facebook Insights and the other packages offered by the channels, and good old Excel. Beyond that it really depends on the nature of the social media effort.

Q18. I think a lot of the issue with measurement is confidence in the measurer (i.e., your source). Whenever you cross-reference measurements (e.g. what Google analytics says vs. what your web marketing automation says like HubSpot), you can get wildly different answers. That has stopped me from putting too much faith in my metrics process. Thoughts?

A18. I might separate the issue of the measurer from the sources of data – really two different issues. Regarding sources of data, this is a true issue in that different databases yield different estimates for things like audience size. Compete versus ComScore is a notorious example. However, I don’t think this is a reason to not measure. It simply means we must state assumptions and sources and be consistent over time in using comparable sources. I believe that standard metrics will eventually lead to sanctioned sources for audience data like Arbitron (now Nielsen Audio) for radio or Nielsen for TV.

Q19. Let’s say a social media post leads someone to a landing page, but they do not take immediate action. But they come back the next week and complete the conversion funnel. How do you credit the original social media post…is this a matter of tracking cookies for x number of days? What is practical?

A19. Yep, most people count the first click and then track for a period of time depending on the type of product. It gets even more complicated if you try to suggest there should also be credit given to what happened before the first social media click – for example, money invested in building the brand. Value attribution is an inexact science for sure, with lots of assumptions and compromises.

Q20. What are best ways to measure target audience reach and engagement rather than wide general reach?

A20. Thanks for asking this. The best way to measure is to clearly define your target. If the target is Females 18 – 34, then you should only take credit for reach and engagement of this specific audience only. Given that most tools rely on voluntary bio data, the information is inconsistent and difficult to come by.

Thanks for reading. @Donbart

A New Framework for Social Media Metrics and Measurement

Last week in Madrid, AMEC (International Association for the Measurement and Evaluation of Communications) held their 5^th European Summit on Measurement. This one was entitled, Unlocking Business Performance – Communications research and analytics in action. One underlying premise of the program this year was that the time for talk is behind us and the time for action and putting into place the principles and practices of sound measurement is upon us. The later part of the program featured an update from Salience Insight Commercial Director Mike Daniels on social media standards including the recently published standardization effort from the cross-industry group called The Conclave which may be found here.

Once Mike discussed where we are with respect to standardization, Richard Bagnall (@richardbagnall), Chairman of the AMEC Social Media Measurement Group, and I as his vice-chair, presented a session on creating a new recommended framework for social media metrics and measurement. Essentially we tried to answer the question: how do we take the standards work coming from The Conclave and operationalize it to create proper social media measurement? Here is an overview of what we presented and what we are encouraging everyone to adopt and use. The framework templates, usage guide and a short video synopsis will be available for download from the AMEC website, Social Media Measurement section, in the next week or so.

Valid Metrics Framework and Social Media

Our journey begins with the Valid Metrics Framework, a measurement planning framework and template developed under the auspices of AMEC. The framework was designed to be flexible enough to address multiple aspects of public relations within a consistent measurement framework and approach.

Some of the most positive aspects of the Valid Metrics Framework are that it:

• Provides a mechanism to link activities to outputs to outcomes
• Tracks through the familiar sales funnel
• Helps create a focus on outcomes.

One of the applications of the Valid Metrics Framework was for use with social media programs. Two potential issues were surfaced by early adopters of the framework in social programs. The intermediary effect, which in traditional public relations is the impact on the media, seemed at odds with the social world of direct interaction between consumers and brands, and consumers with each other. And use of the marketing sales funnel, while familiar, was only relevant in a percentage of social media use cases and perhaps not the best way to model common uses of social media like customer relations and building relationships with stakeholder groups. Also, thought leaders like Forrester Research and McKinsey & Company had noted the traditional communications funnel was not necessarily funnel-shaped in social media. They described the discovery process that occurs when investigating companies and brands that often cause the consideration set to expand rather than be reduced, and the fact that a lot of engagement around brands happens post-conversion event. For all of these reasons our task was develop and recommend a better framework and approach.

Models and Frameworks

When we use the word model, we are referring to a representation of a system, in this case social media. In the original Valid Metrics Framework, the model used was the traditional sales funnel. A framework adds additional dimensions to the model and is operationalized with metrics. In the Valid Metrics Framework the additional dimensions are the phases – activities, intermediary effects and target audience effects. We looked at both of these aspects individually and collectively when considering alternative approaches.

We studied and evaluated about fifteen different social media and communications models. A couple of common patterns emerged. Several of the models, including Forrester’s Customer Lifecycle and McKinsey’s Customer Journey showed a post-purchase engagement/experience step. We judged this important to include in our recommended approach. And we considered the importance of Engagement and Influence, as two key concepts in social media marketing and measurement, and decided to try to make these two elements explicit in our model as well.

Suggested Social Media Metrics Model

The model we developed is derivative of the categories chosen by The Conclave (Note: Richard Bagnall and I also participate in The Conclave) to organize social media metrics and definitions. We took a slightly different perspective on the front end of the model and reordered the back-end to create this model for our new framework. The descriptions of the stages use the definitions from the smmstandrards.org work wherever possible.

You will note Engagement includes both interactions with owned social channels as well as earned social conversation of people ‘talking about you’ in social channels. The definition of Influence is clear and concise, the result of a lot of discussion and prevailing clear thinking. The concept of Impact includes the outcomes of social initiatives as well as the Value those initiatives created. (I usually advise to always attempt to measure impact, and attribute value when it is feasible and makes sense.) Advocacy includes a very helpful definition and conditions that exist with advocacy.

Creating the Framework

To create the framework, we explored various ways to address the ‘phases’ of the Valid Metrics Framework. Two ideas stood out:

• Use a simple structure that captures social media metrics from three key perspectives – programmatic-level, channel-specific and business. Programmatic metrics are those directly tied to social media objectives. Channel-specific metrics are just that, the metrics that are unique to specific social channels – tweets, RTs, Followers, Likes, Talking About This, Pins, Re-Pins. Business metrics are used to show the business impact of the campaign or initiative.

• Use Paid, Owned, and Earned media metrics for integrated programs containing these elements. Borrowing the definitions from Forrester, Paid are social channels you pay to leverage (e.g. promoted tweets, display ads), Owned are channels you own and control (e.g. website, Facebook page) and Earned is where customers become the channel (e.g. WOM, viral)

There are certainly other ways to think about this (e.g. Business Performance Management) and we intend to possibly add others based on industry feedback and suggestions.

The AMEC Social Media Valid Framework

Currently we have developed both versions with sample metrics (taken from the smmstandards.org work where applicable). We are calling them The AMEC Social Media Valid Framework. Here is the version with Program, Channel and Business Metrics shown.

Where Do We Go From Here?

Look for the completed frameworks on the AMEC website shortly. We encourage you to adopt the frameworks for use by your company or clients. If you like them and find them useful, please help promote them widely. And please provide your feedback on the proposed framework on this blog, or through the social channel of your choice. We’re listening and looking forward to the dialogue.

Measurement 2020 and Other Fantasies

At the 3^rd European Summit on Measurement held in Lisbon in June 2011, standardization, education, ROI and measurement ubiquity emerged as the key themes in response to a call to set the Measurement Agenda 2020.  Delegates to the conference voted on 12 priorities they thought were most important to focus on in the period leading up to 2020.  The top four vote-getters became the Measurement Agenda 2020:

1. How to measure the return on investment of public relations (89%)
2. Create and adopt global standards for social media measurement (83%)
3. Measurement of PR campaigns and programs needs to become an intrinsic part of the PR toolkit (73%)
4. Institute a client education program such that clients insist on measurement of outputs, outcomes and business results from PR programs (61%)

For a very nice overview of the Lisbon session and the Barcelona Principles that came before, read this post from Dr. David Rockland of Ketchum who chaired the Barcelona and Lisbon sessions.  David pretty much said it all on these sessions, so I’ll just add a couple of comments and share a few thoughts on what I believe the future of measurement 2020 could be.

The rallying cry coming out of Barcelona has been focused and loud – death to AVEs!  Will there be a similar thematic coming out of Lisbon and what might it be?  My money is on standardization, borne out of cross-industry cooperation.  As David points out in his post, and in the words of AMEC Chairman Mike Daniels, “The Summit identified some significant challenges for the PR profession to address by 2020.  However, what we also accomplished in Lisbon beyond setting the priorities was to harness the commitment and energy of the industry to agree what we need to do together.”  The current cooperation and collaboration between industry groups – AMEC, Institute for Public Relations, PRSA and the Council of PR Firms is unprecedented in my time in this industry and is focused on tangible outcomes.  Cross-organization committees are already at work developing standard metrics for social media measurement for example.  The spirit of cooperation is uplifting.  While the outward thematic appears to be standardization, cooperation is the enabling force.  

I was also struck by the symmetry of the call to end AVEs in Barcelona and the call to codify ways to measure ROI in Lisbon.  One follows the other.  In my opinion the primary reason AVEs exist is because PR practitioners feel pressure to prove the value of what they do, and quite often they are asked to describe the impact in financial terms.  AVEs are perceived as a path of least resistance way to express financial value.  Except, as we all know, AVEs don’t really have anything to do with the impact public relations creates.  They are a misguided proxy for financial value.  Hence the need for research-based methods to determine true return on investment.

All of the priorities coming out of Lisbon are excellent goals for the industry.  And like David Rockland, I believe they will be achieved, and be achieved before 2020.  Here are three other items on my wish list for Measurement 2020:

Word of Mouth/Word of Mouse Integration: For those of us focused in social media and other digital technologies, we can’t allow our digital lens to color what is fundamentally an analog world.  Research studies suggest the majority of word of mouth happens in real life.  From an influence perspective, I don’t think too many would argue that word of mouth from a trusted friend or family member is more powerful than word of mouse from someone you follow on Twitter.  Digital cross-platform research is difficult enough, but when one huge platform is ‘real life’, we have significant challenges in measurement.  WOMMA and others have made early attempts to define measurement approaches for offline WOM, but much work remains.  We need ways to assess its impact and then we need to think about ways to attribute value to that impact.  Mobile is a wild card here as it becomes the preferred platform for online activity.  The need to triangulate online, mobile and ‘real life’ measurement presents significant challenges today, and may still by 2020.

Cookie Wars: We all know the measurement versus privacy showdown is coming, right?  The first shots have already been fired.  The collection of source-level personal data, enabled by cookies, is crucial to measurement and insights but has the potential for misuse or unintended disclosure.  Some sophisticated consumers have had their fill of cookies.  Although the broader issue might be framed as social sharing versus privacy control, how it plays out will have a direct impact on digital analytics and measurement.

Integrated Measurement across the Paid Earned Shared Owned (PESO) Spectrum: Measurement has increasingly become integrated.  It began with integrated traditional (Earned) and social media (Shared) measurement and then progressed rapidly to Earned, Owned and Shared, which is where most integrated measurement programs are today.  Many leading-edge integrated programs today also include advertising or Paid media.  By 2020, integrated measurement across the PESO spectrum will most likely be the norm and not the exception.  A key enabling element here in my view is some base level of agreement on how each area should be measured and standard metrics for each.  It will take significant cooperation between industry groups, vendors, agencies and major customers/clients for cross-discipline standardization to move forward effectively.  We are at the beginning of this movement in 2011.  By 2020, it will be fascinating to look back and see how all this plays out.

When looking ahead to 2020, I am reminded of a measurement discussion pulled together by PRWeek a couple of years ago.  Many of the Measurati attended.  In response to a question of where measurement will be in five years, David Rockland replied (paraphrasing here), ‘Who knows?  Five years ago who would have guessed we would all be focused on how to measure social media?’  So, there is a certain fantasy element to discussing 2020 challenges in measurement.  What are your measurement fantasies?

AVEs Don’t Describe the Value of Media Coverage, They Sensationalize It.

Saturday, Wall Street Journal columnist Carl Bialik, The Numbers Guy, addressed the subject of advertising value equivalency (AVE).  This is perhaps the first example of a mainstream media publication shining a light of the controversial practice of AVEs.  (You can read the story here.)

The primary reason advertising value equivalents exist are because they are perceived to be a way to attribute value to programs that would otherwise be difficult to value directly.  They are a path of least resistance approach to return on investment calculations, but not a valid one.  Let’s take a deeper dive into the three specific examples in the WSJ story, ask the tough questions and discuss more valid ways to think about value attribution and ROI.

American Airlines  

You can enjoy both questionable valuation techniques and hyperbole in this article.  American Airlines stands to “make boatloads of cash” and “the airline company could gain as much as $95.9 million of exposure”.  Of really, let’s take a closer look.

The most incredible part of this financial calculation is the financial calculation itself.  The calculation is apparently based on sign placement within the arena and presumably the ‘impressions’ the brand will receive when people attending the venue see the signage and when TV cameras catch the signs when showing the scoreboard or during the action.  This is a very passive form of advertising that should have as its objective either creating top of mind awareness or perhaps creating more brand affinity.  Rather than using an advertising equivalency model that has no validity, a true measurement of the value created by naming rights would ask a series of questions designed to determine the actual, tangible (or even intangible) impact on the business:

• Revenue: Can incremental revenue generation in the form of higher passenger miles be directly attributed to the exposure created by the naming rights?  Is it possible that incremental revenue would actually be realized on a game by game basis, or would any positive impact be realized over a longer time horizon?  Have new customers been created as a direct result of the exposure generated by the naming rights?
• Brand: Can the increased exposure lead to people perceiving the brand differently and can the difference translate into higher transactional revenues generated or increased brand loyalty?

So where exactly are the ‘boatloads of cash’ American Airlines made?  Are they hitting the income statement in the form of incremental revenue or enhanced brand loyalty (repeat business)?  Are they residing on the balance sheet in terms of brand goodwill?  Given that American’s parent company AMR lost $11.5B dollars in the first decade of the 21^st century, its last profitable year was 2007 and they are projected to lose money in 2011 and 2012, they could use the cash.  Perhaps they could use it to fund a ’bags fly free’ program or for enhancing their Advantage program to create more brand loyalty.  I would strongly suspect American’s shareholders would prefer a do-over on the investments made on naming rights to the ‘boatloads of cash’ they are now enjoying from the investment.

Couple Won’t Cash In on Kiss

15 minutes of fame is rarely worth $10 million.  In this case, the celebrity agent is suggesting the news value of the coverage generated by the kiss is somehow equivalent to advertising value and assigns what appears to be an arbitrary and ridiculously high value to it.  (He later admits he just made the number up.)  Just how was the couple going to monetize their 15 minutes of fame?   Yes, they turned down a few talk show opportunities and perhaps the National Enquirer would have thrown a few dollars their way for an exclusive, but the assertion that any major brand would have paid them to endorse their product is wildly speculative.  I would guess that if you did a survey after the event, a small number of people would remember seeing the coverage, and a very small percentage of the people who did see it would have recalled Scott Jones’ name.  So perhaps Mr. Jones walked away from tens of thousands of potential dollars in the short-term, but nowhere near the sensationalized estimate of $10 million.  15 minutes of fame might be worth 10 thousand dollars, but certainly not $10 million.

Obama Enjoys a Guinness

So Guinness is a winner and received $20 million worth of “free publicity”?  What was the outcome of the publicity?  Again, in order to determine the value of the “free publicity” (this term is despised in the PR industry by the way), Guinness would have to be able to measure incremental revenues directly attributable to the publicity generated.  Did sales of Guinness increase as a result?  Were new customers created?  Did existing customers feel compelled to drink even more?  What was the value of the incremental sales?  These are much more difficult questions to answer but are the correct ones to ask in order to measure the publicity.  Not by focusing on the mythical value of the coverage as measured by flawed advertising equivalency, but measuring the outcome or what happened as a result of the publicity.  The assertion that President’s Obama’s image was softened and will help keep him in the public’s favor is highly dubious thinking.  Perhaps it helps him in Boston, but in the grand scheme of things, this is a Presidential image non-event.

Beginning last Summer in Barcelona,  the public relations industry has come together to publicly state advertising value equivalency is not a valid measure of public relations.  The so-called Barcelona Principles are explicit against AVEs and also call for a focus on measuring outcomes and not (just) outputs.  While it will take some time for the PR industry to totally leave AVEs behind, there is a lot of momentum right now to make this happen sooner rather than later.  No serious measurement effort can use advertising value equivalency to attribute value and be credible.  

Social Media Listening Platforms – Plan, Select, Deploy (Part Three – Deploy)

In Part Two of this series on social media listening platforms we offered a process for selecting a social media listening platform vendor.  Now it’s time to deploy the tool across your organization effectively and with minimal disruption.  And put the tool to work.

Configuration – We talked about value-added services in the first post in this series.  One of the services offered by many listening platform vendors is configuration.  You’ll have to decide if you want to have the vendor perform system configuration or do it yourself.  In some cases you have no choice – you submit keywords, topics and themes to the vendor and the system is programmed for you.  In other cases some basic configuration must be done by the platform vendor but the bulk of the configuration can be a DIY project.

Keywords and Topics – In part one of this series, we discussed the need to think through the keywords required to bring all relevant content into your platform.  The keywords might be company name, product/brand names, competitors, issues, segment names, executives and spokespersons and key messages.  During deployment you will need to build taxonomy around many of the keywords that represent concepts rather than singular ideas or names.  For example, if you have a message that centers on being an innovative company, you will have to decide what expressions in addition to the keyword ‘innovative’ may be classified as innovation –  leading-edge, technology leader, R&D leadership, breakthrough products, etc.  You will also have to decide words and terms to exclude from your analysis.  Both of these processes are iterative – make a change, check content relevancy, adjust, repeat.

Integration – There are a few different types of integration you may want to tackle during platform configuration and deployment.  Each of the possible forms of integration will take a little time to accomplish and may require some back and forth between you and the platform vendor and/or vendor to vendor.  I am a big fan of web analytics and social media integration.  With many listening platforms this is relatively straight forward to accomplish.  You may also want to integrate third-party data sources like Factiva, LexisNexis, VMS or Critical Mention.  Assuming the listening platform vendor you selected supports this type of integration, it also is relatively straight forward.  To address latency issues, make sure you specify load times for the content.

Reports and Workflow – Previously, we addressed many of the basic questions around reports and reporting.  In the deployment phase it’s time to make it real.  Design specific templates for each report you need.  Create a mock-up and share with your stakeholders to make sure everyone is on board with the look, feel and utility of the report.   You will want to test the various delivery mechanisms to be employed including all email clients and mobile platforms you believe may be used.  Generally speaking, assume a significant percentage of the audience may look at the report on a mobile device, making this an especially important dynamic to test.  Once you have the report format established, define your workflow process – who pulls data and when, who creates visuals and by when, who compiles and edits the report and by when, and who is responsible for distribution and against what schedule.

Training – The first decision to make with training is if you want to tackle it yourself or rely on the listening platform vendor to perform the training.  Some vendors have very strong training programs and others not so much.  Some vendors charge for training and some do the bulk of it for free.  You most likely will want to take a train-the-trainer hybrid approach to training – have a core one/two/three people trained by the platform vendor, and then charge this team with training within your company or organization.  With respect to training timing, make sure to begin training only after everyone has a log-in to the system so they can actually use the system during the training.  I usually refer to this as training with live ammo.  If you don’t do this you’ll find the half-life of training is pretty short – folks forget most of what they have learned very rapidly.  I also find a tell-show-do teaching methodology works very well (my friends at Radian6 approach training this way).  Show some slides that cover the basics, show a video or canned demo that brings it to life and then have everyone do some hands-on exercises using the platform.  Remember you will need to address initial training needs as well as ongoing needs as new users are brought on the platform.

Event-specific and Programmatic Planning – Related to keyword analysis and taxonomy build-out, it may be wise to create keyword groups for programs you know you will be asked to listen to and measure, and for any potential events, like a crisis, that you can anticipate or imagine.  With respect to programmatic listening and measurement, generally a combination of the right keywords and date-ranging will allow you to pull in program-specific content.  If programs are known at the time of configuration and deployment, get ahead of the curve and set-up the keyword groups or source filters you may need.

If a company, brand or organization has a social listening program, you are remiss if you don’t include specific keywords that may serve as an early-detection system for potential crisis.  For example, depending on the type of organization and industry, it may be advisable to set up a keyword search like this: Company Name AND fire OR explosion OR shooting OR recall OR kidnapping OR crash.

In today’s real-time world, in my opinion, it is no longer optional to have social media listening capabilities.  As a result of this three-part series on listening platforms, I hope you are better equipped to plan, select and deploy your platform effectively.

Thanks for reading.

Social Media Listening Platforms – Plan, Select, Deploy (Part One – Plan)

It is not difficult to find a social media listening platform/tool – there are over 100 to choose from.  What is difficult is to find the right tool.  It takes a keen understanding of your scope and requirements.  It takes an evaluation and selection process that will surface the best platform to fully meet your requirements.  And it takes a well thought-out process for deploying the platform across the organization in an effective and efficient manner.   There are many questions to be asked and answers to be given.  Asking the right questions at the right time is crucial.

It is helpful to think of the overall process in three phases:

Plan – Define requirements, stakeholders, scope

Select – Create a platform evaluation process tailored to your unique requirements

Deploy – The selected platform across the organization with training, workflow and other important issues addressed.

This three-part series will tackle each phase one at a time.  First up – Plan.

In many ways, the planning phase is the most important.  Overlook an important detail here and you may or may not be able to overcome it later.  Here are ten topic areas to discuss within your organization to make sure you are setting yourself up for success.

1. Stakeholders – What are the primary stakeholder groups within my company or organization?  Possible stakeholder groups might include marketing, corporate communications and customer service/care at the macro level.  Depending on the size of your organization, various regions, divisions, groups or product lines may also be distinct stakeholder groups.  Once you have identified the primary stakeholders, set up time to meet with each group.  Understand how they currently use social listening tools and what, from their perspective, are ‘must have’ capabilities versus ‘nice to have’ capabilities in a social listening platform.  Ask each stakeholder group the applicable questions from the list below.
2. Geographic Scope – What languages and countries are stakeholders interested in including in the platform?  Try to understand the relative priority of each country and language.  Also be sure to comprehend future requirements.  For example, if Chinese is not a priority today but will be within two years, you may want to only consider listening platforms that support two-byte languages.  Also probe to assess if social media content will need to be translated into other languages.  This may be primarily an internal workflow issue or outsourcing issue, but might also be a platform consideration.
3. Value-added Services – It is very important to develop a point of view on how monitoring, analysis and reporting will be done within your organization.  Will each stakeholder group be responsible for doing this themselves or will a centralized analytics and insights group be responsible?  In addition to the self-serve approach, you could consider outsourcing this work to your social listening platform vendor or to one of your agencies – PR, digital or advertising.  In my experience, it is easy for a company or organization to underestimate both the skill and time commitment necessary to make the self-serve approach effective.
4. Content/Data Types – Social media listening platform vendors generally include content from the primary social media properties –  Facebook, Twitter, Blogs, Forums, YouTube and MySpace (being generous here).  Flickr is also included in many.  Currently on vendor roadmaps are properties like Linked-In and perhaps customer review sites.  Make sure the content types the platform supports meets your stakeholder requirements.  It is also very important to understand how the social content is being aggregated and how frequently (see Reporting for more on latency issues).  The fundamental ways in which content is aggregated in social listening platforms are crawling the web, RSS feeds and third-party content aggregators (e.g. Boardreader for Forums).  Many platform vendors employ a hybrid approach.
5. Metrics and Analytics – Most social listening platforms either have a set group of analytics that deliver specific metrics or they offer configurable analytic ‘widgets’ that may be used to create metrics like share of conversation or volume and tone trend.  Some platforms offer a combination of these two approaches.  Based on your needs and measurement strategy/approach, define the analytics and metrics you would ideally like to see (e.g. volume, sentiment, messages, share-of-conversation, association with key topics).  In the vendor selection phase, this list will be useful to compare and contrast vendors.
6. Keywords and Topics – During the planning phase, it is wise to develop a list of the major keywords and topics you believe will be necessary for the listening platform.  These keywords might include the company name, key competitors, industry issues, market segment names,  brand names, product names, key spokespersons, executives and competitor and industry spokespersons.  Social media listening platforms have varying degrees of sophistication with respect to their search capability.  Some have full Boolean logic, others offer very simple AND/OR logic.  The importance of this difference depends to some degree on you company/brand name as well as the sophistication of the people who will be configuring and maintaining your system.  If, for example, your company name is a common word (e.g. Apple, Visa), you will need stronger logic capabilities that include proximity search.
7. Integration – Integration of varying data types – search, web, social, advertising, customer opinion and others – is the present and future of online measurement.  It is therefore important to understand what capabilities, if any, the social listening platform vendor has to integrate with other data types/streams.  Do they offer the ability to connect with web analytics packages via API for example?  The web/social integration is becoming increasingly common.  If you need to integrate traditional media with social, it might be a nice feature if the social listening platform allows third-party content aggregators like Factiva, Lexis Nexis, VMS or Critical Mention.
8. Reporting – During the planning phase it is helpful to think through a series of questions about reports and reporting.  What type of reports are necessary?  Who will be responsible for their creation?  How often will reports be issued?  Does the system need the capability to automatically generate and deliver reports?  What about automated alerts?  There are quite a wide range of report capabilities represented by the various vendors in the listening space.  One potentially critical area to explore during the vendor evaluation phase is related to report frequency and perhaps to report type (think crisis).  That is how often new content is brought into the system.  Content latency issues may cause real problems during a fast-moving crisis.  Generally, the content latency differs by media type.  Best for Twitter and worst (perhaps) for forums, some of which restrict crawling to no more than once per day.  Within Twitter, the type of relationship the vendor has with Twitter should also be explored.  Not all Firehose arrangements are the same.  While most social media listening platforms claim to be ‘real time’, it is interesting to ask the vendors to define what they mean by ‘real time’.  The answers may surprise you.
9. Access – Discuss who needs access to the listening platform and what they want to see and be able to do once they are in the system.  Do your different stakeholder groups (Divisions, product lines, brands, corporate, marketing, etc.) want or need a customized view of the data perhaps presented on a separate dashboard within the system?  It is also a good idea to have a perspective on who your power users will be versus the casual users.  This distinction applies not only to system access, but also in areas like training.
10. Engagement – Some social media listening platforms support engagement with content owners directly from the platform, others do not.  Some engagement capabilities are elegant, others are rudimentary.  Make sure to explore the engagement needs of your stakeholders and understand how important this capability is to them in the short and long-term.  If engagement capabilities are important, you will also want to explore if the system allows users to tag content, assign content, manage assignments and track workflow.

In Part Two, we’ll examine a rigorous process for social media listening platform vendor evaluation and selection.

Don’t Let the Tool Tail Wag the Measurement Dog

Social media listening and measurement tools are sexy.  Well, at least to those of us in research and measurement – it’s all relative right?  In the last three years or so there has been an explosion of social media tool vendors and platform choices.  Tools are sexy and important, but in the grand scheme of things are being overemphasized to some degree.  We are letting tools decide what we can measure without giving sufficient thought to what we should measure.  We are letting the tool tail wag the measurement dog.

There are several steps and decisions that should be addressed prior to selecting a tool or suite of tools.  Consider this diagram as a starting point to help you think through these interim considerations and decisions:

OBJECTIVES

Proper social media objectives should be measurable (indicate change in metric of interest and timeframe) and aligned with desired organizational outcomes.  Understanding the social media objectives will suggest broad parameters the measurement program, and ultimately the tool decision, must operate within.  For example, geographic coverage requirements, type of content to be considered and on-platform engagement capability may all be strongly suggested based on a review of social media objectives.

PROCESS

In addition to comprehending organizational or business outcomes, it is essential to understand the business process the social media program will address or drive.  If the program is marketing oriented, the sales funnel process (Awareness/Consideration/Preference/Sales/Loyalty) may be most appropriate.  For a brand-building campaign, the brand pyramid (Presence/Relevance/Performance/Advantage/Bonding) is what you want to measure your program impact against.  Other business processes that are commonly addressed by social media programs include customer service and support, CRM, corporate reputation and lead generation.

METRICS

Understanding the requisite business process the social media program is driving is crucial because each business process drives specific metrics.  For example, the sales funnel drives a specific metrics set:  percentage of unaided or aided awareness; percentage of the target audience who would consider the product/company; percentage who prefer the product/company; incremental sales revenues; percentage who would purchase the product again number or the number/amount of repeat purchases.  For B2B companies, the lead generation process would drive a different set of metrics: number of incoming leads; percentage/number of qualified leads; lead conversion rate; sales revenues generated.  In addition to the business process metric sets, there are other metrics areas like Exposure and Engagement we will want to address.  Reach/opportunities to see, share of positive discussion, comments/post ratio, number of @ mentions and RTs per 1000 followers are examples of ‘standard’ metrics that might be applicable for many social media programs.

Understanding how the social media program drives a specific business process is also important to our ability to describe the impact or, in some cases, return on investment the program has created.

DATA SETS

Each metric has data requirements, usually two pieces of data per metric – a numerator and a denominator.  Examine the set of metrics you have defined for your social media program.  Catalog all the specific pieces of data you need to compute the various metrics.  For example, the data needed to compute the basic sales funnel metrics and some ‘standard’ metrics might include:

• Number of individuals in the target audience
• Number of survey respondents
• Number of respondents ‘aware’ of the product/company
• Number of respondents who would consider/seriously consider purchasing the product/doing business with the company
• Number of respondents purchasing the product
• Amount of sales revenue directly attributable to the program
• Number of purchasers who purchased again
• Total branded mentions
• Volume of positive and negative mentions
• Number of posts
• Number of comments
• Number of RTs and @ mentions
• Number of followers

TOOLS

Armed with an understanding of all the data needed to calculate the metrics required to measure the social media program, you will be able to assess which tools or classes of tools best deliver the data you need.  Pick the best three to five tools for further evaluation.  You most likely will find no one tool can deliver the complete data set you need.  It is common to need two or more tools, e.g. web analytics package and social content analysis platform, in order to fully meet data requirements.  Budgetary constraints may also limit your ability to capture the entire data set required.

By addressing the interim steps leading up to tool selection, you will be able to make a more informed tool decision.  You also will have a much better chance of measuring what you should measure rather than settling for what you can measure.  No tool before its time.  Let the big dogs run.

The Barcelona Principles: Leaders Speak

Well, the Second European Summit on Measurement held last week in Barcelona has come and gone, but its impact may be felt for some time to come.  The Summit was organized by the International Association for the Measurement and Evaluation of Communication (AMEC) and the Institute for Public Relations.  The most notable outcome of the Summit was the creation of the ‘Barcelona Declaration of Research Principles’.  The Principles were debated and voted upon by about 200 delegates representing 33 countries and five global PR and measurement organizations (AMEC, IPR, PRSA, ICCO, The Global Alliance).  David Rockland, Ph.D. chaired the debate.

Here are the ‘Barcelona Declaration of Research Principles’:

1. Goal setting and measurement are fundamental aspects of any PR programs.
2. Media measurement requires quantity and quality – cuttings in themselves are not enough.
3. Advertising Value Equivalents (AVEs) do not measure the value of PR and do not inform future activity.
4. Social media can and should be measured.
5. Measuring outcomes is preferred to measuring media results.
6. Business results can and should be measured where possible.
7. Transparency and Replicability are paramount to sound measurement.

I asked three of the leaders of the conference to comment on four questions regarding the Summit and what it may mean for the future of measurement.  The leaders are:

Barry Leggetter, FPRCA, FCIRR is Executive Director of AMEC  (barryleggetter@amecorg.com)

Pauline Draper-Watts, is Chairperson of the Institute for Public Relations, Commission for Public Relations Measurement and Evaluation  (pauline.s.draper@gmail.com)

David Rockland, PhD, is Partner/CEO, Ketchum Pleon Change and Managing Director, Global Research (David.Rockland@ketchum.com)

Here are their thoughts on the Summit:

Q1. For those not able to attend the Summit, how would you briefly describe what they missed?

BL: A milestone moment when delegates from 33 countries agreed to take program measurement more seriously, starting with the abandonment of AVE’s

DR: Missed a great opportunity to network with colleagues from 33 countries, hear some engaging speakers, and be part of a moment in time where the industry first adopted a set of measurement principles.

Q2. From your perspective, what are the two or three most significant outcomes of the Second European Summit on Measurement?

BL:

• For AMEC to be successful in getting five global organisations on the same platform for the first time and talk from the same page about the need for the PR and media intelligence industry to act – not just talk – about improved methods of program measurement.
• That the Summit achieved its own break-through status in receiving speaker support from senior level clients from global organisations such as FedEx Corporation, Yahoo, Royal Philips Electronics, Nissan, Telefónica, Banco Santander and others.

PD-W: The percentages (voting) in favor for each of the Barcelona Principles following the discussion

DR: To me a significant outcome was a gathering of the industry in a manner where ideas were shared, friendships and partnership extended, and we agreed as an industry to look ahead to how we can do what we do better and professionalize the practice of public relations.

Q3. How do you hope agencies, companies and organizations operationalize the seven principles?

BL: I introduced quality management processes when a Director of Porter Novelli in the UK in the 90’s – the first agency in the world to make this commitment. It became part of our agency’s way of working. I hope agencies, companies and organisations will similarly make the same commitment to the Barcelona Principles and introduce more stretching methods of program measurement on all programs.

PD-W: Integrating them into the culture and corporate language within the organization so that they are lived out in practice.

DR: My hope would be that first the principles are widely talked about and become SOP for what we do. Second, that the term AVE disappears along with the incredibly counter-productive debate around this subject that has distracted the industry from its own development. And third, that each organization adapts the Principles into their own words and practices; when we see them translated into the languages of the 33 countries represented at the Summit, we’ll know it worked.

Q4. Please complete this sentence: A year from now, we will know the Second European Summit on Measurement was successful …….

BL: …if when I judge my next PR Awards schemes PR consultancies and company in-house PR teams are putting more effort into the Program Measurement heading on the award entry as they are now doing to demonstrating creativity!

PD-W: … the Barcelona principles (modified to reflect the comments made at the Summit and submitted afterwards) are not only adopted but also put into practice throughout the industry.

DR: …if we don’t hear PR practitioners continue to complain we don’t have a “seat at the table” because we lack the metrics and measurement approaches other disciplines have.

Keeping it real…and transparent

• I am a member of the Institute for Public Relations, Commission on Public Relations Measurement & Evaluation
• My agency, Fleishman Hillard, is a member of AMEC
• Ketchum is a sister Omnicom agency

The Digitization of Research and Measurement

This post first appeared as an agency guest post on Jason Fall’s Social Media Explorer blog.  You can see it here.

The field of public relations has undergone two major revolutions in the past 15 years or so.  The advent of the Internet represents the first revolution.  This revolution primarily impacted the way content was created, distributed and consumed.  It also fundamentally changed the nature of communication – remember email became the first killer app of the Internet revolution.  The second revolution is social networks.  Again content creation was impacted, led by consumer generated content in multiple forms.  Perhaps more importantly, peer-to-peer communication between consumers, and two-way communication between consumers and brands/companies, have been enabled and are having a profound impact on the way companies are organized and behave.  The worlds of marketing and public relations have made an analog to digital conversion.  And with it, we are in the midst of the digitization research and measurement.

New Models, New Metrics

Communication models are a linear representation of how a communication process works and are important in providing a framework for evaluation and measurement.   The Outputs – Outtakes – Outcomes communication model often used in public relations today has two primary deficiencies in the era of digitization and social networks – clarity and relevance. 

• Clarity: The model is difficult for many to understand and apply.  Public relations practitioners regularly get Outputs confused with Outtakes or Outcomes.  Outtakes are not often used in the U.S. – they seem much more prevalent in Europe.  The overall taxonomy can be confusing and is defined in different ways by different practitioners or organizations.  Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.
• Relevance: The model was developed when communication was media-centric.  Digitization, consumer-generated content and social networks have shifted communication from a media-centric world to a content-centric world.  How receivers of communication engage and are influenced by content has fundamentally changed.

What is needed is a metrics taxonomy that is easier to explain, understand and apply.  Ideally one that is applicable for traditional and social media.  Here is the model we apply at Fleishman Hillard.

With the new model comes new metrics primarily driven by social media/networks.  Exposure  includes traditional metrics like Impressions and Message Delivery, and digital metrics like Search Rank, Twitter Reach and Average Daily Visitors.  Engagement includes traditional metrics like Readership, but adds new metrics like Subscriptions, Repeat Visitors and Follower Mention %.  Influence in the model refers to influence of the target audience, not who has influence in social networks.  Influence metrics range from increases in Brand Consideration to changes in attitudes and opinions to changes in online click behavior.  Action metrics can range from event attendance to voting for/against legislation to buying a product.

New Data, New Places

Public relations research and measurement has historically been driven by content analysis.  As content increasingly became available in digital form, the techniques of research and measurement didn’t change so much as the way content was aggregated and delivered for analysis.  Then web-based platforms became available from a variety of vendors to digitize and automate content analysis while the metrics being measured – article counts, impressions, message uptake and sentiment for example – basically remained constant with previous, more manual, methods.  Today, the digitization of research and measurement has broadened from this predominately singular focus to include data and interactions from three distinct regions or zones of research and measurement as shown in the figure below.

As company websites, e-Commerce sites and other forms of ‘owned’ media proliferated, web analytics software provided an explosion of data and new metrics like unique visitors, page views, click through rates, duration, referring sites and conversions become widely used and reported.  We became over-served with data and underserved with insight.

The exponential rise in popularity of social networks in the last five years raised the bar again and presented new challenges in digital research and measurement.  Now we were faced with measuring conversations and not just clicks.  Measuring engagement became more important than measuring eyeballs.  The frontier in social media measurement is evolving toward measuring both the conversations and behavior patterns occurring within social networks, and understanding and connecting the underlying influences and motivations for the online behavior.

The third area of interest is in all the real-world, offline interactions and transactions. Scan and other digital sales data is important to understanding, tracking and connecting online and offline behavior and actions.   Connecting mobile transactions, online and offline behavior and WOM is a significant challenge.

Although we have attempted to define three distinct ‘zones’ of digital research and measurement necessary to address the full spectrum of social media and marketing impact, a robust measurement strategy should take a holistic, integrated approach using methodologies, tools, data and metrics from all three zones.  The goal is to be able to track the behavior, interactions and transactions of individuals across all three zones, across multiple platforms and physical locations, understanding how online behavior impacts offline behavior and vice-versa.

New Scope, New Integration

Today at Fleishman Hillard, we recognize the very definition of public relations is rapidly evolving to encompass a much broader and more integrated view of communications and how we connect, engage and build relationships with consumers and other stakeholders on behalf of our clients.  Digitization in all its forms has driven and accelerated this important change.  While public relations has traditionally been oriented toward ‘earned media’ – gaining placements of client stories in print and broadcast media based on the strength of the story and quality of the pitch – today’s content-driven world demands much more.  The scope now must include all the consumer touch points available in our increasingly digital world.  We capture this new scope and integration in a model we refer to as PESO – Paid/Earned/Shared/Owned.  Our PESO model predates the similar Forrester model (Paid/Earned/Owned) and is different in an important way.  We created two categories, Earned and Shared, where the other model has one – Earned.  We believe this better comprehends strategies like blogger outreach and other proactive efforts undertaken by practitioners as ’Earned’,  distinct from efforts that may be passive or reactive.  Here is how we define the elements of our model:

Paid – refers to all forms of paid content that exists on third-party channels or venues.  This includes banner or display advertisements, pay-per-click programs, sponsorships and advertorials.

Earned – includes traditional media outreach as well as blogger relations/outreach where we attempt to influence and encourage third-party content providers to write about our clients and their products and services.

Shared – refers to social networks and technologies controlled by consumers along with online and offline WOM

Owned – includes all websites and web properties controlled by a company or brand including company or product websites, micro-sites, blogs, Facebook pages and Twitter channels.

The enhanced scope and integration represented by the PESO model drives a corresponding broadening and need for integration in digital research and measurement.  One can easily find themselves attempting to measure a highly integrated program that includes the awareness created with paid media, the relevance and information delivered via owned, the credibility delivered by earned media and measuring the conversations and interactions occurring in shared media.  Just from a metrics perspective, the PESO model requires a significant broadening in thinking as shown in the matrix below.

Digitization has changed what we need to research and measure, where we find data and how we perform analysis.  The future will bring more data, better tools and improved methodologies.  Sifting insights from the mounds of data will remain a major challenge.  The intersection of marketing, privacy concerns and research must be navigated.  The constant in all the change brought by digitization is who – human analysts and research.  Discovery and insight, like it was 15 years ago, remains fundamentally a human process.  It remains the analog constant in a world of digitization.

Social Media ROI Part 2: Research Approaches

In Part 1, we attempted to define a framework for thinking about measuring the ROI of social media activities and programs.  In this post we’ll take a relatively high-level view of specific research approaches that are applicable to calculating social business return on investment.

One of the items I stress to my PR/Advertising research students is the need for a researcher to understand the industry/business in which they are operating, how the discipline (e.g. PR or advertising) works as well as how the specific program or initiative is designed to ‘work’.  Two baseline concepts of the ROI Framework presented in Part 1 are the need to establish measurable objectives, and that these objectives should tie to one or more relevant business processes.  Alignment is crucial here and must be addressed as part of the planning process not post-execution.

At the risk of throwing-out a less than fully fleshed-out idea (OK, I am, so lets improve it together), here is a table to help you think through possible ways to align social programs.  The table shows the business functions/departments, a few possible uses of social programs in each area, the applicable business process, a few sample metrics and the basis (revenue, cost savings, cost avoidance) for creating ROI.

ROI Research Approaches

Direct Linkage – This approach is most applicable to social media promotional or e-Commerce efforts.  It generally involves use of unique URLs tied to specific social networks that direct respondents to a company e-Commerce website to redeem coupons or purchase product.  Using this approach, Dell generated over $2MM in incremental sales on their outlet site primarily driven by offers to their over 625K Twitter followers.  Direct linkage approaches mitigate two potential problems with ROI determination – tying the offer directly to an action and isolating the impact of each marketing channel.  Web analytics should provide the data necessary to determine ROI.

Staff Cost Reduction – The CRM or customer service and support functions are one of the more interesting uses of social networks.  There is some early work (e.g. Forrester Research) showing how social programs may directly reduce staff necessary for customer service and support.  For example, when questions can be answered by other customers and not just by the company.  ROI determination involves demonstrating how social programs have reduced staffing costs and call center investment requirements.  ROI may also be generated by enhanced customer loyalty resulting in higher average transaction volumes or more frequent purchases.

Correlation Modeling & Econometrics – Correlation models use statistical techniques to show the relationship between two variables of interest.  For example, we may be interested in how changes in Net Promoter Score correlate with sales.  The primary challenge with a correlation model is isolating the impact of social media from all the other ways – WOM, advertising, promotions, public relations – the change in the variable of interest may have occurred.  The simplest approach is to collect data during times of low or no other communication activity.  If multiple communications channels are in use, econometric models that attempt to statistically isolate the impact of each communication variable should be used.  Econometric modeling is expensive (in the ballpark of $100K to develop a model) and is data sensitive. That is, a lot of data is generally required for the models to work properly.  One also needs a lot of data (generally model designers want two or three years worth of data to isolate effects like seasonality) in order to achieve sufficiently high confidence levels in the correlation.  Other challenges include data normalization and the estimation of the baseline level of sales which is defined as the sales that would occur in the absence of any promotion or marketing.  For retail econometric models with established brands, the baseline sales might be around 50% of the observed volume.

I prefer models that attempt to correlate PR/SM outputs to PR/SM outcomes, and then a second correlation involving PR/SM outcomes (e.g. purchase consideration or Net Promoter) with business outcomes like sales.  Here is a simplified overview of a possible modeling approach.

Econometric models have two important characteristics – they are predictive so once you develop a model, in the absence of changes in the assumptions,  it may be used for forecasting without the need to generate a new model, and it provides a way to address value attribution for non-financial indicators like exposure, engagement or influence.

Exposed/Not Exposed – This form of research attempts to identify those individuals within your target audience who were exposed to programs and content, and compare their purchase intent or purchase history with a control group of audience members who were not exposed to the program and content.  The ‘lift’ created within the exposed group is used to calculate ROI.  The research approach involves use of primary audience research to gather the data necessary to calculate ROI.  You would screen respondents for exposure to specific social programs (this is tricky from a research questionnaire perspective) using visual cues and/or descriptions, being as specific as possible.  Experience shows the percentage of the potential audience exposed to a given program may be fairly low.  Therefore you may need a large sample size to net enough ‘exposed’ respondents to have a statistically projectable sample.  This dynamic drives higher research costs of course.

Integrated, Cross-Platform Research – By utilizing a combination of web analytics, click-tracking, digital content analysis, sales/scan data and primary research it is possible to track behavior of individuals across websites and social networks.   Companies like Compete and ComScore are becoming more integrated in their offerings along these lines, combining online behavioral tracking with panel research.  Early efforts have focused on using a combination of click-tracking, primary research and sales scan data to track opinion, behavior, actions and transactions.  The effort undertaken by ComScore and Dunnhumby to measure MySpace advertising (see April 17 AdAge) is a great early example of the cross-platform approach to ROI determination.

We are still in the early stages of understanding ROI with social business programs.  I look forward to continuing the journey with you.  Thanks for reading!

(Please also see this article in this week’s IABC CW Bulletin for a discussion of social media return on investment – separating myth from methodology.)