Archive | PR Research RSS feed for this section

Let’s Play 20 Questions: Social Media Measurement Style

1 Oct

On August 6, I gave a webinar for Carma, co-sponsored by PRNews called, Social Media Measurement at a Crossroads. The webinar focused on the current state of social media measurement with an emphasis on efforts to develop social media metrics standards. You may download the presentation courtesy of Carma here. There were many good questions asked by the webinar participants. I thought it might be fun to capture 20 of the questions and share the answers I gave in response. And it might be cool if you disagree with an answer, to share your different opinion in the comments.

Q1. What level of social media measurement do you think should be taught at Undergraduate level in PR or Communications degree courses?

A1. Most schools only require one research class in undergraduate education. In this class, all forms of research including measurement are covered. I think all schools should have one general research and analytics course and another specifically for measurement. I would cover traditional and digital in both courses with an emphasis on digital techniques.

Q2. What needs to happen for businesses to be able to integrate Communications Performance Management with Business Performance Management?

A2. Did Philip Sheldrake ask you to ask this question? Well, the first thing that would have to happen is for companies to start demanding it. I’ve not seen much demand for this. Once demand builds, smart people will figure out how to make it happen. The AMEC Social Media Measurement Committee is going to take on the challenge of developing a balanced scorecard approach to the social media valid framework to see where that takes us.

Q3. Speaking about social business, are you suggesting social media becomes the strategic imperative with marketing, customer service, PR, employee engagement subordinate?  So these functions will be driven by SM specialists?

A3. No, not at all. I think what we’ll see if that social media permeates all of these functions and creates new capabilities and connections between groups and between customers and companies. It is up to PR or HR people to learn something about social media, SM specialists are not going to take over the world.

Q4. Are the proposed standard social media metrics valid for native ads as well?

A4. I have not thought much about this, but my initial reaction is that the metrics for native ads would be same. A promoted tweet would have the same engagement metrics as any other tweet, although one would certainly hope the performance on some of the metrics would be better.

Q5. What do you mean when you say triage social media content for customer service and support?

A5. This would refer to evaluating and routing social content to different entities or people within an organization (customer care versus technical support versus legal, for example) that are best able to understand and act on the feedback and/or respond to the post.

Q6. Don, what do you put more emphasis on these days, Likes and Follows or Shares and Comments?

A6. I believe the emphasis should be on the stronger indications of engagement, shares and comments, than on simple Likes and Follows.

Q7. How well-known and widely accepted are the Conclave standards in the social space as a whole?

A7. The first complete set of standards were published in early June, 2013. They are known by social media measurement insiders, but I think it is fair to say they are not yet widely known. We need to promote their existence and use.

Q8. How would you measure perception and attitudes through social media?

A8. Generally we would measure consumer conversations about a topic and then do some analysis to see if there are clusters of comments that represent different and distinct viewpoints, attitudes or opinions about the issue or topic. We might also want to do an audience segmentation analysis to see how these attitudes differ by stakeholder group.

Q9. Any specific comments geared towards non-profit organizations?

A9. The basics of measurement – write measurable goals, align goals with organizational KPIs, assess performance against targets –  are the same for for-profit and not-for-profit organizations. How value is created is the primary difference.

Q10. Any suggestions to measure business impact for B2B organizations? Is there a way to understand the impact of social for B2B organizations?  

A10. Most B2B companies have a focus on sales leads. Therefore demonstrating how social is helping create leads or improve lead closure rates is important. There are a lot of uses of social listening in B2B companies as well – how the company is positioned on key issues, who is talking about the company, how products and services are being discussed, etc.

Q11. What are your favorite tools to use in terms of actually measuring your programs/channels/campaigns? Do you identify the tools as you are defining the metrics (do we have the ability to measure X, Y, Z?) or do you select tools after you define your metrics (this is what we need to know, let’s find A, B, C, solutions to measure these things?)?

A11. Generally Google Analytics, a social listening platform (Radian6, Brandwatch, Netbase, Visible, etc.), channel analytics programs like Facebook Insights and also Excel. Ideally you should define metrics first, then the data required for each metric, then look at the tools best able to get the specific data you need.

Q12. What are the most common or most surprising questions you have gotten from CMOs or other key stakeholders regarding social media measurement?

A12. CMOs want to know how social media contributes value to marketing – if they are sales funnel oriented they want to know how social is helping drive the funnel for example. They are also interested if you are helping on front-end or downstream funnel metrics.

Q13. What advice do you have for small businesses for use of and measuring success of social media campaigns effectively (few resources).

A13. Start with the free tools (Hootsuite, Excel, Facebook Insights, Twitter Analytics, Google Analytics) and then work your way up to some of the paid social listening platforms. There is no ‘best’ platform to start with – it really depends on your needs and what you intend to do with the platform. Many companies start with measuring their own channels and evolve to listening to earned/ shared social conversations.

Q14. Which social media analytics do the C-suite find most valuable?

A14. The C-suite don’t really care about social media analytics so much, They care about how social media is helping drive the business metrics forward. That said, C-level folks are usually interesting in competitive benchmarking in social and positioning on key issues and topics that are important to the business. Anything pertaining to online reputation is also an area of interest for many.

Q15. How do you determine what are the correct things to measure?

A15. Measure what matters to the organization. Measurement is about performance against objectives so make sure your measurement program is aligned with business objectives. Don’t measurement ‘social media’, measure what you are trying to accomplish with social media.

Q16. How can someone who is interested in the movement toward standard metrics get involved helping to move the PR industry forward? In other words, how can someone get involved in the debate?

A16. I would suggest interacting directly on the smmstandards.org website. Volunteer to help. Leave suggestions. You could also get involved through one of the PR associations – IPR, PRSA or CoPRF.

Q17. What software would you recommend be used by PR firms to most cost effectively measure social media efforts for clients?

A17. A good social listening platform, Google Analytics, Facebook Insights and the other packages offered by the channels, and good old Excel. Beyond that it really depends on the nature of the social media effort.

Q18. I think a lot of the issue with measurement is confidence in the measurer (i.e., your source). Whenever you cross-reference measurements (e.g. what Google analytics says vs. what your web marketing automation says like HubSpot), you can get wildly different answers. That has stopped me from putting too much faith in my metrics process. Thoughts?

A18. I might separate the issue of the measurer from the sources of data – really two different issues. Regarding sources of data, this is a true issue in that different databases yield different estimates for things like audience size. Compete versus ComScore is a notorious example. However, I don’t think this is a reason to not measure. It simply means we must state assumptions and sources and be consistent over time in using comparable sources. I believe that standard metrics will eventually lead to sanctioned sources for audience data like Arbitron (now Nielsen Audio) for radio or Nielsen for TV.

Q19. Let’s say a social media post leads someone to a landing page, but they do not take immediate action. But they come back the next week and complete the conversion funnel. How do you credit the original social media post…is this a matter of tracking cookies for x number of days? What is practical?

A19. Yep, most people count the first click and then track for a period of time depending on the type of product. It gets even more complicated if you try to suggest there should also be credit given to what happened before the first social media click – for example, money invested in building the brand. Value attribution is an inexact science for sure, with lots of assumptions and compromises.

Q20. What are best ways to measure target audience reach and engagement rather than wide general reach?

A20. Thanks for asking this. The best way to measure is to clearly define your target. If the target is Females 18 – 34, then you should only take credit for reach and engagement of this specific audience only. Given that most tools rely on voluntary bio data, the information is inconsistent and difficult to come by.

Thanks for reading. @Donbart

A New Framework for Social Media Metrics and Measurement

12 Jun

Last week in Madrid, AMEC (International Association for the Measurement and Evaluation of Communications) held their 5th European Summit on Measurement. This one was entitled, Unlocking Business Performance – Communications research and analytics in action. One underlying premise of the program this year was that the time for talk is behind us and the time for action and putting into place the principles and practices of sound measurement is upon us. The later part of the program featured an update from Salience Insight Commercial Director Mike Daniels on social media standards including the recently published standardization effort from the cross-industry group called The Conclave which may be found here.

Once Mike discussed where we are with respect to standardization, Richard Bagnall (@richardbagnall), Chairman of the AMEC Social Media Measurement Group, and I as his vice-chair, presented a session on creating a new recommended framework for social media metrics and measurement. Essentially we tried to answer the question: how do we take the standards work coming from The Conclave and operationalize it to create proper social media measurement? Here is an overview of what we presented and what we are encouraging everyone to adopt and use. The framework templates, usage guide and a short video synopsis will be available for download from the AMEC website, Social Media Measurement section, in the next week or so.

Valid Metrics Framework and Social Media

Our journey begins with the Valid Metrics Framework, a measurement planning framework and template developed under the auspices of AMEC. The framework was designed to be flexible enough to address multiple aspects of public relations within a consistent measurement framework and approach.

VMFII

Some of the most positive aspects of the Valid Metrics Framework are that it:

  • Provides a mechanism to link activities to outputs to outcomes
  • Tracks through the familiar sales funnel
  • Helps create a focus on outcomes.

One of the applications of the Valid Metrics Framework was for use with social media programs. Two potential issues were surfaced by early adopters of the framework in social programs. The intermediary effect, which in traditional public relations is the impact on the media, seemed at odds with the social world of direct interaction between consumers and brands, and consumers with each other. And use of the marketing sales funnel, while familiar, was only relevant in a percentage of social media use cases and perhaps not the best way to model common uses of social media like customer relations and building relationships with stakeholder groups. Also, thought leaders like Forrester Research and McKinsey & Company had noted the traditional communications funnel was not necessarily funnel-shaped in social media. They described the discovery process that occurs when investigating companies and brands that often cause the consideration set to expand rather than be reduced, and the fact that a lot of engagement around brands happens post-conversion event. For all of these reasons our task was develop and recommend a better framework and approach.

Models and Frameworks

When we use the word model, we are referring to a representation of a system, in this case social media. In the original Valid Metrics Framework, the model used was the traditional sales funnel. A framework adds additional dimensions to the model and is operationalized with metrics. In the Valid Metrics Framework the additional dimensions are the phases – activities, intermediary effects and target audience effects. We looked at both of these aspects individually and collectively when considering alternative approaches.

We studied and evaluated about fifteen different social media and communications models. A couple of common patterns emerged. Several of the models, including Forrester’s Customer Lifecycle and McKinsey’s Customer Journey showed a post-purchase engagement/experience step. We judged this important to include in our recommended approach. And we considered the importance of Engagement and Influence, as two key concepts in social media marketing and measurement, and decided to try to make these two elements explicit in our model as well.

Suggested Social Media Metrics Model

The model we developed is derivative of the categories chosen by The Conclave (Note: Richard Bagnall and I also participate in The Conclave) to organize social media metrics and definitions. We took a slightly different perspective on the front end of the model and reordered the back-end to create this model for our new framework. The descriptions of the stages use the definitions from the smmstandrards.org work wherever possible.

New Model

You will note Engagement includes both interactions with owned social channels as well as earned social conversation of people ‘talking about you’ in social channels. The definition of Influence is clear and concise, the result of a lot of discussion and prevailing clear thinking. The concept of Impact includes the outcomes of social initiatives as well as the Value those initiatives created. (I usually advise to always attempt to measure impact, and attribute value when it is feasible and makes sense.) Advocacy includes a very helpful definition and conditions that exist with advocacy.

Creating the Framework

To create the framework, we explored various ways to address the ‘phases’ of the Valid Metrics Framework. Two ideas stood out:

  • Use a simple structure that captures social media metrics from three key perspectives – programmatic-level, channel-specific and business. Programmatic metrics are those directly tied to social media objectives. Channel-specific metrics are just that, the metrics that are unique to specific social channels – tweets, RTs, Followers, Likes, Talking About This, Pins, Re-Pins. Business metrics are used to show the business impact of the campaign or initiative.
  • Use Paid, Owned, and Earned media metrics for integrated programs containing these elements. Borrowing the definitions from Forrester, Paid are social channels you pay to leverage (e.g. promoted tweets, display ads), Owned are channels you own and control (e.g. website, Facebook page) and Earned is where customers become the channel (e.g. WOM, viral)

There are certainly other ways to think about this (e.g. Business Performance Management) and we intend to possibly add others based on industry feedback and suggestions.

The AMEC Social Media Valid Framework

Currently we have developed both versions with sample metrics (taken from the smmstandards.org work where applicable). We are calling them The AMEC Social Media Valid Framework. Here is the version with Program, Channel and Business Metrics shown.

Valid Metrics Framwork

Where Do We Go From Here?

Look for the completed frameworks on the AMEC website shortly. We encourage you to adopt the frameworks for use by your company or clients. If you like them and find them useful, please help promote them widely. And please provide your feedback on the proposed framework on this blog, or through the social channel of your choice. We’re listening and looking forward to the dialogue.

Measurement 2020 and Other Fantasies

23 Sep

At the 3rd European Summit on Measurement held in Lisbon in June 2011, standardization, education, ROI and measurement ubiquity emerged as the key themes in response to a call to set the Measurement Agenda 2020.  Delegates to the conference voted on 12 priorities they thought were most important to focus on in the period leading up to 2020.  The top four vote-getters became the Measurement Agenda 2020:

  1. How to measure the return on investment of public relations (89%)
  2. Create and adopt global standards for social media measurement (83%)
  3. Measurement of PR campaigns and programs needs to become an intrinsic part of the PR toolkit (73%)
  4. Institute a client education program such that clients insist on measurement of outputs, outcomes and business results from PR programs (61%)

For a very nice overview of the Lisbon session and the Barcelona Principles that came before, read this post from Dr. David Rockland of Ketchum who chaired the Barcelona and Lisbon sessions.  David pretty much said it all on these sessions, so I’ll just add a couple of comments and share a few thoughts on what I believe the future of measurement 2020 could be.

The rallying cry coming out of Barcelona has been focused and loud – death to AVEs!  Will there be a similar thematic coming out of Lisbon and what might it be?  My money is on standardization, borne out of cross-industry cooperation.  As David points out in his post, and in the words of AMEC Chairman Mike Daniels, “The Summit identified some significant challenges for the PR profession to address by 2020.  However, what we also accomplished in Lisbon beyond setting the priorities was to harness the commitment and energy of the industry to agree what we need to do together.”  The current cooperation and collaboration between industry groups – AMEC, Institute for Public Relations, PRSA and the Council of PR Firms is unprecedented in my time in this industry and is focused on tangible outcomes.  Cross-organization committees are already at work developing standard metrics for social media measurement for example.  The spirit of cooperation is uplifting.  While the outward thematic appears to be standardization, cooperation is the enabling force.  

I was also struck by the symmetry of the call to end AVEs in Barcelona and the call to codify ways to measure ROI in Lisbon.  One follows the other.  In my opinion the primary reason AVEs exist is because PR practitioners feel pressure to prove the value of what they do, and quite often they are asked to describe the impact in financial terms.  AVEs are perceived as a path of least resistance way to express financial value.  Except, as we all know, AVEs don’t really have anything to do with the impact public relations creates.  They are a misguided proxy for financial value.  Hence the need for research-based methods to determine true return on investment.

All of the priorities coming out of Lisbon are excellent goals for the industry.  And like David Rockland, I believe they will be achieved, and be achieved before 2020.  Here are three other items on my wish list for Measurement 2020:

Word of Mouth/Word of Mouse Integration: For those of us focused in social media and other digital technologies, we can’t allow our digital lens to color what is fundamentally an analog world.  Research studies suggest the majority of word of mouth happens in real life.  From an influence perspective, I don’t think too many would argue that word of mouth from a trusted friend or family member is more powerful than word of mouse from someone you follow on Twitter.  Digital cross-platform research is difficult enough, but when one huge platform is ‘real life’, we have significant challenges in measurement.  WOMMA and others have made early attempts to define measurement approaches for offline WOM, but much work remains.  We need ways to assess its impact and then we need to think about ways to attribute value to that impact.  Mobile is a wild card here as it becomes the preferred platform for online activity.  The need to triangulate online, mobile and ‘real life’ measurement presents significant challenges today, and may still by 2020.

Cookie Wars: We all know the measurement versus privacy showdown is coming, right?  The first shots have already been fired.  The collection of source-level personal data, enabled by cookies, is crucial to measurement and insights but has the potential for misuse or unintended disclosure.  Some sophisticated consumers have had their fill of cookies.  Although the broader issue might be framed as social sharing versus privacy control, how it plays out will have a direct impact on digital analytics and measurement.

Integrated Measurement across the Paid Earned Shared Owned (PESO) Spectrum: Measurement has increasingly become integrated.  It began with integrated traditional (Earned) and social media (Shared) measurement and then progressed rapidly to Earned, Owned and Shared, which is where most integrated measurement programs are today.  Many leading-edge integrated programs today also include advertising or Paid media.  By 2020, integrated measurement across the PESO spectrum will most likely be the norm and not the exception.  A key enabling element here in my view is some base level of agreement on how each area should be measured and standard metrics for each.  It will take significant cooperation between industry groups, vendors, agencies and major customers/clients for cross-discipline standardization to move forward effectively.  We are at the beginning of this movement in 2011.  By 2020, it will be fascinating to look back and see how all this plays out.

When looking ahead to 2020, I am reminded of a measurement discussion pulled together by PRWeek a couple of years ago.  Many of the Measurati attended.  In response to a question of where measurement will be in five years, David Rockland replied (paraphrasing here), ‘Who knows?  Five years ago who would have guessed we would all be focused on how to measure social media?’  So, there is a certain fantasy element to discussing 2020 challenges in measurement.  What are your measurement fantasies?

AVEs Don’t Describe the Value of Media Coverage, They Sensationalize It.

26 Jun

Saturday, Wall Street Journal columnist Carl Bialik, The Numbers Guy, addressed the subject of advertising value equivalency (AVE).  This is perhaps the first example of a mainstream media publication shining a light of the controversial practice of AVEs.  (You can read the story here.)

The primary reason advertising value equivalents exist are because they are perceived to be a way to attribute value to programs that would otherwise be difficult to value directly.  They are a path of least resistance approach to return on investment calculations, but not a valid one.  Let’s take a deeper dive into the three specific examples in the WSJ story, ask the tough questions and discuss more valid ways to think about value attribution and ROI.

American Airlines  

You can enjoy both questionable valuation techniques and hyperbole in this article.  American Airlines stands to “make boatloads of cash” and “the airline company could gain as much as $95.9 million of exposure”.  Of really, let’s take a closer look.

The most incredible part of this financial calculation is the financial calculation itself.  The calculation is apparently based on sign placement within the arena and presumably the ‘impressions’ the brand will receive when people attending the venue see the signage and when TV cameras catch the signs when showing the scoreboard or during the action.  This is a very passive form of advertising that should have as its objective either creating top of mind awareness or perhaps creating more brand affinity.  Rather than using an advertising equivalency model that has no validity, a true measurement of the value created by naming rights would ask a series of questions designed to determine the actual, tangible (or even intangible) impact on the business:

  • Revenue: Can incremental revenue generation in the form of higher passenger miles be directly attributed to the exposure created by the naming rights?  Is it possible that incremental revenue would actually be realized on a game by game basis, or would any positive impact be realized over a longer time horizon?  Have new customers been created as a direct result of the exposure generated by the naming rights?
  • Brand: Can the increased exposure lead to people perceiving the brand differently and can the difference translate into higher transactional revenues generated or increased brand loyalty?

So where exactly are the ‘boatloads of cash’ American Airlines made?  Are they hitting the income statement in the form of incremental revenue or enhanced brand loyalty (repeat business)?  Are they residing on the balance sheet in terms of brand goodwill?  Given that American’s parent company AMR lost $11.5B dollars in the first decade of the 21st century, its last profitable year was 2007 and they are projected to lose money in 2011 and 2012, they could use the cash.  Perhaps they could use it to fund a ’bags fly free’ program or for enhancing their Advantage program to create more brand loyalty.  I would strongly suspect American’s shareholders would prefer a do-over on the investments made on naming rights to the ‘boatloads of cash’ they are now enjoying from the investment.

Couple Won’t Cash In on Kiss

15 minutes of fame is rarely worth $10 million.  In this case, the celebrity agent is suggesting the news value of the coverage generated by the kiss is somehow equivalent to advertising value and assigns what appears to be an arbitrary and ridiculously high value to it.  (He later admits he just made the number up.)  Just how was the couple going to monetize their 15 minutes of fame?   Yes, they turned down a few talk show opportunities and perhaps the National Enquirer would have thrown a few dollars their way for an exclusive, but the assertion that any major brand would have paid them to endorse their product is wildly speculative.  I would guess that if you did a survey after the event, a small number of people would remember seeing the coverage, and a very small percentage of the people who did see it would have recalled Scott Jones’ name.  So perhaps Mr. Jones walked away from tens of thousands of potential dollars in the short-term, but nowhere near the sensationalized estimate of $10 million.  15 minutes of fame might be worth 10 thousand dollars, but certainly not $10 million.

Obama Enjoys a Guinness

So Guinness is a winner and received $20 million worth of “free publicity”?  What was the outcome of the publicity?  Again, in order to determine the value of the “free publicity” (this term is despised in the PR industry by the way), Guinness would have to be able to measure incremental revenues directly attributable to the publicity generated.  Did sales of Guinness increase as a result?  Were new customers created?  Did existing customers feel compelled to drink even more?  What was the value of the incremental sales?  These are much more difficult questions to answer but are the correct ones to ask in order to measure the publicity.  Not by focusing on the mythical value of the coverage as measured by flawed advertising equivalency, but measuring the outcome or what happened as a result of the publicity.  The assertion that President’s Obama’s image was softened and will help keep him in the public’s favor is highly dubious thinking.  Perhaps it helps him in Boston, but in the grand scheme of things, this is a Presidential image non-event.

Beginning last Summer in Barcelona,  the public relations industry has come together to publicly state advertising value equivalency is not a valid measure of public relations.  The so-called Barcelona Principles are explicit against AVEs and also call for a focus on measuring outcomes and not (just) outputs.  While it will take some time for the PR industry to totally leave AVEs behind, there is a lot of momentum right now to make this happen sooner rather than later.  No serious measurement effort can use advertising value equivalency to attribute value and be credible.  

Social Media Listening Platforms – Plan, Select, Deploy (Part Three – Deploy)

17 Jun

In Part Two of this series on social media listening platforms we offered a process for selecting a social media listening platform vendor.  Now it’s time to deploy the tool across your organization effectively and with minimal disruption.  And put the tool to work.

Configuration – We talked about value-added services in the first post in this series.  One of the services offered by many listening platform vendors is configuration.  You’ll have to decide if you want to have the vendor perform system configuration or do it yourself.  In some cases you have no choice – you submit keywords, topics and themes to the vendor and the system is programmed for you.  In other cases some basic configuration must be done by the platform vendor but the bulk of the configuration can be a DIY project.

Keywords and Topics – In part one of this series, we discussed the need to think through the keywords required to bring all relevant content into your platform.  The keywords might be company name, product/brand names, competitors, issues, segment names, executives and spokespersons and key messages.  During deployment you will need to build taxonomy around many of the keywords that represent concepts rather than singular ideas or names.  For example, if you have a message that centers on being an innovative company, you will have to decide what expressions in addition to the keyword ‘innovative’ may be classified as innovation –  leading-edge, technology leader, R&D leadership, breakthrough products, etc.  You will also have to decide words and terms to exclude from your analysis.  Both of these processes are iterative – make a change, check content relevancy, adjust, repeat.

Integration – There are a few different types of integration you may want to tackle during platform configuration and deployment.  Each of the possible forms of integration will take a little time to accomplish and may require some back and forth between you and the platform vendor and/or vendor to vendor.  I am a big fan of web analytics and social media integration.  With many listening platforms this is relatively straight forward to accomplish.  You may also want to integrate third-party data sources like Factiva, LexisNexis, VMS or Critical Mention.  Assuming the listening platform vendor you selected supports this type of integration, it also is relatively straight forward.  To address latency issues, make sure you specify load times for the content.

Reports and Workflow – Previously, we addressed many of the basic questions around reports and reporting.  In the deployment phase it’s time to make it real.  Design specific templates for each report you need.  Create a mock-up and share with your stakeholders to make sure everyone is on board with the look, feel and utility of the report.   You will want to test the various delivery mechanisms to be employed including all email clients and mobile platforms you believe may be used.  Generally speaking, assume a significant percentage of the audience may look at the report on a mobile device, making this an especially important dynamic to test.  Once you have the report format established, define your workflow process – who pulls data and when, who creates visuals and by when, who compiles and edits the report and by when, and who is responsible for distribution and against what schedule.

Training – The first decision to make with training is if you want to tackle it yourself or rely on the listening platform vendor to perform the training.  Some vendors have very strong training programs and others not so much.  Some vendors charge for training and some do the bulk of it for free.  You most likely will want to take a train-the-trainer hybrid approach to training – have a core one/two/three people trained by the platform vendor, and then charge this team with training within your company or organization.  With respect to training timing, make sure to begin training only after everyone has a log-in to the system so they can actually use the system during the training.  I usually refer to this as training with live ammo.  If you don’t do this you’ll find the half-life of training is pretty short – folks forget most of what they have learned very rapidly.  I also find a tell-show-do teaching methodology works very well (my friends at Radian6 approach training this way).  Show some slides that cover the basics, show a video or canned demo that brings it to life and then have everyone do some hands-on exercises using the platform.  Remember you will need to address initial training needs as well as ongoing needs as new users are brought on the platform.

Event-specific and Programmatic Planning – Related to keyword analysis and taxonomy build-out, it may be wise to create keyword groups for programs you know you will be asked to listen to and measure, and for any potential events, like a crisis, that you can anticipate or imagine.  With respect to programmatic listening and measurement, generally a combination of the right keywords and date-ranging will allow you to pull in program-specific content.  If programs are known at the time of configuration and deployment, get ahead of the curve and set-up the keyword groups or source filters you may need.

If a company, brand or organization has a social listening program, you are remiss if you don’t include specific keywords that may serve as an early-detection system for potential crisis.  For example, depending on the type of organization and industry, it may be advisable to set up a keyword search like this: Company Name AND fire OR explosion OR shooting OR recall OR kidnapping OR crash.

In today’s real-time world, in my opinion, it is no longer optional to have social media listening capabilities.  As a result of this three-part series on listening platforms, I hope you are better equipped to plan, select and deploy your platform effectively.

Thanks for reading.

Social Media Listening Platforms – Plan, Select, Deploy (Part One – Plan)

19 May

It is not difficult to find a social media listening platform/tool – there are over 100 to choose from.  What is difficult is to find the right tool.  It takes a keen understanding of your scope and requirements.  It takes an evaluation and selection process that will surface the best platform to fully meet your requirements.  And it takes a well thought-out process for deploying the platform across the organization in an effective and efficient manner.   There are many questions to be asked and answers to be given.  Asking the right questions at the right time is crucial.

It is helpful to think of the overall process in three phases:

Plan – Define requirements, stakeholders, scope

Select – Create a platform evaluation process tailored to your unique requirements

Deploy – The selected platform across the organization with training, workflow and other important issues addressed.

This three-part series will tackle each phase one at a time.  First up – Plan.

In many ways, the planning phase is the most important.  Overlook an important detail here and you may or may not be able to overcome it later.  Here are ten topic areas to discuss within your organization to make sure you are setting yourself up for success.

  1. Stakeholders – What are the primary stakeholder groups within my company or organization?  Possible stakeholder groups might include marketing, corporate communications and customer service/care at the macro level.  Depending on the size of your organization, various regions, divisions, groups or product lines may also be distinct stakeholder groups.  Once you have identified the primary stakeholders, set up time to meet with each group.  Understand how they currently use social listening tools and what, from their perspective, are ‘must have’ capabilities versus ‘nice to have’ capabilities in a social listening platform.  Ask each stakeholder group the applicable questions from the list below.
  2. Geographic Scope – What languages and countries are stakeholders interested in including in the platform?  Try to understand the relative priority of each country and language.  Also be sure to comprehend future requirements.  For example, if Chinese is not a priority today but will be within two years, you may want to only consider listening platforms that support two-byte languages.  Also probe to assess if social media content will need to be translated into other languages.  This may be primarily an internal workflow issue or outsourcing issue, but might also be a platform consideration.
  3. Value-added Services – It is very important to develop a point of view on how monitoring, analysis and reporting will be done within your organization.  Will each stakeholder group be responsible for doing this themselves or will a centralized analytics and insights group be responsible?  In addition to the self-serve approach, you could consider outsourcing this work to your social listening platform vendor or to one of your agencies – PR, digital or advertising.  In my experience, it is easy for a company or organization to underestimate both the skill and time commitment necessary to make the self-serve approach effective.
  4. Content/Data Types – Social media listening platform vendors generally include content from the primary social media properties –  Facebook, Twitter, Blogs, Forums, YouTube and MySpace (being generous here).  Flickr is also included in many.  Currently on vendor roadmaps are properties like Linked-In and perhaps customer review sites.  Make sure the content types the platform supports meets your stakeholder requirements.  It is also very important to understand how the social content is being aggregated and how frequently (see Reporting for more on latency issues).  The fundamental ways in which content is aggregated in social listening platforms are crawling the web, RSS feeds and third-party content aggregators (e.g. Boardreader for Forums).  Many platform vendors employ a hybrid approach.
  5. Metrics and Analytics – Most social listening platforms either have a set group of analytics that deliver specific metrics or they offer configurable analytic ‘widgets’ that may be used to create metrics like share of conversation or volume and tone trend.  Some platforms offer a combination of these two approaches.  Based on your needs and measurement strategy/approach, define the analytics and metrics you would ideally like to see (e.g. volume, sentiment, messages, share-of-conversation, association with key topics).  In the vendor selection phase, this list will be useful to compare and contrast vendors.
  6. Keywords and Topics – During the planning phase, it is wise to develop a list of the major keywords and topics you believe will be necessary for the listening platform.  These keywords might include the company name, key competitors, industry issues, market segment names,  brand names, product names, key spokespersons, executives and competitor and industry spokespersons.  Social media listening platforms have varying degrees of sophistication with respect to their search capability.  Some have full Boolean logic, others offer very simple AND/OR logic.  The importance of this difference depends to some degree on you company/brand name as well as the sophistication of the people who will be configuring and maintaining your system.  If, for example, your company name is a common word (e.g. Apple, Visa), you will need stronger logic capabilities that include proximity search.
  7. Integration – Integration of varying data types – search, web, social, advertising, customer opinion and others – is the present and future of online measurement.  It is therefore important to understand what capabilities, if any, the social listening platform vendor has to integrate with other data types/streams.  Do they offer the ability to connect with web analytics packages via API for example?  The web/social integration is becoming increasingly common.  If you need to integrate traditional media with social, it might be a nice feature if the social listening platform allows third-party content aggregators like Factiva, Lexis Nexis, VMS or Critical Mention.
  8. Reporting – During the planning phase it is helpful to think through a series of questions about reports and reporting.  What type of reports are necessary?  Who will be responsible for their creation?  How often will reports be issued?  Does the system need the capability to automatically generate and deliver reports?  What about automated alerts?  There are quite a wide range of report capabilities represented by the various vendors in the listening space.  One potentially critical area to explore during the vendor evaluation phase is related to report frequency and perhaps to report type (think crisis).  That is how often new content is brought into the system.  Content latency issues may cause real problems during a fast-moving crisis.  Generally, the content latency differs by media type.  Best for Twitter and worst (perhaps) for forums, some of which restrict crawling to no more than once per day.  Within Twitter, the type of relationship the vendor has with Twitter should also be explored.  Not all Firehose arrangements are the same.  While most social media listening platforms claim to be ‘real time’, it is interesting to ask the vendors to define what they mean by ‘real time’.  The answers may surprise you.
  9. Access – Discuss who needs access to the listening platform and what they want to see and be able to do once they are in the system.  Do your different stakeholder groups (Divisions, product lines, brands, corporate, marketing, etc.) want or need a customized view of the data perhaps presented on a separate dashboard within the system?  It is also a good idea to have a perspective on who your power users will be versus the casual users.  This distinction applies not only to system access, but also in areas like training.
  10. Engagement – Some social media listening platforms support engagement with content owners directly from the platform, others do not.  Some engagement capabilities are elegant, others are rudimentary.  Make sure to explore the engagement needs of your stakeholders and understand how important this capability is to them in the short and long-term.  If engagement capabilities are important, you will also want to explore if the system allows users to tag content, assign content, manage assignments and track workflow.

In Part Two, we’ll examine a rigorous process for social media listening platform vendor evaluation and selection.

Don’t Let the Tool Tail Wag the Measurement Dog

19 Jul

Social media listening and measurement tools are sexy.  Well, at least to those of us in research and measurement – it’s all relative right?  In the last three years or so there has been an explosion of social media tool vendors and platform choices.  Tools are sexy and important, but in the grand scheme of things are being overemphasized to some degree.  We are letting tools decide what we can measure without giving sufficient thought to what we should measure.  We are letting the tool tail wag the measurement dog.

There are several steps and decisions that should be addressed prior to selecting a tool or suite of tools.  Consider this diagram as a starting point to help you think through these interim considerations and decisions:

OBJECTIVES

Proper social media objectives should be measurable (indicate change in metric of interest and timeframe) and aligned with desired organizational outcomes.  Understanding the social media objectives will suggest broad parameters the measurement program, and ultimately the tool decision, must operate within.  For example, geographic coverage requirements, type of content to be considered and on-platform engagement capability may all be strongly suggested based on a review of social media objectives.

PROCESS

In addition to comprehending organizational or business outcomes, it is essential to understand the business process the social media program will address or drive.  If the program is marketing oriented, the sales funnel process (Awareness/Consideration/Preference/Sales/Loyalty) may be most appropriate.  For a brand-building campaign, the brand pyramid (Presence/Relevance/Performance/Advantage/Bonding) is what you want to measure your program impact against.  Other business processes that are commonly addressed by social media programs include customer service and support, CRM, corporate reputation and lead generation.

METRICS

Understanding the requisite business process the social media program is driving is crucial because each business process drives specific metrics.  For example, the sales funnel drives a specific metrics set:  percentage of unaided or aided awareness; percentage of the target audience who would consider the product/company; percentage who prefer the product/company; incremental sales revenues; percentage who would purchase the product again number or the number/amount of repeat purchases.  For B2B companies, the lead generation process would drive a different set of metrics: number of incoming leads; percentage/number of qualified leads; lead conversion rate; sales revenues generated.  In addition to the business process metric sets, there are other metrics areas like Exposure and Engagement we will want to address.  Reach/opportunities to see, share of positive discussion, comments/post ratio, number of @ mentions and RTs per 1000 followers are examples of ‘standard’ metrics that might be applicable for many social media programs.

Understanding how the social media program drives a specific business process is also important to our ability to describe the impact or, in some cases, return on investment the program has created.

DATA SETS

Each metric has data requirements, usually two pieces of data per metric – a numerator and a denominator.  Examine the set of metrics you have defined for your social media program.  Catalog all the specific pieces of data you need to compute the various metrics.  For example, the data needed to compute the basic sales funnel metrics and some ‘standard’ metrics might include:

  • Number of individuals in the target audience
  • Number of survey respondents
  • Number of respondents ‘aware’ of the product/company
  • Number of respondents who would consider/seriously consider purchasing the product/doing business with the company
  • Number of respondents purchasing the product
  • Amount of sales revenue directly attributable to the program
  • Number of purchasers who purchased again
  • Total branded mentions
  • Volume of positive and negative mentions
  • Number of posts
  • Number of comments
  • Number of RTs and @ mentions
  • Number of followers

TOOLS

Armed with an understanding of all the data needed to calculate the metrics required to measure the social media program, you will be able to assess which tools or classes of tools best deliver the data you need.  Pick the best three to five tools for further evaluation.  You most likely will find no one tool can deliver the complete data set you need.  It is common to need two or more tools, e.g. web analytics package and social content analysis platform, in order to fully meet data requirements.  Budgetary constraints may also limit your ability to capture the entire data set required.

By addressing the interim steps leading up to tool selection, you will be able to make a more informed tool decision.  You also will have a much better chance of measuring what you should measure rather than settling for what you can measure.  No tool before its time.  Let the big dogs run.

Follow

Get every new post delivered to your Inbox.

Join 171 other followers