Tag Archives: social media analysis

Let’s Play 20 Questions: Social Media Measurement Style

1 Oct

On August 6, I gave a webinar for Carma, co-sponsored by PRNews called, Social Media Measurement at a Crossroads. The webinar focused on the current state of social media measurement with an emphasis on efforts to develop social media metrics standards. You may download the presentation courtesy of Carma here. There were many good questions asked by the webinar participants. I thought it might be fun to capture 20 of the questions and share the answers I gave in response. And it might be cool if you disagree with an answer, to share your different opinion in the comments.

Q1. What level of social media measurement do you think should be taught at Undergraduate level in PR or Communications degree courses?

A1. Most schools only require one research class in undergraduate education. In this class, all forms of research including measurement are covered. I think all schools should have one general research and analytics course and another specifically for measurement. I would cover traditional and digital in both courses with an emphasis on digital techniques.

Q2. What needs to happen for businesses to be able to integrate Communications Performance Management with Business Performance Management?

A2. Did Philip Sheldrake ask you to ask this question? Well, the first thing that would have to happen is for companies to start demanding it. I’ve not seen much demand for this. Once demand builds, smart people will figure out how to make it happen. The AMEC Social Media Measurement Committee is going to take on the challenge of developing a balanced scorecard approach to the social media valid framework to see where that takes us.

Q3. Speaking about social business, are you suggesting social media becomes the strategic imperative with marketing, customer service, PR, employee engagement subordinate?  So these functions will be driven by SM specialists?

A3. No, not at all. I think what we’ll see if that social media permeates all of these functions and creates new capabilities and connections between groups and between customers and companies. It is up to PR or HR people to learn something about social media, SM specialists are not going to take over the world.

Q4. Are the proposed standard social media metrics valid for native ads as well?

A4. I have not thought much about this, but my initial reaction is that the metrics for native ads would be same. A promoted tweet would have the same engagement metrics as any other tweet, although one would certainly hope the performance on some of the metrics would be better.

Q5. What do you mean when you say triage social media content for customer service and support?

A5. This would refer to evaluating and routing social content to different entities or people within an organization (customer care versus technical support versus legal, for example) that are best able to understand and act on the feedback and/or respond to the post.

Q6. Don, what do you put more emphasis on these days, Likes and Follows or Shares and Comments?

A6. I believe the emphasis should be on the stronger indications of engagement, shares and comments, than on simple Likes and Follows.

Q7. How well-known and widely accepted are the Conclave standards in the social space as a whole?

A7. The first complete set of standards were published in early June, 2013. They are known by social media measurement insiders, but I think it is fair to say they are not yet widely known. We need to promote their existence and use.

Q8. How would you measure perception and attitudes through social media?

A8. Generally we would measure consumer conversations about a topic and then do some analysis to see if there are clusters of comments that represent different and distinct viewpoints, attitudes or opinions about the issue or topic. We might also want to do an audience segmentation analysis to see how these attitudes differ by stakeholder group.

Q9. Any specific comments geared towards non-profit organizations?

A9. The basics of measurement – write measurable goals, align goals with organizational KPIs, assess performance against targets –  are the same for for-profit and not-for-profit organizations. How value is created is the primary difference.

Q10. Any suggestions to measure business impact for B2B organizations? Is there a way to understand the impact of social for B2B organizations?  

A10. Most B2B companies have a focus on sales leads. Therefore demonstrating how social is helping create leads or improve lead closure rates is important. There are a lot of uses of social listening in B2B companies as well – how the company is positioned on key issues, who is talking about the company, how products and services are being discussed, etc.

Q11. What are your favorite tools to use in terms of actually measuring your programs/channels/campaigns? Do you identify the tools as you are defining the metrics (do we have the ability to measure X, Y, Z?) or do you select tools after you define your metrics (this is what we need to know, let’s find A, B, C, solutions to measure these things?)?

A11. Generally Google Analytics, a social listening platform (Radian6, Brandwatch, Netbase, Visible, etc.), channel analytics programs like Facebook Insights and also Excel. Ideally you should define metrics first, then the data required for each metric, then look at the tools best able to get the specific data you need.

Q12. What are the most common or most surprising questions you have gotten from CMOs or other key stakeholders regarding social media measurement?

A12. CMOs want to know how social media contributes value to marketing – if they are sales funnel oriented they want to know how social is helping drive the funnel for example. They are also interested if you are helping on front-end or downstream funnel metrics.

Q13. What advice do you have for small businesses for use of and measuring success of social media campaigns effectively (few resources).

A13. Start with the free tools (Hootsuite, Excel, Facebook Insights, Twitter Analytics, Google Analytics) and then work your way up to some of the paid social listening platforms. There is no ‘best’ platform to start with – it really depends on your needs and what you intend to do with the platform. Many companies start with measuring their own channels and evolve to listening to earned/ shared social conversations.

Q14. Which social media analytics do the C-suite find most valuable?

A14. The C-suite don’t really care about social media analytics so much, They care about how social media is helping drive the business metrics forward. That said, C-level folks are usually interesting in competitive benchmarking in social and positioning on key issues and topics that are important to the business. Anything pertaining to online reputation is also an area of interest for many.

Q15. How do you determine what are the correct things to measure?

A15. Measure what matters to the organization. Measurement is about performance against objectives so make sure your measurement program is aligned with business objectives. Don’t measurement ‘social media’, measure what you are trying to accomplish with social media.

Q16. How can someone who is interested in the movement toward standard metrics get involved helping to move the PR industry forward? In other words, how can someone get involved in the debate?

A16. I would suggest interacting directly on the smmstandards.org website. Volunteer to help. Leave suggestions. You could also get involved through one of the PR associations – IPR, PRSA or CoPRF.

Q17. What software would you recommend be used by PR firms to most cost effectively measure social media efforts for clients?

A17. A good social listening platform, Google Analytics, Facebook Insights and the other packages offered by the channels, and good old Excel. Beyond that it really depends on the nature of the social media effort.

Q18. I think a lot of the issue with measurement is confidence in the measurer (i.e., your source). Whenever you cross-reference measurements (e.g. what Google analytics says vs. what your web marketing automation says like HubSpot), you can get wildly different answers. That has stopped me from putting too much faith in my metrics process. Thoughts?

A18. I might separate the issue of the measurer from the sources of data – really two different issues. Regarding sources of data, this is a true issue in that different databases yield different estimates for things like audience size. Compete versus ComScore is a notorious example. However, I don’t think this is a reason to not measure. It simply means we must state assumptions and sources and be consistent over time in using comparable sources. I believe that standard metrics will eventually lead to sanctioned sources for audience data like Arbitron (now Nielsen Audio) for radio or Nielsen for TV.

Q19. Let’s say a social media post leads someone to a landing page, but they do not take immediate action. But they come back the next week and complete the conversion funnel. How do you credit the original social media post…is this a matter of tracking cookies for x number of days? What is practical?

A19. Yep, most people count the first click and then track for a period of time depending on the type of product. It gets even more complicated if you try to suggest there should also be credit given to what happened before the first social media click – for example, money invested in building the brand. Value attribution is an inexact science for sure, with lots of assumptions and compromises.

Q20. What are best ways to measure target audience reach and engagement rather than wide general reach?

A20. Thanks for asking this. The best way to measure is to clearly define your target. If the target is Females 18 – 34, then you should only take credit for reach and engagement of this specific audience only. Given that most tools rely on voluntary bio data, the information is inconsistent and difficult to come by.

Thanks for reading. @Donbart

Social Media Measurement at a Crossroads

21 Aug

We are at a crossroads in social media measurement. Expectations for rigorous and relevant measurement have risen more quickly than delivery. Too many are fixated on quantitative outputs – speeds and feeds – at the expense of understanding the outcomes achieved by social media marketing and social business. There is still too much emphasis on vanity metrics and not enough on business results. And, if you take a step back, there is simply too much talk about all this and not enough action. At the risk of exacerbating the last point, let me explain.

 

Social Media Measurement Started with the Wrong Orientation

In the late 1990s and early 2000s, digital measurement focused on website analytics. The orientation was heavily quantitative. How many unique visitors? How many page views? How long did people remain on site? By 2007, with Facebook now three years old and Twitter completing it’s inaugural year, social media measurement was becoming a hot topic.

Crossroads1Early social media measurement practitioners generally came from the web analytics world. Early social media measurement efforts focused on quantifying outputs and not addressing the outcome of the program. The orientation was on ‘How Many?’ and not ‘What Happened?’ The quantitative orientation also came at the expense of qualitative assessment. The emphasis was on getting easily accessible statistics and not on content analysis to understand meaning and implications. These issues remain today, although we have made significant progress toward shifting the orientation to outcomes and business results.

In the early adopter phase of social media, social media measurement was under little pressure to go beyond quantitative output analysis. Many brands, companies and organizations viewed social media participation as a bit of an experiment to see how it best could be used within their organizations. But this was soon to change.

Struggle Between Easy/Superficial and Hard/Meaningful

It is difficult to pinpoint when social media crossed the chasm into a mainstream business activity. An IDC study in the Fall of 2009 suggested the state of social media still best fit the early adopter and not mainstream use pattern at that point in time. 2011 felt like the year the leap happened to me. With it came a new and emerging set of expectations around social media measurement.

Crossroads2In measurement, it is a truism that the metrics that are easiest to measure are seldom the ones that are most meaningful. It may be easy to measure outputs, but it is often much more difficult and expensive to measure outcomes. It is much easier to determine brand mentions in social media than it is to assess whether or not social programming has changed opinions and attitudes of the target.  It is infinitely easier to measure unique visitors per month than it is to determine the return on investment of a social media initiative.

Now that social media clearly is a mainstream business activity, the pressure to demonstrate the impact and value of social media has greatly increased. As the resources and investment against social media and social business become meaningful line items in the budget, the game changes. Demonstrating business impact and value requires an understanding of the business model of the company or organization and how social media/business creates impact (e.g. change in awareness, increase in purchase consideration, increase in active advocates around an issue) in that environment. Measuring impact is more difficult than measuring audience or engagement. It often involves primary audience research so the price tag is higher.

This is a key struggle we face – will we continue to take the easy, less expensive, minimal-value-of-the-findings approach or we will take social media measurement to another level, focusing on outcomes, investing in audience research and applying rigorous analytics to get at meaning and insight? The imperative is clear, how we respond will be telling. 

A Final Turn to the Right

One of the key themes at this year’s AMEC measurement conference in Madrid was creating a bias toward action. The time to (just) talk about measurement is in the past, the time for action is now. I might suggest this goes double for social media measurement. Here are three areas we can address that will help make the leap from talk to action.Crossroads3

  1. Every social media initiative has a measurement plan. Let’s make this happen. Literally any social media initiative, program or activity should have a measurement plan defined before implementation begins. Start with writing social media objectives that are measurable. Align social media metrics with business KPIs. Select metrics across multiple dimensions – programmatic, channel-specific and business-level metrics, for example. Or perhaps paid, owned, earned and shared metrics if your program is integrated across these dimensions. Collect data. Assess performance against objectives. Rinse and repeat
  2. Take a stand on standards. An exciting cross-industry effort has produced a set of proposed standards for social media metrics. Adopting standard definitions and metrics for social media is an important stage of measurement maturity that other marketing disciplines like advertising and direct marketing have already reached.
  3. Understand, articulate & demonstrate business impact.  The heat is on to demonstrate how social media is helping drive the business or organization forward. We must do a better job of connecting the dots between business KPIs, social media objectives and social media metrics and measurement. In some cases, we want to go beyond understanding attitudinal and behavioral changes to understand the financial value of the impact created. Capturing the financial value of social media requires expertise, data, time and money. We would always like to measure impact, and when it makes sense, we may push further to attribute financial value.

It will be interesting to see what the next year in social media measurement brings. The move toward standardization alone should be fascinating to watch. I have tried to make the argument we are at a crossroads or inflection point in social media measurement maturity. What ‘worked’ for us in the past will not work in the future. We know the expectations. The great unknown is how we respond.

Note: This post was inspired by a Carma webinar,co-sponsored by PRNews, I gave recently. You may download slides from that webinar here.

Social Media Measurement at a Crossroads

25 Jul

Please join me on August 6, from 2:00 – 3:00pm EDT for CARMA’s fourth quarterly webinar, co-sponsored by PRNews, titled “Social Media Measurement at a Crossroads.” Here is a little more information on the webinar:

With social media clearly entrenched as a mainstream business activity, the need to measure the impact on the organization has never been greater. While social media practitioners talk about Like or Follower growth, organizations want to understand how social media is helping drive the business or cause forward. 

Another challenge in social media measurement is the lack of standard definitions, approaches and metrics. In response, a cross-industry push to define social media standards was initiated and initial standards recently published. Social media measurement is clearly at a crossroads where new thinking and approaches are emerging.

In this session you will learn:

  • How to align KPIs and metrics to demonstrate organizational impact and value. 
  • What industry efforts are being made toward standardization and the implications for how you approach social media measurement
  • New models, metrics and frameworks you can use today to develop more effective social media measurement programs.

I will be joined by PRNews Group Editor Matthew Schwartz, who will moderate the discussion and lead a Q&A session.

Here is a link to register for the session. Feel free to leave a comment with any questions you would like answered during the webinar and I will do my best to address them. Hope you can join!

Three Keys to Insight Discovery in Social Listening

13 Dec

True social insights, as opposed to social findings or social observations, have the potential to inform, shape or drive marketing and even business strategy decisions, not just social strategy decisions. Discovering that tweeting with a link on Tuesday between 10:00 – 11:00 AM drives higher levels of engagement is a social finding, not an insight.

Social media is a microcosm of the larger Big Data problem/opportunity – too much data, not enough insights. Or if you prefer, too much noise, not enough signal. If you want to improve your ability to discover insights, here are three simple approaches you can take to improve your insight hunting.

First, start all analysis with a hypothesis or series of questions the analysis is designed to answer. It is much easier to prove or disprove a hypothesis, or answer specific questions, than it is to “find out what people are saying about us in social media”. The more specific the request, the better the answer is going to be. The hypothesis may be one you develop based on preliminary analysis or it may come from the ‘customer’ for the insight. Here are two examples of hypothesis:

  • “Conversation about us in social media is quite negative. My boss believes ‘everyone’ is aligned against us. I disagree. My hypothesis is that there is a very vocal and active minority of consumers who are posting large volumes of negative content about us. And I believe this group is a small fraction of the total number of people who post. The majority of consumers are actually neutral toward us.”
  • “When we look at Twitter, Facebook and Blogs we see pretty low levels of conversation about our product and the medical condition it treats. We believe there is actually a fair amount of conversation, but the conversations are occurring in Forums which may not be crawled by most social listening tools.”

While the hypothesis is a great way to begin to focus on what is important in the data, a further focusing mechanism is the second insight discovery key – the concept of targeted listening. With targeted listening we are not trying to capture all conversations that mention the brand or product. That is a ‘boil the ocean’ approach. Instead, we listen for very specific types of conversations or conversations by very specific groups of individuals within social conversations. The trick is to have the discipline to only listen to your focus areas and not be tempted to boil the ocean in hopes of finding a few pearls. Here are three examples of targeted listening strategies:

  • An insurance company resists the temptation to try to capture ‘all’ mentions of the brand and decides to focus only on conversations where customers are thinking about non-renewal or switching companies.
  • A gaming company launches a new product and listens to understand what features are being discussed, what people like most/least about the new game and to gauge their specific reactions to the cover art.
  • A consumer products company listens only for consumers who are actively in the purchasing process for the type of products they offer.

The third key to discovering insights is to provide context for decision-making. Remember with insights we are trying to inform, shape and guide decision-making. Context is incredibly important to making better decisions faster. Good social analysts understand how marketing and business work and how strategic alternatives might impact results. Understanding this helps you put your insights in the proper context for decision-making.

Here is an example of how context can lead to better decisions. Company X has a crisis. You are asked to do real-time listening of the crisis and help the PR team decide when and how to engage in the conversation. You come back the next day with a line chart showing a large spike in content mentioning the crisis – thousands of mentions. You know the sentiment in negative to neutral and on which channels the content appears. Unfortunately you have not given the people deciding Trend.BlogPost

whether or not to engage enough information to make a decision. What information would provide the necessary context for decision-making? What questions do we need to try to answer? Here are a few:

  • How much above ‘normal levels’ is the spike in content? (Normative data)
  • How does this event compare to that event we had last year? Or, how does the event compare to competitor X who had their own crisis last year? (Comparative data) Comparisons help decision makers determine ‘how bad is bad’.
  • How long do we anticipate seeing negative content at relatively high levels? (Comparative data) This might be the most important question to answer to provide context and guidance for the engagement decision. If we anticipate volume will drop back to normal in a reasonable period of time, then not engaging may be a viable and effective strategy depending on the brand involved and the nature of the crisis.
  • Which stakeholder groups are active in the conversations?  With robust social analysis we always want to look at both the post – what is being said, and the source – who is saying it. In a crisis, who is talking is particularly important.

Normative data, comparative data and examining both post and source data are all effective techniques to provide context for decision-making. InsightButton

The tough part about discovering insights is there are no shortcuts and it is a human activity. No social media analysis platform that I have found has an insight button. The key barrier is lack of people who understand how to search for and discover insights. Hope these tips make you a more effective explorer.

Happy Holidays!

Social Media Listening Platforms – Plan, Select, Deploy (Part Two – Select)

2 Jun

In Part One, we discussed a range of topics designed to help you plan and define the scope and requirements for selecting and deploying a social media listening platform across your company or organization.  In Part Two, we will use the knowledge and perspective we gained in planning to orchestrate a thorough and effective platform selection process.

Here is a scalable selection process that will help you surface and select the social media listening platform that best meets your unique situation and requirements.

1. Define the individuals who will be involved in the selection process – Inclusion is a powerful card to play here.  Inclusion brings different perspectives together.  Inclusion greatly improves chances for success when it is time to authorize purchase of a platform and get it deployed properly across the organization or company.  Inclusion will increase the likelihood of acceptance and use of the platform across the organization.  Include representatives from the major stakeholder groups identified during the planning process.  You might include someone from your IT department.  You might also include the individuals who must authorize the purchase.  A group of up to ten is most workable.  After ten or so, I believe you will most likely experience diminishing returns on the incremental people added to the process.

2. Develop a list of selection criteria organized by major category – Based on the planning process we undertook in Part One, develop a list of categories that are most important to learn more about.  Here are ten categories you might consider including:

  • Content Sources/Types & Aggregation Strategy – What types of social content are brought into the system?  How is the content aggregated (e.g. RSS, crawling, third-party aggregators)?  How often is each type of content aggregated?  
  • Data and Search Considerations – How long is content archived, and is back data available?  What data cleansing strategies are in place to address spam, splogs and duplicate content?  Is full Boolean logic available for constructing searches?
  • Metrics and Analytics – What specific metrics are ‘standard’ in the system?  Is automated sentiment analysis offered at the brand or post level?  What audience-level data is available?
  • Data Presentation  – What dashboard features and functionality are offered?  Can dashboards be customized by user or group?  Are drill-down capabilities available for all analytics on the dashboard?    
  • Engagement and Workflow Functionality – Does the platform offer the ability to engage directly with content owners?  Can ‘owned’ content be managed on-platform?  What workflow management and reporting capabilities are offered?
  • Integration – What additional types of data may be integrated in the system – traditional media, web analytics, email, call center, CRM, etc?
  • Reporting Capability – Does the platform have a report function?  Can reports be customized?  Automated?
  • Geographic Scope – What countries and languages are addressed by the system?  Are two-byte languages supported?
  • Cost Structure – What is the cost basis – seat charge, subscription, content volume and/or number of searches?  How does pricing vary with increases in the cost basis?
  • Value-added Services – Does the listening platform vendor offer system configuration services?  Do they perform analysis and reporting?

Within each major category, list the specific criteria most relevant and important to your requirements.  For example, within the Data and Search Considerations category, you might list ten specific criteria that you want to assess for each vendor:

  • How often is Twitter data refreshed?  Can refresh timing be specified?
  • How often is new content from other sources crawled/brought into the system?
  • How long can each content type be archived?
  • Is back data available?  How far back and at what cost?
  • What data cleansing strategies are in place?
  • Can data be easily exported in CSV/Excel format and is bulk data extraction supported?
  • Can users build and customize topics and searches?
  • What types of Boolean operators are supported?
  • Is proximity search supported?
  • Do users have the ability to date-range data for analysis?

3. Develop a scorecard to use in evaluating the potential listening platform vendors/partners – Using the major categories and specific criteria you have defined, develop an overall scorecard to be used in the evaluation process.  Think about creating a weighting system at the category level to help prioritize the importance of each category.  Assign a number of points to each criteria within a given category.  A scorecard might contain ten categories each containing ten criteria.  Begin by assigning a one-point value to each criteria (100 points total) and then apply weighting at the category level.

4. Develop the initial vendor consideration set – List all the social media platform vendors you wish to consider.  Pick ones you are familiar with and have positive experiences with as a starting point.  Talk to colleagues within, and experts outside, the organization to gain their perspective on the platforms that should be considered.  Read blog posts and reviews of the platforms to gain additional outside perspective.  Visit vendor websites and watch demo videos.  Pull it all together and gain consensus amongst your team on the platforms that will be considered.

5. Do some homework and narrow the list to a manageable number (perhaps five to ten)- If your initial vendor consideration set is too large (if it has more than ten vendors it is too large), do some additional homework and narrow your list to a more manageable number.

6. Develop and distribute an RFI based on evaluation criteria – Using the categories and criteria you developed, create a request for proposal, asking the listening platform vendors the questions that are most critical to meeting your requirements.  Specify the format (e.g. PowerPoint, Word) you would like responses to take.  Give the vendors about two weeks to respond.

7. Evaluate and score vendor responses – Once the RFI documents are received, each should be reviewed carefully and scored according to the criteria and weighting decided previously.  Depending on the number of vendors being evaluated and ease of getting the entire evaluation team together, there may be merit in blocking out an afternoon to gather as a group, read through the responses, and decide how each will be scored.  This is a bit of a ‘pulling off the band-aid’  approach that will save time and allow for spirited discussion and consensus scoring.  If this is impractical for whatever reason in your company or organization, assign one of more RFIs to individuals who will then develop the scorecards.  The scorecards may then be reviewed together in a meeting or conference call, and consensus reached on scoring.  Obviously the potential issue with multiple people independently creating scorecards is consistency.  You want the evaluation to be as fair and consistent as possible given whatever constraints you are working under.

8. Develop a short list of vendors – If your number of vendors under consideration is over five, use the scorecards to reduce the list to three to five platforms that will undergo further evaluation.  These are your finalists.  You should always promptly notify vendors not moving forward in the process, and offer to provide feedback via phone or email on why they were not selected to move forward.  This professionalism will be much appreciated by the vendors, and represents a good learning opportunity for all involved if done well.

9. Deploy test scenarios – At this point we have narrowed the list of contenders and are ready to proceed with some specific tests designed to illuminate the real-world capabilities of the platforms.  Here are three possible test scenarios.  You can use all three for a very rigorous evaluation, or just one or two if that fits your needs better.

  • Test scenario 1: Give each vendor a defined list of search terms (brands, competitors, issues) and the languages/countries you want to evaluate.   You should use search terms that are directly relevant to your company or organization.  Explain what type of analysis you would like performed and ask each to address insight generation.  Platform vendors are given one week to prepare an analysis.  If practical, you could ask each vendor to give a presentation of the results in person.  Alternatively, use a web conference to review the results.
  • Test scenario 2: This is a real-time exercise designed to assess vendor data volume by country/language and signal-to-noise ratio of relevant content.  Get on a web conference with each social media listening platform vendor.  Give them a new list of three search terms and ask that they go into their platform, configure the system for the three search terms and then pull in relevant content for the past 30 days.  Once that is accomplished, ask them to export the data as an CSV or Excel file and email you the results while everyone is still on the line.  A more detailed off-line review of the results should be undertaken, including translation of languages, to assess relevancy of the results.
  • Test scenario 3: This has been referred to by a colleague as the Dr. Evil test…In conjunction with test scenario two, it may be interesting to ‘plant’ known content that matches the search terms on different Twitter channels, Facebook pages and Forums in each country that is of interest to you.  When you receive your data export, examine to determine if the known content was found.

10. Pick a winner – At this point you have the RFIs, scorecards and test results.  You are ready to make your decision.  Convene the evaluation team, discuss the results and make a decision.  With luck, a clear winner will have emerged from the process.  Contact the winner and negotiate terms of a contract.  Don’t notify the non-winners until after a contract is in place, just in case you need to move to your second choice for whatever reason.

In Part Three, we will discuss how to maximize your potential for success when actually deploying the social media listening platform across your organization.

10 Important Considerations When Selecting A Social Media Measurement Vendor

1 Jul

The majority of conversation today within the measurement community pertains to social media* measurement.  And the number of firms offering tracking, monitoring or social media measurement solutions is exploding, doubling or more from 2007 to 2008.  So, how do you select a firm that best meets your needs or the needs of your clients?  Here are ten items to consider as you start down that path. 

 

Underlying Philosophy & Capabilities – There are many different approaches and underlying philosophies driving the various social media measurement vendors.  Some of the approaches translate directly into unique capabilities.  For example, CoreX, a company focused on defensive monitoring, is driven by a belief that participating in a discussion may unintentionally influence its outcome.  Therefore your identity is hidden when you use the tool.  Compete tracks a consumer’s online behavior/clicks while most other vendors track consumer artifacts like comments or posts.   Kaava focuses on communities and not blogs and believes they better represent consumer opinion.  Pick a partner with a philosophy that is compatible with your own views and needs.

Metrics and Analytics – Give some thought to the specific metrics you want to track and the analytics that will give you the best data upon which to make decisions.  While most firms track basic metrics like volume of posts or comments and number of links, more complex or algorithmic metrics vary widely.  There are many different interpretations of Influence for example.  Also evaluate the range of analytics that are available. If links analysis or social network analysis is important to you, some social media vendors offer these while others do not. The good news is the analytics and visualization tools available are rapidly becoming more sophisticated and useful.

Content – It is important to understand how content is brought into the social media measurement site, and how many different sources are looked at in order to find relevant content.  Some sites claim 10 million sources while others claim 30 million.  You also want to ensure the sites trolled are the ‘right’ sites for you or your clients.  Develop a list of your 100 most important sites and ask the prospective vendor to verify whether or not each of the 100 is included in their universe of content.  Also note which categories of content are tracked – discussion groups, blogs, review sites, traditional online media outlets, etc.      

Language Support – Are you most interested in the United States, Europe or perhaps China?  How you answer may well lead you to consider an entirely different set of vendor alternatives.  If Europe is most important to you, you might consider a European specialist like London-based Attentio, or perhaps Onalytica.  If you are most interested in social media conversations in China, you might consider a vendor like CIC that offers a deep understanding of Chinese pop culture and works in Shanghainese as well as Mandarin and Cantonese.  Just offering language translation is vastly easier than understanding cultural nuances of language usage within a given country.  If your requirements are global, either make sure your analysis partner supports the languages that are important to you, or select two or three regional partners that collectively can support your global requirements.

Real-time Orientation – Do you need information for crisis or issues monitoring or perhaps investing (Collective Intellect)?  If so, you will need a vendor that provides near real-time updates of information.  The search orientation model offered by many vendors is more akin to batch processing.  You define terms you are interested in and then bots go and gather the information for your review and analysis.  If crisis or issues monitoring is your primary requirement, make sure your vendor also offers capabilities like email or phone alerts.    

SaaS or Consultative Models – Do you want a platform to do your own social media monitoring or do you want a firm to do the research and prepare a monthly report for you?  Hands-on or hands-off?  Some vendors (BuzzLogic, NetMap Analytics, Radian 6 and Visible Technologies to name a few), offer a software-as-a-service model while, at the other end of the spectrum, others (MotiveQuest) are very oriented toward consulting – helping to answer the “so what should we do about it” question.  Most of the firms offering a SaaS use a dashboard as their main interface.  If creating dashboards for reports and/or having a dashboard interface is important to you, make sure to ask whether or not this capability exists and have it demonstrated to you to assess the overall ease-of-use and utility.

Automated or Human Analysis – One of the ongoing controversies in the measurement field is what can be successfully automated in content analysis and what must or should be left to human analysis.  A few vendors, Umbria for example, are fully automated, even for sentiment analysis.  The majority of social media measurement vendors employ a hybrid approach, with simple items like post or comment counts and number of links fully automated, but sentiment analysis left to human analysis.  Some vendors perform the sentiment analysis while others leave the user to define sentiment for themselves.   

Ability to Directly Engage with Consumers – Some monitoring and analysis vendors allow you to directly engage in consumer discussion without leaving their platform.  Visible Technologies and Digital Influence Group are two examples.  This may be advantageous in situations where you are attempting to ‘Listen and Engage’, rather than just Measure online conversations.

Customizability – As you look at specific vendors and the tools and metrics they offer, you would be wise to ask whether or not any of the approaches, content sources, metrics or algorithms can be altered to better meet your needs.  For example, if specific social media sites are critical to you, make sure they are being included in the content population.  Filtering capability allows you to exclude sources you may not want to consider.  Some tools, like Radian 6 for example, allow the user to customize (via a graphic equalizer-like interface), the weighting given to specific elements when calculating Influence.  You can put more of less emphasis on elements like Number of On-topic Posts or Number of Links to better fit your own definition of online Influence.

Cost – Costs vary greatly, driven by many of the factors above.  Obviously human analysis and consulting are major cost drivers.  Tools range from free (always a strong price point!) to one to two thousand dollars a month, to $100,000 or more per year.  Decide what you can reasonably spend before you do too much vendor analysis.  For example, there is no sense in looking at vendors oriented toward consulting if you have only $1000 per month to spend.

 

There are many additional factors you may want to consider when comparing firms.  Number of years in business, clients/customers they do business with, and their ownership structure to name a few.  I hope you will find the ten considerations presented useful as you try to find the best social media analysis for you.  Happy hunting!

 

For more complete information to help you select the right social media measurement company, please visit Social Target.  Founder Nathan Gilliatt produces the industry’s best reference guide of social media analysis vendor information.  His reference guide has informed much of what you see above.  His 2008 reference guide will be available soon. You can order it here.

*I am using ‘social media’ broadly to include blogs, online discussion groups, forums, review sites, CGM sites and social networking sites

Disclaimer – My employer’s parent company, IPG, has a non-exclusive agreement with Radian 6.   

Follow

Get every new post delivered to your Inbox.

Join 169 other followers