Social Media Metrics & Measurement Continue to Evolve

This week on 11/12 June, AMEC is holding their next International Summit on Measurement. Many of you will recognize AMEC as the framer of the Barcelona Principles at their annual meeting in 2010. The theme this year is upping the game to deliver relevant insights along with traditional measurement reporting on performance against objectives and KPIs, in order to provide a richer environment and context for making strategic business decisions.

 

 

In this post, I wanted to shine the light on a workshop during the Summit called, Metrics that matter: Making sense of social media measurement. The session, led by Richard Bagnall, promises to look at the latest social media measurement trends, provide a look ahead at what might be right around the corner in the next 12 months, and unveil a revised and enhanced AMEC social media measurement framework guide that should make it easier to implement the frameworks in your own planning environments.

At last year’s Summit in Madrid, we presented the initial work in developing models and frameworks to support rigorous and valid social media measurement. It consisted primarily of three elements – a new model for social media measurement, a couple of alternatives ways to think about populating the model with relevant metrics, and a social media measurement planning framework/template. The initial metrics approaches focused on two alternatives, metrics focused across programmatic, channel and business dimensions as well as an approach based on the Paid, Earned, Shared and Owned integrated channel metrics approach which enjoys traction in many organizations. Based on using and customizing the approach for multiple clients, here are a few things to consider as you review and think about using the new frameworks and usage guidance to be unveiled this week.

As campaigns become increasingly integrated across media types it makes sense to also reflect this integrated view in measurement. Ideally, measurement should reflect a similar level of integration as the campaign or program being measured. As we measure performance from the four channels, we should keep in mind what we really would like to understand is how the channel efforts amplify or build on each other.

The work presented at last year’s Summit showed example or illustrative metrics for each of the media types across the measurement model. Expect this year’s version to take a stronger view of the most relevant or best metrics to use when employing this approach.

Another key development would be the presentation of different metrics models to use with the measurement model. The intention all along was there could be a range of choices to fit different corporate cultures and planning environments.

 

Measurement Planning Template

The measurement planning template is the heart of your social media measurement planning effort. It is best used by conducting a facilitated discussion with all measurement stakeholders around each of the key elements in the planning template. Setting aside half-a-day to complete the exercise is not excessive. Here is how we think about using the template in our social media measurement planning.

Business or Organizational Objectives: The agreed upon overarching organizational objective(s) the social media effort is designed to impact. These objectives may be given to the social media team or they may be the result of conversations and negotiations.

KPIs (Key Performance Indicators): One or two high-level metrics that are aligned with the business/organizational objectives and are outcome-oriented. Again, the KPIs may be given to the measurement team or the measurement team can help guide the conversation to develop them.

Program Elements: Outline the major elements of the program. The elements should reflect the scope and integration of the campaign. They could range from a simple social media program to one that includes social media, online advertising, influencer outreach, e-mail marketing and other elements.

Program Objectives: Capture or write the measurable objective associated with each major aspect of the program above. Frequently you will need to help rewrite the metrics to make them measurable. You may also need to rewrite them to actually make them objectives (what) and not strategies (how).

Measurement Story: This is an attempt to marry the concepts of measurement and story telling. At the end of the program, what measurement story would you like to be able to tell your key stakeholders? This should be based on accomplishing your specific KPIs through the success of the programs used and your ability to prove that through data and measurement. We typically develop the measurement story right after settling on the business objectives and KPIs, and use it as a guidepost to ensure we have the means and methods to tell the desired story.

Key Metrics: The metrics directly tied to program objectives. Should be aligned with the objectives as well as the higher order KPIs. The most important KPIs and metrics and often captured on a dashboard for monitoring and/or reporting.

 

Determining the importance of metrics

A question sometimes comes up about the best way to determine the most important metrics – what rises to the level of inclusion on a high-level dashboard? Here’s a few tips to consider:

Place metrics in their respective place in the measurement model (Exposure/Engagement/Influence/Impact/Advocacy) Metrics that appear toward the right side of the model are generally more compelling than those addressing just Exposure or Engagement. These generally get to the outcomes rather than just outputs of program activities.

Examine how closely aligned each metric is with program objectives. Metrics that directly support the objective are generally more important than those that indirectly support the objective.

Look at the degree to which the metric is explicitly part of the Measurement Story. Metrics directly aligned with the Measurement Story represent better potential dashboard metrics than those that are tangential to the story.

Metrics that provide context are generally stronger than those that don’t. For example, RTs per 1000 Followers tells you much more than just the Number of RTs – it gives an indication of community engagement. Likewise, Engagement Rate (Number of Engagements divided by Total Reach) is a better metric that just the Number of Engagements.

Remember the audience – at the end of the day, when you’re showing your stakeholders what you accomplished this year, what would they be most excited to hear? What will they want to see that ultimately demonstrates the best use of their dollars?

 

After the Summit I’ll post a reaction to Professor Jim Macnamara’s unveiling of his new his new paradigm and model for measurement and evaluation. Jim is incredibly smart and thoughtful so I’ll be curious to see and discuss what he proposes. Enjoy the Summit in person or through social media and contribute to the dialogue.

Thanks for reading.

 

 

I would like to acknowledge our colleagues at R/GA for their contributions to the thinking you see here.

Five Social Media Measurement Questions I Hope (NOT) To See in 2014

I get asked lots of great questions about social media measurement. Following are five not so great ones I hope not to hear in 2014. 

How do you measure social media?

I get this question quite often and I enjoy it each time because if provides me the opportunity to make an important point about measurement and be a little snarky at the same time. Good stuff! When I get this question, my answer is always the same; “I don’t measure ‘social media’, I measure what you are trying to accomplish with social media.” This may seem like I’m playing semantic games, but the distinction is very important. Measurement is fundamentally about performance against objectives. So, we measure our performance against the objectives established in the social media plan. A lot of what passes for measurement in social media is really data collection – tracking Followers or Likes, blog traffic or consumer engagement on Facebook. Unless you have measurable objectives and targets in each of these areas, you are collecting data not measuring. What do you want to happen as a result of your social media campaign or initiative? Measure that.

How much is a Like worth?

This question doesn’t come up quite as often as in 2012, but it is still asked and, unfortunately, answered largely based on flawed logic and/or research design. You may recall the first two ‘research’ studies attempting to answer this question came up with widely disparate values – somewhere around $3.14 in one case and over 100 dollars in the other. This alone should raise major red flags. Setting the flawed research aside, trying to assign a value to a Like happens because people are desperate to assign financial value to social media and determine ROI. Those are noble things to do, but we need to focus on the other end of the customer journey – have we created engagement, has the engagement changed opinions, attitudes, beliefs or behavior, and how those changes translate to Impact. Unless you understand the Impact created by your social media program you really can’t attribute value properly. I would argue that Likes, which can be bought or gamified, really have no inherent value.

Can I use a banner ad cost to calculate social media AVE?

This question is somewhat related to the ‘Like worth’ question in that it reflects a desire to quickly and easily assign financial value, when in fact assigning financial value is often hard and expensive. In this case, the questioner is attempting to take the highly flawed and discredited concept of Advertising Value Equivalency (AVE) and apply it to social media. Where this question typically comes up is in blogger relations where a company/brand/organization has worked with a blogger to earn ‘coverage’ on the blog and wants to assign a financial value to the post. They would like to say that the post is worth X, with X being the cost of a banner ad on the blog (setting aside, of course, that many blogs do not accept advertising). Equating cost with value is comparing apples to oranges. First, a better practice is not to assign value to each post, but to all the posts together in a campaign. Then instead of trying to say the campaign is worth say 14.35 ads, try to explain the actual impact the campaign has created on the target audience – e.g. increase in awareness, increase in purchase intent or, higher propensity to purchase more often. Once you understand the impact, decide if you have the data, time, expertise and budget to assign financial impact to the impact created.

Which social media listening tool do you recommend?

The correct answer to this question is, “it depends.” This is a bad question simply because there is no one ‘best’ social media listening tool for all circumstances and use cases. I believe you should always develop a set of platform requirements driven by the social listening stakeholders in your organization. Once these needs and requirements are understood, develop a custom RFI designed around the specific requirements you have identified. Have each of your potential platform partners respond to the RFI. Have the best respondents given you a platform demonstration according to a custom demonstration script you have developed. Pick the listening tool that best meets your unique requirements. The last three evaluations I have conducted for clients resulted in three different ‘winners’. There is no ‘best social listening tool, so find the tool that meets your requirements the best.

How many Impressions did we get with our latest social media campaign?

This is not a terrible question at all unless it is the only question asked or is perceived to be the key metric for measuring social media campaign performance. Too often, organizations use Impressions as their primary social media metric instead of engagement, influence or action-oriented metrics. Also, keep in mind Impressions represent an opportunity to see content, they are not the actual number of people who saw the content, that number is MUCH lower. Impressions always overstate the actual number of people who were exposed to your content and message.

If you plan to report on campaign impressions, please seriously consider only taking credit for those impressions that are directly against your target audience. If your target is 25 – 34 year old Males, you should only report on the impressions against this target group. Why take credit for 45 – 60 year old Female impressions when the product is not at all relevant to this audience?  Target audience impressions are really what you should be concerned about and what you should be reporting. Many people know and understand this but still persist in reporting all impressions because the number is usually much larger – meaningless but larger.

If you do report on Impressions, please consider using the emerging industry standard definitions developed by The Coalition. This will help ensure we define Impressions consistently and don’t confuse Reach with Impressions.

See things differently? Have your own pet peeve social media measurement questions to share? As always, thanks for reading. All the best in 2014!

@Donbart

Image credit: amasterpics123 / 123RF Stock Photo

Let’s Play 20 Questions: Social Media Measurement Style

On August 6, I gave a webinar for Carma, co-sponsored by PRNews called, Social Media Measurement at a Crossroads. The webinar focused on the current state of social media measurement with an emphasis on efforts to develop social media metrics standards. You may download the presentation courtesy of Carma here. There were many good questions asked by the webinar participants. I thought it might be fun to capture 20 of the questions and share the answers I gave in response. And it might be cool if you disagree with an answer, to share your different opinion in the comments.

Q1. What level of social media measurement do you think should be taught at Undergraduate level in PR or Communications degree courses?

A1. Most schools only require one research class in undergraduate education. In this class, all forms of research including measurement are covered. I think all schools should have one general research and analytics course and another specifically for measurement. I would cover traditional and digital in both courses with an emphasis on digital techniques.

Q2. What needs to happen for businesses to be able to integrate Communications Performance Management with Business Performance Management?

A2. Did Philip Sheldrake ask you to ask this question? Well, the first thing that would have to happen is for companies to start demanding it. I’ve not seen much demand for this. Once demand builds, smart people will figure out how to make it happen. The AMEC Social Media Measurement Committee is going to take on the challenge of developing a balanced scorecard approach to the social media valid framework to see where that takes us.

Q3. Speaking about social business, are you suggesting social media becomes the strategic imperative with marketing, customer service, PR, employee engagement subordinate?  So these functions will be driven by SM specialists?

A3. No, not at all. I think what we’ll see if that social media permeates all of these functions and creates new capabilities and connections between groups and between customers and companies. It is up to PR or HR people to learn something about social media, SM specialists are not going to take over the world.

Q4. Are the proposed standard social media metrics valid for native ads as well?

A4. I have not thought much about this, but my initial reaction is that the metrics for native ads would be same. A promoted tweet would have the same engagement metrics as any other tweet, although one would certainly hope the performance on some of the metrics would be better.

Q5. What do you mean when you say triage social media content for customer service and support?

A5. This would refer to evaluating and routing social content to different entities or people within an organization (customer care versus technical support versus legal, for example) that are best able to understand and act on the feedback and/or respond to the post.

Q6. Don, what do you put more emphasis on these days, Likes and Follows or Shares and Comments?

A6. I believe the emphasis should be on the stronger indications of engagement, shares and comments, than on simple Likes and Follows.

Q7. How well-known and widely accepted are the Conclave standards in the social space as a whole?

A7. The first complete set of standards were published in early June, 2013. They are known by social media measurement insiders, but I think it is fair to say they are not yet widely known. We need to promote their existence and use.

Q8. How would you measure perception and attitudes through social media?

A8. Generally we would measure consumer conversations about a topic and then do some analysis to see if there are clusters of comments that represent different and distinct viewpoints, attitudes or opinions about the issue or topic. We might also want to do an audience segmentation analysis to see how these attitudes differ by stakeholder group.

Q9. Any specific comments geared towards non-profit organizations?

A9. The basics of measurement – write measurable goals, align goals with organizational KPIs, assess performance against targets –  are the same for for-profit and not-for-profit organizations. How value is created is the primary difference.

Q10. Any suggestions to measure business impact for B2B organizations? Is there a way to understand the impact of social for B2B organizations?  

A10. Most B2B companies have a focus on sales leads. Therefore demonstrating how social is helping create leads or improve lead closure rates is important. There are a lot of uses of social listening in B2B companies as well – how the company is positioned on key issues, who is talking about the company, how products and services are being discussed, etc.

Q11. What are your favorite tools to use in terms of actually measuring your programs/channels/campaigns? Do you identify the tools as you are defining the metrics (do we have the ability to measure X, Y, Z?) or do you select tools after you define your metrics (this is what we need to know, let’s find A, B, C, solutions to measure these things?)?

A11. Generally Google Analytics, a social listening platform (Radian6, Brandwatch, Netbase, Visible, etc.), channel analytics programs like Facebook Insights and also Excel. Ideally you should define metrics first, then the data required for each metric, then look at the tools best able to get the specific data you need.

Q12. What are the most common or most surprising questions you have gotten from CMOs or other key stakeholders regarding social media measurement?

A12. CMOs want to know how social media contributes value to marketing – if they are sales funnel oriented they want to know how social is helping drive the funnel for example. They are also interested if you are helping on front-end or downstream funnel metrics.

Q13. What advice do you have for small businesses for use of and measuring success of social media campaigns effectively (few resources).

A13. Start with the free tools (Hootsuite, Excel, Facebook Insights, Twitter Analytics, Google Analytics) and then work your way up to some of the paid social listening platforms. There is no ‘best’ platform to start with – it really depends on your needs and what you intend to do with the platform. Many companies start with measuring their own channels and evolve to listening to earned/ shared social conversations.

Q14. Which social media analytics do the C-suite find most valuable?

A14. The C-suite don’t really care about social media analytics so much, They care about how social media is helping drive the business metrics forward. That said, C-level folks are usually interesting in competitive benchmarking in social and positioning on key issues and topics that are important to the business. Anything pertaining to online reputation is also an area of interest for many.

Q15. How do you determine what are the correct things to measure?

A15. Measure what matters to the organization. Measurement is about performance against objectives so make sure your measurement program is aligned with business objectives. Don’t measurement ‘social media’, measure what you are trying to accomplish with social media.

Q16. How can someone who is interested in the movement toward standard metrics get involved helping to move the PR industry forward? In other words, how can someone get involved in the debate?

A16. I would suggest interacting directly on the smmstandards.org website. Volunteer to help. Leave suggestions. You could also get involved through one of the PR associations – IPR, PRSA or CoPRF.

Q17. What software would you recommend be used by PR firms to most cost effectively measure social media efforts for clients?

A17. A good social listening platform, Google Analytics, Facebook Insights and the other packages offered by the channels, and good old Excel. Beyond that it really depends on the nature of the social media effort.

Q18. I think a lot of the issue with measurement is confidence in the measurer (i.e., your source). Whenever you cross-reference measurements (e.g. what Google analytics says vs. what your web marketing automation says like HubSpot), you can get wildly different answers. That has stopped me from putting too much faith in my metrics process. Thoughts?

A18. I might separate the issue of the measurer from the sources of data – really two different issues. Regarding sources of data, this is a true issue in that different databases yield different estimates for things like audience size. Compete versus ComScore is a notorious example. However, I don’t think this is a reason to not measure. It simply means we must state assumptions and sources and be consistent over time in using comparable sources. I believe that standard metrics will eventually lead to sanctioned sources for audience data like Arbitron (now Nielsen Audio) for radio or Nielsen for TV.

Q19. Let’s say a social media post leads someone to a landing page, but they do not take immediate action. But they come back the next week and complete the conversion funnel. How do you credit the original social media post…is this a matter of tracking cookies for x number of days? What is practical?

A19. Yep, most people count the first click and then track for a period of time depending on the type of product. It gets even more complicated if you try to suggest there should also be credit given to what happened before the first social media click – for example, money invested in building the brand. Value attribution is an inexact science for sure, with lots of assumptions and compromises.

Q20. What are best ways to measure target audience reach and engagement rather than wide general reach?

A20. Thanks for asking this. The best way to measure is to clearly define your target. If the target is Females 18 – 34, then you should only take credit for reach and engagement of this specific audience only. Given that most tools rely on voluntary bio data, the information is inconsistent and difficult to come by.

Thanks for reading. @Donbart

Social Media Measurement at a Crossroads

We are at a crossroads in social media measurement. Expectations for rigorous and relevant measurement have risen more quickly than delivery. Too many are fixated on quantitative outputs – speeds and feeds – at the expense of understanding the outcomes achieved by social media marketing and social business. There is still too much emphasis on vanity metrics and not enough on business results. And, if you take a step back, there is simply too much talk about all this and not enough action. At the risk of exacerbating the last point, let me explain.

 

Social Media Measurement Started with the Wrong Orientation

In the late 1990s and early 2000s, digital measurement focused on website analytics. The orientation was heavily quantitative. How many unique visitors? How many page views? How long did people remain on site? By 2007, with Facebook now three years old and Twitter completing it’s inaugural year, social media measurement was becoming a hot topic.

Early social media measurement practitioners generally came from the web analytics world. Early social media measurement efforts focused on quantifying outputs and not addressing the outcome of the program. The orientation was on ‘How Many?’ and not ‘What Happened?’ The quantitative orientation also came at the expense of qualitative assessment. The emphasis was on getting easily accessible statistics and not on content analysis to understand meaning and implications. These issues remain today, although we have made significant progress toward shifting the orientation to outcomes and business results.

In the early adopter phase of social media, social media measurement was under little pressure to go beyond quantitative output analysis. Many brands, companies and organizations viewed social media participation as a bit of an experiment to see how it best could be used within their organizations. But this was soon to change.

Struggle Between Easy/Superficial and Hard/Meaningful

It is difficult to pinpoint when social media crossed the chasm into a mainstream business activity. An IDC study in the Fall of 2009 suggested the state of social media still best fit the early adopter and not mainstream use pattern at that point in time. 2011 felt like the year the leap happened to me. With it came a new and emerging set of expectations around social media measurement.

In measurement, it is a truism that the metrics that are easiest to measure are seldom the ones that are most meaningful. It may be easy to measure outputs, but it is often much more difficult and expensive to measure outcomes. It is much easier to determine brand mentions in social media than it is to assess whether or not social programming has changed opinions and attitudes of the target.  It is infinitely easier to measure unique visitors per month than it is to determine the return on investment of a social media initiative.

Now that social media clearly is a mainstream business activity, the pressure to demonstrate the impact and value of social media has greatly increased. As the resources and investment against social media and social business become meaningful line items in the budget, the game changes. Demonstrating business impact and value requires an understanding of the business model of the company or organization and how social media/business creates impact (e.g. change in awareness, increase in purchase consideration, increase in active advocates around an issue) in that environment. Measuring impact is more difficult than measuring audience or engagement. It often involves primary audience research so the price tag is higher.

This is a key struggle we face – will we continue to take the easy, less expensive, minimal-value-of-the-findings approach or we will take social media measurement to another level, focusing on outcomes, investing in audience research and applying rigorous analytics to get at meaning and insight? The imperative is clear, how we respond will be telling. 

A Final Turn to the Right

One of the key themes at this year’s AMEC measurement conference in Madrid was creating a bias toward action. The time to (just) talk about measurement is in the past, the time for action is now. I might suggest this goes double for social media measurement. Here are three areas we can address that will help make the leap from talk to action.

1. Every social media initiative has a measurement plan. Let’s make this happen. Literally any social media initiative, program or activity should have a measurement plan defined before implementation begins. Start with writing social media objectives that are measurable. Align social media metrics with business KPIs. Select metrics across multiple dimensions – programmatic, channel-specific and business-level metrics, for example. Or perhaps paid, owned, earned and shared metrics if your program is integrated across these dimensions. Collect data. Assess performance against objectives. Rinse and repeat
2. Take a stand on standards. An exciting cross-industry effort has produced a set of proposed standards for social media metrics. Adopting standard definitions and metrics for social media is an important stage of measurement maturity that other marketing disciplines like advertising and direct marketing have already reached.
3. Understand, articulate & demonstrate business impact.  The heat is on to demonstrate how social media is helping drive the business or organization forward. We must do a better job of connecting the dots between business KPIs, social media objectives and social media metrics and measurement. In some cases, we want to go beyond understanding attitudinal and behavioral changes to understand the financial value of the impact created. Capturing the financial value of social media requires expertise, data, time and money. We would always like to measure impact, and when it makes sense, we may push further to attribute financial value.

It will be interesting to see what the next year in social media measurement brings. The move toward standardization alone should be fascinating to watch. I have tried to make the argument we are at a crossroads or inflection point in social media measurement maturity. What ‘worked’ for us in the past will not work in the future. We know the expectations. The great unknown is how we respond.

Note: This post was inspired by a Carma webinar,co-sponsored by PRNews, I gave recently. You may download slides from that webinar here.

Social Media Measurement at a Crossroads

Please join me on August 6, from 2:00 – 3:00pm EDT for CARMA’s fourth quarterly webinar, co-sponsored by PRNews, titled “Social Media Measurement at a Crossroads.” Here is a little more information on the webinar:

With social media clearly entrenched as a mainstream business activity, the need to measure the impact on the organization has never been greater. While social media practitioners talk about Like or Follower growth, organizations want to understand how social media is helping drive the business or cause forward. 

Another challenge in social media measurement is the lack of standard definitions, approaches and metrics. In response, a cross-industry push to define social media standards was initiated and initial standards recently published. Social media measurement is clearly at a crossroads where new thinking and approaches are emerging.

In this session you will learn:

• How to align KPIs and metrics to demonstrate organizational impact and value. 
• What industry efforts are being made toward standardization and the implications for how you approach social media measurement
• New models, metrics and frameworks you can use today to develop more effective social media measurement programs.

I will be joined by PRNews Group Editor Matthew Schwartz, who will moderate the discussion and lead a Q&A session.

Here is a link to register for the session. Feel free to leave a comment with any questions you would like answered during the webinar and I will do my best to address them. Hope you can join!

New Framework for Social Media Measurement – Update & Debate

The complete presentation, including speaker notes, I gave at the AMEC Summit in Madrid on June 7 introducing the AMEC Social Media Valid Framework is now available for download here. Please visit and download the slides if you are interested.

Whenever you throw a new idea or concept out there you really hope people see it. And you really, really hope they care enough about it to comment. Or better still, to challenge it and debate its merits. Simply put, critical thinking makes the concept better. Enter stage left, our provocateur, Philip Sheldrake, author of The Business of Influence. Philip has also been an active thought leader in the whole push toward social media standards through AMEC committees and the work of The Conclave. I respect his opinion very much. Philip wrote a rather long essay on his blog in response to my original post on the new Valid Framework for Social Media. Richard Bagnall, Chairman of the Social Media Measurement committee of AMEC posted a response to some of Philip’s concerns. I would like to address a few others now.  

First, one area I am not going to address in this post is whether or not any framework for social media measurement should be driven by Business Performance Management (BPM) principles and approaches. That subject is being debated very well in the comments under Philip’s original post – pretty interesting thread.

Stakeholders v. Customers v. Audiences

Philip questions if the framework, based on the words chosen to describe it, is oriented more toward customers (i.e. social media marketing) and not the broader concept of stakeholders. The intent was and is the make the framework broad enough to comprehend the majority of use cases for social media. Marketing is just one of the use cases. To be fair, the use of ‘audience’ in the description of Impact is a direct lift from the proposed standards document. We were trying to not reinvent the wheel. That said, I agree with you that saying stakeholders or publics is a more accurate description than ‘audience’ in many cases.

Influence

Hard to argue about influence with the guy who wrote a book on it, so I won’t. Philip makes the point that the model offers a definition of influence (Ability to cause or contribute to a change in opinion or behavior) but does not include the qualifying statements or concepts from the WOMMA Guidebook – the potential to influence (before) and the actual, observed influence (during/after). He goes on to say he prefers emphasis on the second. Fully agree. The short definition was offered for clarity and brevity. We will plan to reword this slightly to broaden. You’ll note the illustrative metrics shown in the framework with metrics are consistent with the during/after (e.g. change in purchase consideration) rather than before orientation. If you think about it, the before piece is really about targeting (who are the influencers we should engage?) rather than measurement (did we change opinion, attitudes or behavior of our target stakeholders?).

Value

Philip makes the point that in the model verbiage, the only reference to Value was financial impact whilst the standards definition reads, “the importance, worth or usefulness of something”. Philip also acknowledges this definition was a “last minute tweak”. It was actually being debated as the slides were being developed. We’ll broaden the definition to include tangible and intangible value to be in lock-step with the standards document.

Advocacy

Philip poses two great questions regarding this phase. His first point is a good model that includes advocacy should also be able to comprehend opposition and advocacy for competing agendas. Fully agree these are important to consider and the model certainly does not preclude that. Comprehending these factors would be addressed by one or more metrics within the framework (e.g. Net Positive Advocacy – positive minus negative advocacy) or simply as a point of analysis rather than measurement per se.

Philip’s second question is simply why Advocacy is shown after Impact. It is shown in this order to recognize the fact that most advocacy actually occurs post impact or conversion event. For example, in a sales context, advocacy generally occurs after someone has bought and had great experiences with the product or service. To be more accurate, we might also show Advocacy after Influence and before Impact. Certainly issue advocacy often happens as a direct outcome of a change in opinion or attitude. Rather than show it in two places, to keep it simple, we choose to show it in the sequence where it is most relevant. In practice one could have both pre and post-impact advocacy metrics in your measurement plan.  

Paid Owned Earned  

Philip simply questions the strategic value of this taxonomy and says it props-up organizational silos and reinforces misconceptions such as PR equals (only) Earned Media. I think the questions are valid ones. But as a practical matter many companies and organizations do use the taxonomy. That is a key reason why it is one of the framework alternatives. And while it can reinforce silos and misperceptions, I have actually seen it have the opposite impact – it recognizes integrated programs require integrated measurement and bring three key elements (different departments and sometimes different agencies) together under a common framework. Interestingly in PR, we now routinely develop programs that include earned, paid and owned elements. This reinforces the positive perception public relations today is not simply equal to earned media.

Impact

Finally, Philip questions whether any framework can consider programmatic-level or channel-specific metrics in terms of Impact. I think it can and does. Impact refers to outcomes. We can have macro and micro conversions. And we can have outcomes that are specific to programs (e.g. event attendance, voter registrations, subscriptions to a content series) or channels (download a whitepaper from the website); although I will agree there will be few channel-specific outcomes.

Keep in mind the metrics shown in the frameworks are illustrative, as Richard pointed out in his remarks, they are not meant to be exhaustive, definitive or recommended. They are illustrative of the emerging standard metrics for social media.

 

Thanks for caring.  

A New Framework for Social Media Metrics and Measurement

Last week in Madrid, AMEC (International Association for the Measurement and Evaluation of Communications) held their 5^th European Summit on Measurement. This one was entitled, Unlocking Business Performance – Communications research and analytics in action. One underlying premise of the program this year was that the time for talk is behind us and the time for action and putting into place the principles and practices of sound measurement is upon us. The later part of the program featured an update from Salience Insight Commercial Director Mike Daniels on social media standards including the recently published standardization effort from the cross-industry group called The Conclave which may be found here.

Once Mike discussed where we are with respect to standardization, Richard Bagnall (@richardbagnall), Chairman of the AMEC Social Media Measurement Group, and I as his vice-chair, presented a session on creating a new recommended framework for social media metrics and measurement. Essentially we tried to answer the question: how do we take the standards work coming from The Conclave and operationalize it to create proper social media measurement? Here is an overview of what we presented and what we are encouraging everyone to adopt and use. The framework templates, usage guide and a short video synopsis will be available for download from the AMEC website, Social Media Measurement section, in the next week or so.

Valid Metrics Framework and Social Media

Our journey begins with the Valid Metrics Framework, a measurement planning framework and template developed under the auspices of AMEC. The framework was designed to be flexible enough to address multiple aspects of public relations within a consistent measurement framework and approach.

Some of the most positive aspects of the Valid Metrics Framework are that it:

• Provides a mechanism to link activities to outputs to outcomes
• Tracks through the familiar sales funnel
• Helps create a focus on outcomes.

One of the applications of the Valid Metrics Framework was for use with social media programs. Two potential issues were surfaced by early adopters of the framework in social programs. The intermediary effect, which in traditional public relations is the impact on the media, seemed at odds with the social world of direct interaction between consumers and brands, and consumers with each other. And use of the marketing sales funnel, while familiar, was only relevant in a percentage of social media use cases and perhaps not the best way to model common uses of social media like customer relations and building relationships with stakeholder groups. Also, thought leaders like Forrester Research and McKinsey & Company had noted the traditional communications funnel was not necessarily funnel-shaped in social media. They described the discovery process that occurs when investigating companies and brands that often cause the consideration set to expand rather than be reduced, and the fact that a lot of engagement around brands happens post-conversion event. For all of these reasons our task was develop and recommend a better framework and approach.

Models and Frameworks

When we use the word model, we are referring to a representation of a system, in this case social media. In the original Valid Metrics Framework, the model used was the traditional sales funnel. A framework adds additional dimensions to the model and is operationalized with metrics. In the Valid Metrics Framework the additional dimensions are the phases – activities, intermediary effects and target audience effects. We looked at both of these aspects individually and collectively when considering alternative approaches.

We studied and evaluated about fifteen different social media and communications models. A couple of common patterns emerged. Several of the models, including Forrester’s Customer Lifecycle and McKinsey’s Customer Journey showed a post-purchase engagement/experience step. We judged this important to include in our recommended approach. And we considered the importance of Engagement and Influence, as two key concepts in social media marketing and measurement, and decided to try to make these two elements explicit in our model as well.

Suggested Social Media Metrics Model

The model we developed is derivative of the categories chosen by The Conclave (Note: Richard Bagnall and I also participate in The Conclave) to organize social media metrics and definitions. We took a slightly different perspective on the front end of the model and reordered the back-end to create this model for our new framework. The descriptions of the stages use the definitions from the smmstandrards.org work wherever possible.

You will note Engagement includes both interactions with owned social channels as well as earned social conversation of people ‘talking about you’ in social channels. The definition of Influence is clear and concise, the result of a lot of discussion and prevailing clear thinking. The concept of Impact includes the outcomes of social initiatives as well as the Value those initiatives created. (I usually advise to always attempt to measure impact, and attribute value when it is feasible and makes sense.) Advocacy includes a very helpful definition and conditions that exist with advocacy.

Creating the Framework

To create the framework, we explored various ways to address the ‘phases’ of the Valid Metrics Framework. Two ideas stood out:

• Use a simple structure that captures social media metrics from three key perspectives – programmatic-level, channel-specific and business. Programmatic metrics are those directly tied to social media objectives. Channel-specific metrics are just that, the metrics that are unique to specific social channels – tweets, RTs, Followers, Likes, Talking About This, Pins, Re-Pins. Business metrics are used to show the business impact of the campaign or initiative.

• Use Paid, Owned, and Earned media metrics for integrated programs containing these elements. Borrowing the definitions from Forrester, Paid are social channels you pay to leverage (e.g. promoted tweets, display ads), Owned are channels you own and control (e.g. website, Facebook page) and Earned is where customers become the channel (e.g. WOM, viral)

There are certainly other ways to think about this (e.g. Business Performance Management) and we intend to possibly add others based on industry feedback and suggestions.

The AMEC Social Media Valid Framework

Currently we have developed both versions with sample metrics (taken from the smmstandards.org work where applicable). We are calling them The AMEC Social Media Valid Framework. Here is the version with Program, Channel and Business Metrics shown.

Where Do We Go From Here?

Look for the completed frameworks on the AMEC website shortly. We encourage you to adopt the frameworks for use by your company or clients. If you like them and find them useful, please help promote them widely. And please provide your feedback on the proposed framework on this blog, or through the social channel of your choice. We’re listening and looking forward to the dialogue.

Digital Analytics – From Back Office to Front Page

Analytics have exploded into prominence in the past 15 months. What was once a mysterious statistical discipline understood by few has been elevated as the enabling technology that allows companies to unlock the potential of Big Data. Big Data was everywhere in 2012. There was a track devoted to it at the World Economic Forum in January, 2012. In October, the Harvard Business Review had a cover section on Big Data which characterized analytics as sexy and dubbed its leading practitioners Data Scientists. And at Ketchum, we made analytics training mandatory for ALL employees in 2012, a first for our industry.

The digitization of all forms of analog data is at the heart of the Big Data explosion.  From our click behavior online to purchases we make with a loyalty card to places we go in our vehicles, everything is captured as digital data potentially available for analysis. And with the accelerating use of cameras and sensors, the volume of data promises to keep rising for years to come.

For all the talk about Big Data, no one really wants Big Data. They want the insights hidden within the data that only digital analytics can unlock. Probably the hottest area within digital analytics is predictive analytics. Predictive analytics essentially predict how consumers will behave given certain conditions, assumptions and stimuli. This has powerful and tangible benefits to marketing. An Aberdeen Group study published in December 2011 found marketing organizations that applied data mining and statistical modeling to optimize marketing efforts saw as much as a 2X lift from marketing campaigns, a 76% higher click-through-rate and a 73% higher sales lift.

Marketing organizations are seeing the tremendous power and potential of predictive analytics across a broad spectrum of marketing activities. For example:

• Proactively optimizing marketing campaigns to improve engagement and conversions
• Identifying customers most likely to switch companies and targeting offers to them designed to keep them as customers
• Delivering customized offers designed to appeal to a prospect’s specific interests or life situation
• Predicting which ‘first visit’ customers are most likely to return or not return, and sending offers targeting the ‘not likely to return’ group while avoiding the costs of making offers to those likely to return anyway.

In 2013, we have seen analytics continue to be a mainstream news subject. Analytics is only in the early migration phase from early adopters to mainstream use. Expect this trend to continue and accelerate in the coming years as more marketers discover analytics are a clear path to improving marketing effectiveness, efficiency, and ultimately, the bottom line.

Three Keys to Insight Discovery in Social Listening

True social insights, as opposed to social findings or social observations, have the potential to inform, shape or drive marketing and even business strategy decisions, not just social strategy decisions. Discovering that tweeting with a link on Tuesday between 10:00 – 11:00 AM drives higher levels of engagement is a social finding, not an insight.

Social media is a microcosm of the larger Big Data problem/opportunity – too much data, not enough insights. Or if you prefer, too much noise, not enough signal. If you want to improve your ability to discover insights, here are three simple approaches you can take to improve your insight hunting.

First, start all analysis with a hypothesis or series of questions the analysis is designed to answer. It is much easier to prove or disprove a hypothesis, or answer specific questions, than it is to “find out what people are saying about us in social media”. The more specific the request, the better the answer is going to be. The hypothesis may be one you develop based on preliminary analysis or it may come from the ‘customer’ for the insight. Here are two examples of hypothesis:

• “Conversation about us in social media is quite negative. My boss believes ‘everyone’ is aligned against us. I disagree. My hypothesis is that there is a very vocal and active minority of consumers who are posting large volumes of negative content about us. And I believe this group is a small fraction of the total number of people who post. The majority of consumers are actually neutral toward us.”

• “When we look at Twitter, Facebook and Blogs we see pretty low levels of conversation about our product and the medical condition it treats. We believe there is actually a fair amount of conversation, but the conversations are occurring in Forums which may not be crawled by most social listening tools.”

While the hypothesis is a great way to begin to focus on what is important in the data, a further focusing mechanism is the second insight discovery key – the concept of targeted listening. With targeted listening we are not trying to capture all conversations that mention the brand or product. That is a ‘boil the ocean’ approach. Instead, we listen for very specific types of conversations or conversations by very specific groups of individuals within social conversations. The trick is to have the discipline to only listen to your focus areas and not be tempted to boil the ocean in hopes of finding a few pearls. Here are three examples of targeted listening strategies:

• An insurance company resists the temptation to try to capture ‘all’ mentions of the brand and decides to focus only on conversations where customers are thinking about non-renewal or switching companies.

• A gaming company launches a new product and listens to understand what features are being discussed, what people like most/least about the new game and to gauge their specific reactions to the cover art.

• A consumer products company listens only for consumers who are actively in the purchasing process for the type of products they offer.

The third key to discovering insights is to provide context for decision-making. Remember with insights we are trying to inform, shape and guide decision-making. Context is incredibly important to making better decisions faster. Good social analysts understand how marketing and business work and how strategic alternatives might impact results. Understanding this helps you put your insights in the proper context for decision-making.

Here is an example of how context can lead to better decisions. Company X has a crisis. You are asked to do real-time listening of the crisis and help the PR team decide when and how to engage in the conversation. You come back the next day with a line chart showing a large spike in content mentioning the crisis – thousands of mentions. You know the sentiment in negative to neutral and on which channels the content appears. Unfortunately you have not given the people deciding

whether or not to engage enough information to make a decision. What information would provide the necessary context for decision-making? What questions do we need to try to answer? Here are a few:

• How much above ‘normal levels’ is the spike in content? (Normative data)

• How does this event compare to that event we had last year? Or, how does the event compare to competitor X who had their own crisis last year? (Comparative data) Comparisons help decision makers determine ‘how bad is bad’.

• How long do we anticipate seeing negative content at relatively high levels? (Comparative data) This might be the most important question to answer to provide context and guidance for the engagement decision. If we anticipate volume will drop back to normal in a reasonable period of time, then not engaging may be a viable and effective strategy depending on the brand involved and the nature of the crisis.

• Which stakeholder groups are active in the conversations?  With robust social analysis we always want to look at both the post – what is being said, and the source – who is saying it. In a crisis, who is talking is particularly important.

Normative data, comparative data and examining both post and source data are all effective techniques to provide context for decision-making. 

The tough part about discovering insights is there are no shortcuts and it is a human activity. No social media analysis platform that I have found has an insight button. The key barrier is lack of people who understand how to search for and discover insights. Hope these tips make you a more effective explorer.

Happy Holidays!

Where is Your Organization on the Social Media Listening Maturity Model?

Quite often I am asked to consult with a company on their social media listening strategy. Their initial question more times than not is about the listening platform they should use. But it is increasingly common for the questions to be more sophisticated and the ambition behind them to be much greater. Companies with experience in social listening know that it is all too easy to focus on rudimentary analysis of brand mentions and topics, Followers and Likes and never get to the truly actionable insights that lead to marketing or business actions. Experience in listening is an important element here but you also need a path to follow. I thought a maturity model approach to social media listening could provide a possible path to consider and would provide a construct that could be used in consulting with a company on their social listening strategy.

Maturity models are sort of hot – there seems to be a proliferation in the last two years or so. One that I find particularly insightful and helpful when thinking about social listening is Forrester’s Social Maturity Model.  Two really important points the folks at Forrester make is that listening is not the goal, social intelligence is, and that social intelligence informs actions taken by marketing or some other area of the business. Action being the operative word here. Social intelligence is a closely related topic to social business, and if social business is more your thing the Dachis Group has an interesting social business maturity model.  Big data more your bag? Check out IBM’s big data governance model. After looking at the models out there, I could not find one specific enough to social media listening so I took a stab at creating one.

Social Media Listening Maturity Model 

There are five stages in the Social Media Listening Maturity Model, beginning with reactive alerts and ending with social intelligence. Let’s take a brief look at each stage and some of the overarching differences or changes one sees with social listening maturity.

Reactive Alerts – Many companies or brands begin by establishing a reactive alert system that notifies them whenever their brand is mentioned or is mentioned with specific keywords. Think Google Alerts. Companies in this stage may only periodically check social media channels to see what may have changed or is new since the last check-in.

Monitoring Social Media – At the next stage, the company has begun active monitoring of all ‘owned’ social embassies. They also are monitoring social media conversations, often focused on trying to detect any ‘bad’ news, mentions or conversations.

Companies in these first two stages generally have a reactive stance toward social media, viewing it as another way to find out about news and circumstances that may harm or otherwise impact the organization. It is common for companies in these stages to use one or more of the various free tools available to gather web and social media data.

Social Listening – The third stage is most likely where the largest percentage of companies reside today. Companies in stage three are listening to social conversations about their company, brands and products. They are tracking mentions of competitors and calculating share of conversation. Many also track issues and topics that are important to their brands/products/company. At this stage many begin to put additional emphasis on ‘who’ is talking (source) not just what is being said (post). Most companies in the social listening phase have transitioned from free tools to paid platforms.

Companies in the first three stages often suffer from having too much data and not enough insights. They are up to their necks in ‘big data’ but lack the experience and expertise to analyze the data and reduce it down to crisp, actionable insights supported by the data. They look for the Insight button on the tools they use but increasingly realize insights are the product of human analysts, not tools or data.

Strategic Listening – The transition to strategic listening brings with it a bias toward ‘listening with a purpose’. I first heard this turn of phrase from my friends at Radian6 and use it often. Listening with a purpose is just that – listening to specific sets of conversations with a specific goal or objective in mind. Often in insight work, the goal or objective may take the form of a hypothesis we are trying to test. Here are a few examples of listening with a purpose:

• Listening for conversations of consumers in a particular phase of the buying decision process
• Listening to customers whose subscriptions or policies are about to expire that are expressing thoughts of changing vendors
• Identifying, tracking and building relationships with key influencers
• Listening for consumer reactions to new packaging or product features
• Mining the emotional content of specific stakeholder groups to determine potential risk around a sensitive issue.

During this phase, an Enterprise listening strategy is often developed and implemented. Some also begin to integrate data from sources beyond social media – search, web analytics and customer data for example.

Social Intelligence – Forrester defines social intelligence as the process of turning social media data into actionable marketing and business strategy. Social intelligence therefore is not about the best times to tweet or whether or not a twitter party would be an effective tactic, it is about informing strategic decisions that impact the company’s success. For me, three concepts are crucial:

1. Action – social intelligence is designed to drive true actions.
2. Integration – although the definition focuses on social media data and insights, the fact is that true insights often require more than just social data. Integrating data from multiple data sources – consumer survey, behavioral tracking, social posts, search analytics, advertising data, customer records, scan/sales data – allows for greater understanding and richer insights. Integration of multiple data types often requires multiple tools and platforms to aggregate and analyze the data.
3. Sharing – For social intelligence to truly take root within an organization, the data and insights should involve cross-disciplinary groups that can look at the data from different perspectives and collectively arrive at better insights than any one group could in a vacuum. The insights then need to be systematically shared broadly across the organization so they may be acted upon in a manner that will create the most impact. Social intelligence can be a catalyst to the silos within an organization tumbling down.

Since the social listening and social intelligence ‘markets’ are relatively immature, this model will continue to evolve and be refined.

Where is your company today on the social media listening maturity model?