New Framework for Social Media Measurement – Update & Debate

The complete presentation, including speaker notes, I gave at the AMEC Summit in Madrid on June 7 introducing the AMEC Social Media Valid Framework is now available for download here. Please visit and download the slides if you are interested.

Whenever you throw a new idea or concept out there you really hope people see it. And you really, really hope they care enough about it to comment. Or better still, to challenge it and debate its merits. Simply put, critical thinking makes the concept better. Enter stage left, our provocateur, Philip Sheldrake, author of The Business of Influence. Philip has also been an active thought leader in the whole push toward social media standards through AMEC committees and the work of The Conclave. I respect his opinion very much. Philip wrote a rather long essay on his blog in response to my original post on the new Valid Framework for Social Media. Richard Bagnall, Chairman of the Social Media Measurement committee of AMEC posted a response to some of Philip’s concerns. I would like to address a few others now.  

First, one area I am not going to address in this post is whether or not any framework for social media measurement should be driven by Business Performance Management (BPM) principles and approaches. That subject is being debated very well in the comments under Philip’s original post – pretty interesting thread.

Stakeholders v. Customers v. Audiences

Philip questions if the framework, based on the words chosen to describe it, is oriented more toward customers (i.e. social media marketing) and not the broader concept of stakeholders. The intent was and is the make the framework broad enough to comprehend the majority of use cases for social media. Marketing is just one of the use cases. To be fair, the use of ‘audience’ in the description of Impact is a direct lift from the proposed standards document. We were trying to not reinvent the wheel. That said, I agree with you that saying stakeholders or publics is a more accurate description than ‘audience’ in many cases.

Influence

Hard to argue about influence with the guy who wrote a book on it, so I won’t. Philip makes the point that the model offers a definition of influence (Ability to cause or contribute to a change in opinion or behavior) but does not include the qualifying statements or concepts from the WOMMA Guidebook – the potential to influence (before) and the actual, observed influence (during/after). He goes on to say he prefers emphasis on the second. Fully agree. The short definition was offered for clarity and brevity. We will plan to reword this slightly to broaden. You’ll note the illustrative metrics shown in the framework with metrics are consistent with the during/after (e.g. change in purchase consideration) rather than before orientation. If you think about it, the before piece is really about targeting (who are the influencers we should engage?) rather than measurement (did we change opinion, attitudes or behavior of our target stakeholders?).

Value

Philip makes the point that in the model verbiage, the only reference to Value was financial impact whilst the standards definition reads, “the importance, worth or usefulness of something”. Philip also acknowledges this definition was a “last minute tweak”. It was actually being debated as the slides were being developed. We’ll broaden the definition to include tangible and intangible value to be in lock-step with the standards document.

Advocacy

Philip poses two great questions regarding this phase. His first point is a good model that includes advocacy should also be able to comprehend opposition and advocacy for competing agendas. Fully agree these are important to consider and the model certainly does not preclude that. Comprehending these factors would be addressed by one or more metrics within the framework (e.g. Net Positive Advocacy – positive minus negative advocacy) or simply as a point of analysis rather than measurement per se.

Philip’s second question is simply why Advocacy is shown after Impact. It is shown in this order to recognize the fact that most advocacy actually occurs post impact or conversion event. For example, in a sales context, advocacy generally occurs after someone has bought and had great experiences with the product or service. To be more accurate, we might also show Advocacy after Influence and before Impact. Certainly issue advocacy often happens as a direct outcome of a change in opinion or attitude. Rather than show it in two places, to keep it simple, we choose to show it in the sequence where it is most relevant. In practice one could have both pre and post-impact advocacy metrics in your measurement plan.  

Paid Owned Earned  

Philip simply questions the strategic value of this taxonomy and says it props-up organizational silos and reinforces misconceptions such as PR equals (only) Earned Media. I think the questions are valid ones. But as a practical matter many companies and organizations do use the taxonomy. That is a key reason why it is one of the framework alternatives. And while it can reinforce silos and misperceptions, I have actually seen it have the opposite impact – it recognizes integrated programs require integrated measurement and bring three key elements (different departments and sometimes different agencies) together under a common framework. Interestingly in PR, we now routinely develop programs that include earned, paid and owned elements. This reinforces the positive perception public relations today is not simply equal to earned media.

Impact

Finally, Philip questions whether any framework can consider programmatic-level or channel-specific metrics in terms of Impact. I think it can and does. Impact refers to outcomes. We can have macro and micro conversions. And we can have outcomes that are specific to programs (e.g. event attendance, voter registrations, subscriptions to a content series) or channels (download a whitepaper from the website); although I will agree there will be few channel-specific outcomes.

Keep in mind the metrics shown in the frameworks are illustrative, as Richard pointed out in his remarks, they are not meant to be exhaustive, definitive or recommended. They are illustrative of the emerging standard metrics for social media.

 

Thanks for caring.  

A New Framework for Social Media Metrics and Measurement

Last week in Madrid, AMEC (International Association for the Measurement and Evaluation of Communications) held their 5^th European Summit on Measurement. This one was entitled, Unlocking Business Performance – Communications research and analytics in action. One underlying premise of the program this year was that the time for talk is behind us and the time for action and putting into place the principles and practices of sound measurement is upon us. The later part of the program featured an update from Salience Insight Commercial Director Mike Daniels on social media standards including the recently published standardization effort from the cross-industry group called The Conclave which may be found here.

Once Mike discussed where we are with respect to standardization, Richard Bagnall (@richardbagnall), Chairman of the AMEC Social Media Measurement Group, and I as his vice-chair, presented a session on creating a new recommended framework for social media metrics and measurement. Essentially we tried to answer the question: how do we take the standards work coming from The Conclave and operationalize it to create proper social media measurement? Here is an overview of what we presented and what we are encouraging everyone to adopt and use. The framework templates, usage guide and a short video synopsis will be available for download from the AMEC website, Social Media Measurement section, in the next week or so.

Valid Metrics Framework and Social Media

Our journey begins with the Valid Metrics Framework, a measurement planning framework and template developed under the auspices of AMEC. The framework was designed to be flexible enough to address multiple aspects of public relations within a consistent measurement framework and approach.

Some of the most positive aspects of the Valid Metrics Framework are that it:

• Provides a mechanism to link activities to outputs to outcomes
• Tracks through the familiar sales funnel
• Helps create a focus on outcomes.

One of the applications of the Valid Metrics Framework was for use with social media programs. Two potential issues were surfaced by early adopters of the framework in social programs. The intermediary effect, which in traditional public relations is the impact on the media, seemed at odds with the social world of direct interaction between consumers and brands, and consumers with each other. And use of the marketing sales funnel, while familiar, was only relevant in a percentage of social media use cases and perhaps not the best way to model common uses of social media like customer relations and building relationships with stakeholder groups. Also, thought leaders like Forrester Research and McKinsey & Company had noted the traditional communications funnel was not necessarily funnel-shaped in social media. They described the discovery process that occurs when investigating companies and brands that often cause the consideration set to expand rather than be reduced, and the fact that a lot of engagement around brands happens post-conversion event. For all of these reasons our task was develop and recommend a better framework and approach.

Models and Frameworks

When we use the word model, we are referring to a representation of a system, in this case social media. In the original Valid Metrics Framework, the model used was the traditional sales funnel. A framework adds additional dimensions to the model and is operationalized with metrics. In the Valid Metrics Framework the additional dimensions are the phases – activities, intermediary effects and target audience effects. We looked at both of these aspects individually and collectively when considering alternative approaches.

We studied and evaluated about fifteen different social media and communications models. A couple of common patterns emerged. Several of the models, including Forrester’s Customer Lifecycle and McKinsey’s Customer Journey showed a post-purchase engagement/experience step. We judged this important to include in our recommended approach. And we considered the importance of Engagement and Influence, as two key concepts in social media marketing and measurement, and decided to try to make these two elements explicit in our model as well.

Suggested Social Media Metrics Model

The model we developed is derivative of the categories chosen by The Conclave (Note: Richard Bagnall and I also participate in The Conclave) to organize social media metrics and definitions. We took a slightly different perspective on the front end of the model and reordered the back-end to create this model for our new framework. The descriptions of the stages use the definitions from the smmstandrards.org work wherever possible.

You will note Engagement includes both interactions with owned social channels as well as earned social conversation of people ‘talking about you’ in social channels. The definition of Influence is clear and concise, the result of a lot of discussion and prevailing clear thinking. The concept of Impact includes the outcomes of social initiatives as well as the Value those initiatives created. (I usually advise to always attempt to measure impact, and attribute value when it is feasible and makes sense.) Advocacy includes a very helpful definition and conditions that exist with advocacy.

Creating the Framework

To create the framework, we explored various ways to address the ‘phases’ of the Valid Metrics Framework. Two ideas stood out:

• Use a simple structure that captures social media metrics from three key perspectives – programmatic-level, channel-specific and business. Programmatic metrics are those directly tied to social media objectives. Channel-specific metrics are just that, the metrics that are unique to specific social channels – tweets, RTs, Followers, Likes, Talking About This, Pins, Re-Pins. Business metrics are used to show the business impact of the campaign or initiative.

• Use Paid, Owned, and Earned media metrics for integrated programs containing these elements. Borrowing the definitions from Forrester, Paid are social channels you pay to leverage (e.g. promoted tweets, display ads), Owned are channels you own and control (e.g. website, Facebook page) and Earned is where customers become the channel (e.g. WOM, viral)

There are certainly other ways to think about this (e.g. Business Performance Management) and we intend to possibly add others based on industry feedback and suggestions.

The AMEC Social Media Valid Framework

Currently we have developed both versions with sample metrics (taken from the smmstandards.org work where applicable). We are calling them The AMEC Social Media Valid Framework. Here is the version with Program, Channel and Business Metrics shown.

Where Do We Go From Here?

Look for the completed frameworks on the AMEC website shortly. We encourage you to adopt the frameworks for use by your company or clients. If you like them and find them useful, please help promote them widely. And please provide your feedback on the proposed framework on this blog, or through the social channel of your choice. We’re listening and looking forward to the dialogue.

Will Barcelona Measurement Debate Shake Up the Industry?

In mid-June, the Second European Summit on Measurement will be held in Barcelona.  The Summit is jointly organized by the International Association for Measurement and Evaluation of Communication (AMEC) and the Institute for Public Relations.  The highlight of the conference will be a debate to “articulate and agree on standard metrics and measurement techniques…” according to a press release issued last week.

David Rockland, head of research for Ketchum, will lead the session.  Mr. Rockland has a lofty vision and high hopes for the debate, “We regard this as the industry’s ‘Commitment Conference’.  This is a very powerful moment in time in the history of public relations.  Until now, public relations has been undervalued due to its inability to measure itself.  The goal of this summit is to establish consistency in order to increase credibility.”

The three-day conference is expected to be attended by about 150 people, including many measurement thought leaders.  It should be interesting to see what comes out of this event.  Will it serve as a wake-up call to the industry?  Time will tell.  I’ll post some thoughts once the summit concludes.

Keeping it real…and transparent

• I am a member of the Institute for Public Relations, Commission on Public Relations Measurement & Evaluation
• My agency, Fleishman Hillard, has agreed to join AMEC (pending paperwork)
• Ketchum is a sister Omnicom agency