New Framework for Social Media Measurement – Update & Debate

21 Jun

The complete presentation, including speaker notes, I gave at the AMEC Summit in Madrid on June 7 introducing the AMEC Social Media Valid Framework is now available for download here. Please visit and download the slides if you are interested.

Whenever you throw a new idea or concept out there you really hope people see it. And you really, really hope they care enough about it to comment. Or better still, to challenge it and debate its merits. Simply put, critical thinking makes the concept better. Enter stage left, our provocateur, Philip Sheldrake, author of The Business of Influence. Philip has also been an active thought leader in the whole push toward social media standards through AMEC committees and the work of The Conclave. I respect his opinion very much. Philip wrote a rather long essay on his blog in response to my original post on the new Valid Framework for Social Media. Richard Bagnall, Chairman of the Social Media Measurement committee of AMEC posted a response to some of Philip’s concerns. I would like to address a few others now.  

First, one area I am not going to address in this post is whether or not any framework for social media measurement should be driven by Business Performance Management (BPM) principles and approaches. That subject is being debated very well in the comments under Philip’s original post – pretty interesting thread.

Stakeholders v. Customers v. Audiences

Philip questions if the framework, based on the words chosen to describe it, is oriented more toward customers (i.e. social media marketing) and not the broader concept of stakeholders. The intent was and is the make the framework broad enough to comprehend the majority of use cases for social media. Marketing is just one of the use cases. To be fair, the use of ‘audience’ in the description of Impact is a direct lift from the proposed standards document. We were trying to not reinvent the wheel. That said, I agree with you that saying stakeholders or publics is a more accurate description than ‘audience’ in many cases.

Influence

Hard to argue about influence with the guy who wrote a book on it, so I won’t. Philip makes the point that the model offers a definition of influence (Ability to cause or contribute to a change in opinion or behavior) but does not include the qualifying statements or concepts from the WOMMA Guidebook – the potential to influence (before) and the actual, observed influence (during/after). He goes on to say he prefers emphasis on the second. Fully agree. The short definition was offered for clarity and brevity. We will plan to reword this slightly to broaden. You’ll note the illustrative metrics shown in the framework with metrics are consistent with the during/after (e.g. change in purchase consideration) rather than before orientation. If you think about it, the before piece is really about targeting (who are the influencers we should engage?) rather than measurement (did we change opinion, attitudes or behavior of our target stakeholders?).

Value

Philip makes the point that in the model verbiage, the only reference to Value was financial impact whilst the standards definition reads, “the importance, worth or usefulness of something”. Philip also acknowledges this definition was a “last minute tweak”. It was actually being debated as the slides were being developed. We’ll broaden the definition to include tangible and intangible value to be in lock-step with the standards document.

Advocacy

Philip poses two great questions regarding this phase. His first point is a good model that includes advocacy should also be able to comprehend opposition and advocacy for competing agendas. Fully agree these are important to consider and the model certainly does not preclude that. Comprehending these factors would be addressed by one or more metrics within the framework (e.g. Net Positive Advocacy – positive minus negative advocacy) or simply as a point of analysis rather than measurement per se.

Philip’s second question is simply why Advocacy is shown after Impact. It is shown in this order to recognize the fact that most advocacy actually occurs post impact or conversion event. For example, in a sales context, advocacy generally occurs after someone has bought and had great experiences with the product or service. To be more accurate, we might also show Advocacy after Influence and before Impact. Certainly issue advocacy often happens as a direct outcome of a change in opinion or attitude. Rather than show it in two places, to keep it simple, we choose to show it in the sequence where it is most relevant. In practice one could have both pre and post-impact advocacy metrics in your measurement plan.  

Paid Owned Earned  

Philip simply questions the strategic value of this taxonomy and says it props-up organizational silos and reinforces misconceptions such as PR equals (only) Earned Media. I think the questions are valid ones. But as a practical matter many companies and organizations do use the taxonomy. That is a key reason why it is one of the framework alternatives. And while it can reinforce silos and misperceptions, I have actually seen it have the opposite impact – it recognizes integrated programs require integrated measurement and bring three key elements (different departments and sometimes different agencies) together under a common framework. Interestingly in PR, we now routinely develop programs that include earned, paid and owned elements. This reinforces the positive perception public relations today is not simply equal to earned media.

Impact

Finally, Philip questions whether any framework can consider programmatic-level or channel-specific metrics in terms of Impact. I think it can and does. Impact refers to outcomes. We can have macro and micro conversions. And we can have outcomes that are specific to programs (e.g. event attendance, voter registrations, subscriptions to a content series) or channels (download a whitepaper from the website); although I will agree there will be few channel-specific outcomes.

Keep in mind the metrics shown in the frameworks are illustrative, as Richard pointed out in his remarks, they are not meant to be exhaustive, definitive or recommended. They are illustrative of the emerging standard metrics for social media.

 

Thanks for caring.  

3 Responses to “New Framework for Social Media Measurement – Update & Debate”

  1. philipsheldrake June 24, 2013 at 1:26 pm #

    Hi Don, thanks for your comments over on my blog post, and for responding at greater length here. Like you, I won’t press my p.o.v. on business performance management here. I’ll dive in if I may on the points you do discuss here, although, as much as we should celebrate areas of agreement, please simply interpret omissions as agreement.

    AUDIENCES

    Let’s both feed back on this one while the SMM Standards remain open for comment.

    ADVOCACY

    I just searched the web for “Net Positive Advocacy” and the only two results are for metricsman.wordpress.com :-) If this is what I think it is, I would just express caution on the basis that 10-6 = 5-1 = 4-0 but the information loss that otherwise distinguishes these sums is unfortunate and may lead to inappropriate action.

    The standards assert that “impact and value represent the ultimate outcome of a social media effort.” I’m still struggling to reconcile ultimate (being or happening at the end of a process; final) with advocacy coming after it! Saying that, we all know this stuff doesn’t play out in perfect sequence with a beginning and end but with each and every stage happening concurrently.

    If anything in this process is ultimate, I still consider it to be influence – changing others’ minds and behaviour and having them change our own. Of course, if you influence someone to be an advocate, the influence propagates further.

    IMPACT

    I interpreted the standards definition of impact – “the ultimate outcome” – as something the chairman might reference in the introduction to an annual report. (S)he’s talking about the business in terms that all stakeholders and other parties can understand, and not programmatic-level or channel-specific metrics.

    But I thought I’d remind myself of the definition of “outcome” on the AMEC Glossary (link below): “Something that has happened as the result of a campaign. In public relations this would typically be defined as a measurable change in awareness, knowledge, attitude, opinion, behaviour or reputation metrics.”

    How does this stack up against the three levels of metrics in the proposed framework?…

    > Programmatic metrics – directly tied to social media objectives
    > Channel-specific metrics – unique to specific social channels (e.g., tweets, RTs, followers, likes, talking about this, pins, re-pins)
    > Business metrics – the business impact of the campaign or initiative.

    Of course, “outcome” definitely relates to business metrics.

    As for programmatic metrics, I find it difficult to equate the framework’s example metrics (e.g., referral traffic to website) with outcome. Social media objectives may be considered on occasions in isolation from other communications objectives, and indeed other organisational objectives, but perhaps not as frequently as in tandem.

    And I maintain that “outcome” cannot be applied to channel-specific metrics without confusing outcomes with outputs.

    http://amecorg.com/2012/06/glossary_plain_speaking/#outcome

    http://amecorg.com/2012/06/glossary_plain_speaking/#output

  2. metricsman June 25, 2013 at 10:02 am #

    Thanks for your thoughtful comments, Philip. Few items in response:

    > I am not (intentionally) advocating a Net Advocacy metric. I’m just trying to provide a simple example of a way to comprehend both positive and negative advocacy. I certainly would analyze them separately before trying to combine the concepts into a single metric. First order of approximation for me is to track Positive Advocacy and then add layers of analysis sophistication as needs dictate.

    > Not sure I agree that Influence might be the end Impact result, I think in most cases Influence is the strategy and what happens as a result of the influence change (i.e. behavior) is the objective.

    > Regarding the programmatic-level ‘Impact’, I think it depends a bit on how you define things. Let’s take the case of a blogger outreach program designed to drive trial usage of a product. We might measure two outputs – number of blogger meetings held and the number of posts generated by bloggers offering samples. The programmatic-level outcome in this context could be thought of as the number of trial units distributed. Of course, there is an associated business-level outcome of the number of net new customers created as a result of the trial. My assertion is that both of these are valid and may co-exist. But, perhaps I am wrong and this will prove to be a point of confusion with the model that leads us to say that Impact only occurs at the business-level. I’m not quite there yet, but would love to hear from others on this.

    Thanks again, DB @Donbart

Trackbacks/Pingbacks

  1. Social Media Metrics: combineer drie sterke modellen | Social Media Modellen - July 3, 2013

    […] de uitleg op het blog van MetricsMan wordt uitgebreid uitgelegd hoe deze drie modellen elkaar aanvullen. Deze presentatie geeft het […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 170 other followers

%d bloggers like this: