The proliferation of Twitter analytics tools continues. One of the most popular categories are tools that purport to calculate Twitter influence, that is, identify the individuals who influence others. They go by clever names – Tweetlevel, Twinfluence and Twittergrader to mention a few. (Here is a good list of Twitter tools from Brian Solis). They are fun – who doesn’t want to punch in their Twitter user name and see how “influential” you are? But are they truly useful? Should anyone base serious marketing or business decisions on these tools?
All these tools take a similar conceptual approach:
- Define the terms that are believed to be related to Influence.
- Arrange the terms in a sophisticated looking formula. Better yet – call it an algorithm.
- Factor the terms of the formula according to some proprietary coefficient weighting. (Tip: Labeling them ‘proprietary’ heightens the perception of mathematical precision.)
- Add a lot of terms to the formula so it seems complicated to the casual observer. (Tip: Complexity will heighten believability. Make it too understandable and believability may suffer.)
The game here is to create an aura of rigor when one in fact does not exist.
Equations are easy, coefficients are hard
I suspect in all the models discussed or available, the critical weighting of variables – assigning beta coefficients – was done by judgment, not math. To correctly assign coefficients, one would use statistical techniques involving means and standard deviations to determine the coefficient of each independent variable (number of followers, how often content is re-tweeted, etc.) and determine the relationships (correlation) to our dependant variable – Influence. The dependant variable should be observable and measurable. Here’s where it further breaks down. The problem here is no one is actually measuring true Influence – the ability of one individual to change another’s opinions, attitudes or behavior. You can’t surmise whether or not an opinion or attitude has been impacted, you have to conduct research. Opinions and attitudes exist within individuals. You cannot assess this by proxy, looking strictly at online metrics. Online behavior can be measured without primary research, but offline behaviors have to be observed or reported.
Influence is contextual
Influence is contextual not absolute. An individual may have the ability to influence certain people in specific subject areas. Authority and trust are important constituent elements of influence. Do they have the authority to speak within a particular area and are their words and deeds trusted? The notion of coming up with an influence score without context is inherently flawed. It might be interesting, but it is not actionable.
According to the results generated by this class of tools, I believe they are probably assessing popularity much more than Influence in a meaningful way. According to Tweetlevel this morning, Ashton Kutcher is the second most influential person on Twitter. Who exactly does he influence and in which areas? Mr. Kutcher is popular (number one in Popularity) but I’m skeptical of his true influence. He is also the most trusted person on Twitter according to Tweetlevel. The second most trusted person – Perez Hilton. Enough said.
Marketing not math
While I have been critical of these tools, I am not naïve enough to believe the intent was to create a rigorous analytic tool that could be used to target individuals that might have the most influence over your target customers. These tools are most likely designed to put a hand on your wallet, not insights in your marketing effort. Do they work for marketing purposes? Hard to say, but I’m sure they do in some cases. But, proceed with caution. You are walking a slippery slope in my view if you believe that developing a pseudo algorithm and slick website is the best indication that a company has the digital chops and experience to help drive your social business efforts. Getting to know the individuals involved and the work they have performed for companies like yours is preferable. Beware of shiny objects, they are not always as they seem.
metricsman.wordpress.com 
This is a very well-thought analysis–it threw me back to stats class when you started to talk about standard deviations and coefficients. So, I enjoyed it AND I agree wholeheartedly. Thanks for taking the time to think this through and post!
Hi Don, thanks for the mention! I’ve been toying with Klout.com over the last few days…was asked to cover them for TechCrunch. They have a very interesting formula for identifying influencers as well as influencers by topic. Would love your thoughts on it. Cheers!
David,
Thanks very much for stopping by and leaving a comment. Sorry about the mini-stats talk – I promise not to do it too often!
Brian,
Thanks very much for commenting. I’ll look into Klout and let you know what I think. I doubt my point about these tools will change too much however, measuring influence by proxy – follower count, RTs, posting velocity, etc. really needs to be verified with some true influence research. Model verification would help me change my mind. Your blog is outstanding BTW – strong conceptual thinking and very well-written.
Thanks again, Don B @donbart
I think your contextual point is important. Context is also often related to geography. Knowing who is influential on Twitter (however you define that) is one thing.
Knowing who is influential in your bit of the world if you are outside the US is much more important than knowing that a system dominated by US users overall has US people most influential – surprise.
The Twitter trends are in their own way a good illustration of this problem. They tell us what people in the US are talking about, sometimes what people in the UK are talking about and very rarely what people elsewhere are talking about. There aren’t actually very many uses for getting that mix of information.
Hi Mark,
You make an outstanding point about geography and culture being important contextual touchstones. I was remiss in not mentioning that in my post. You are, of course, absolutely right about Twitter being primarily U.S.-centric at this point. Thanks for reminding us to have more of a global perspective, I appreciate it.
-Don B
Very nice analysis and very timely for me. I also Tweeted it. Thanks. Mark
Thanks for stopping by, Mark, and for your kind words. -Don B
Agreed that these tools are all “secret sauce” with no way of evaluating what it is they evaluate.
I measure the @WSUSpokane account using a good-sized list of these, looking for positive trends rather than at any “absolute” or real meaning to the numbers.
Twitalyzer looks at signal-to-noise ratio, generosity, and some other elements that I find interesting as an attempt at an all-around look at what your Twitter account does and how others respond.
I wish I knew what it means when Twitterholic tells me that @WSUSpokane is ranked 4th in our location by followers in our hometown. That’s a measurement that might really matter since one goal is to reach community members, but I don’t know how it’s calculated.
@BarbChamberlain
Director of Communications and Public Affairs
Washington State University Spokane
@WSUSpokane
http://www.spokane.wsu.edu
Great post Don. I always appreciate analysis of the analytics – there should be more. Business wants social media, like traditional media, to prove either the existence of a relative audience, or to prove a change of behaviour. Your analysis of the twitter analytic tools show how they really fail to do either.
This is a nice dose of reality. I constantly remind people when I speak that Twitter is not a silver bullet and needs to be part of your communications mix. These tools are interesting and I’ve used them, but they are not very helpful. We need to continue to focus on social media as it fits into the overall brand of our company/clients. Thanks for the great post.
Allan
@allanschoenberg/@CMEGroup
Hi Barb,
Thanks for stopping by and for your comment. Your focus on trends rather than the absolute numbers is a wise course. Your frustration in not knowing how metrics are calculated is a core issue with compound metrics and indexes. You really can’t get to the potentially valuable ‘clean’ metrics beneath the hood.
I’m not sure how Twitterholic makes their calculation’s either, but my guess would be of the accounts based in Spokane, you have the 4th.-highest number of followers who reside in Spokane. Of course it would be nice to better understand the underlying numbers, but this does seem like it could be meaningful to you. -Don B
Hi Andrew (or should I say Dr. Laing?),
Thanks for your comment. Hope you are well. I like your summation – but other than that, they are fun and entertaining, right? 😉 -Don B
Allan,
Thanks for stopping by and for sharing your wise advice – couldn’t agree more. -Don B
This is such a great article. Basically, if you’ve set up measurable objectives, you can’t evaluate them against a twitter influence score, as it’s a virtual guesstimate of impact. Thus, another important reason to put money into legitimate measurement systems all throughout your programs.
Don – I think it’s fair to say we are all struggling with answering these questions about social media in general. What does it all mean and how do we determine its effectiveness? Descriptive stats gleaned from Web data cannot replace research into actual business outcomes.
Thanks again.
Sean
Great post, Don. I chanced upon your blog while on KD Paine’s blog. You make an excellent point about influence being contextual. I haven’t come across a single analytics tool that assigns levels of influence to individuals based on the context. I agree with you about the so-called algorithms and the equations. Too complicated and difficult to know how they determined coefficients.
Thanks again for the insight!
Great post – thank you. I certainly agree with the points about ‘influence’. That is one of the great things about social media isn’t it; it levels the playing field when it comes to influencing other people! I also agree that Klout is a good tool; the best I have seen so far. If you are interested I have published their top UK Twitter Influencers list at http://rich-baker.com/2010/01/04/chris-moyles-lily-allen-top-twitter-influencers/
This blog was… how do I say it? Relevant!! Finally I have found something which helped me.
Thanks!
you are really a just right webmaster. The web site loading pace is amazing.
It kind of feels that you are doing any unique trick. In addition, The contents are masterpiece.
you have performed a wonderful task on this topic!