Inflationary Twitter Audience Numbers Hurt Social Media Credibility

6 Jul

In yesterday’s New York Times, you may have read the article, Spinning the Web: P.R. in Silicon Valley, an interesting although not overly insightful piece.  From a social media measurement perspective, two items caught my eye.  The first, referring to Brian Solis, Principal of FutureWorks, about how he calculates social media audience figures:

“Instead of calculating the impressions an article gets by estimating a publication’s circulation and pass-along rate, Mr. Solis counts the number of people who tweeted about a company and their combined following, the number of retweets or clicks on links, as well as traffic from Facebook and other social networks.”

Toward the end of the article, we learn:

“By 6:30 p.m. on the day Wordnik went live, Brew’s staff calculated that 1.43 million people had seen tweets about it.”

Setting aside for a moment that the article and these sorts of audience metrics take a broadcast-oriented view of Twitter (Mr. Solis discusses the shortcomings of the NYT viewpoint here), the emerging view of audience measures for Twitter is to calculate the Followers of each person tweeting about the subject of interest, and then adding Follower numbers for each person retweeting the subject and so on.  The issue here, much as it is in traditional public relations, is that the audience figure that results from these sorts of calculations grossly overstates, by one or two orders of magnitude or more, the actual “audience” for these tweets.  It is a hypothetical number that assumes everyone that possibly could see a tweet has in fact seen it, and everyone who sees it is relevant to you/your brand.  This is fantasy of course.421922_p~3d-Cinema-Audience-Posters-763348

On the issue of relevant audience, here is a quick example.  At the time I pulled these figures, the audited circulation of the New York Times was 4,974,000.  Most PR practitioners getting a ‘hit’ in the NYT would claim this as their audience.  However:

  • If you were only trying to reach a C-Suite audience with your message, the actual audience reached would be 598,000 or 12% of the total circulation
  • If you were trying to reach Women, your audience would be 1,937,000 or 39% of the total
  • If you were trying to reach 25 – 54 year old Men, your potential audience would have been 2,930,000, or 59% of the total number.

There is a large difference between how many people theoretically can see a tweet, versus how many actually saw it/read it, versus how many of those seeing the tweet find it relevant to them, versus how many engaged with it by hitting a link or retweeting.  Part of my issue with this is the language we use to report the figures.  For the Brew staff to use these numbers to estimate 1.43 million people “had seen tweets about it” is wrong.  If they had said 1.43 million people had an opportunity to see the tweet, it would have been more realistic, although still greatly overstating actual relevant audience.

This problem of audience inflation has already been institutionalized in public relations.  The use of Impressions as an output metric does not mean a true impression in the branding sense, but rather an opportunity to see the content.  To make matters worse, many PR practitioners believe Impressions should be factored by either dubious pass-along readership figures and/or use of a multiplier to account for the mythical credibility advantage PR enjoys over impressions generated from advertising.  The simple fact is there is no research-supported, fact-based argument for using any adder or multiplier in public relations when calculating potential audience (here’s an IPR white paper on this subject I co-authored).

For Twitter and other social networks we lack demographics and data about tweet readership averages (i.e. what is the probability that any one tweet is actually read) that would allow for more precise audience estimates.  In the absence of data, believable assumptions should be used:

  • Out of all the opportunities to see, how many actually read the tweet?  10%?
  • Of those reading the tweet, how many find it relevant to them (or from the other perspective, how many of the readers are in your intended target audience)?  Maybe 10% again?

You can see how our audience estimate has already been reduced by a factor of 100.  This may well still be overstating the actual, relevant audience.  The issue here is that unrealistic and overstated audience figures have the potential to hurt credibility and call into question other data and metrics that may be more grounded in fact.  Actually the more meaningful metrics pertain to engagement or outcomes rather than exposure/outputs.  It is more meaningful that 40,000 visited the Wordnik website as a result of the campaign discussed in the NYT article than the overstated 1.43 million who were estimated to have seen the tweets.  40,000 is real.  1.43 million is fantasy.

15 Responses to “Inflationary Twitter Audience Numbers Hurt Social Media Credibility”

  1. Gail July 6, 2009 at 9:43 am #

    Such good points, Don, and these two especially struck me: First, The assumption that every one of your followers sees everyone of your tweets is completely off-base, and anyone who has used Twitter for more than a couple of days would know. It’s exactly akin to largely discredited “multiplier” metric that counts pass-along readership for magazines. Secondly,reaching the right people is the goal of savvy marketers, not just a lot of random people. Thanks for your attempt to stop the music behind the “social media shimmy” and keep us all honest.

  2. Richard Bagnall July 6, 2009 at 10:18 am #

    Hi Don,

    Great post and thoroughly agree with you.

    Cheers, Richard

  3. metricsman July 6, 2009 at 10:23 am #

    Hi Gail,
    Thanks for your comment. I like “social media shimmy”. Can I quote you on that?

  4. Steve Dodd July 6, 2009 at 11:28 am #

    Don, this is terrific! I’m seeing all of the Twitter stats being produced and they hype is killing the credibility of the SM marketplace. It would be nice to see something that is a bit more accurate and realistic.

  5. Sean Williams July 6, 2009 at 12:21 pm #

    Don, you hit it on the head (again.)

    There seems to be no limit to the sophistry surrounding the measurement of communication activity. I’m in a pretty dark place, myself, right now about social media in general, in part because of the lack of objective metrics to determine its effectiveness. Opportunities to see aren’t particularly useful for me, as I’m still attempting to build awareness — I need metrics on what actually happens to a Tweet.

    Case in point — I have about 160 followers on Twitter right now, but thus far, I’m averaging about 60 visits on the day I post to my blog and Tweet the link out.

    Even if I see a certain Retweet (who has like, 7,000 followers), that visit count doesn’t go up very much.

    At this stage, that’s ok, because this is one-half an experiment — but the other 50% of me wants to see more traffic.

    The fact is, no matter how many people have an opportunity to see, my business will depend on the ones who do more than see…

    Oof, it’s a measurement enigma, wrapped up inside a puzzle.

  6. Sean Williams July 6, 2009 at 12:22 pm #

    @commammo

    You see, I even forgot to include my Twitter name!

  7. metricsman July 6, 2009 at 1:00 pm #

    Hi Sean,
    Thanks for your thoughtful (again) comment. I’m with you on metrics like OTS, and in fact, argued in a previous post that no real value creation occurs unless possible exposure leads to actual exposure and engagement. Our value in PR or SM is what happens as a result of the output (content) i.e. engagement/outcome, not in the output itself.

    On your blog traffic, part of it I believe is a function of time. IMO your continued and regular blogging and participation in social networks will pay off. You will notice a lot of pretty regular Joes that have sent out 4000+ tweets have over 1000 followers. Unless you’re gaming the system by using follow services (e.g. someone with 4 tweets and over 1000 followers), it takes time to build a community organically. Today, well over half of my blog traffic is coming directly from Twitter. A year ago it was 0%.

  8. metricsman July 6, 2009 at 1:46 pm #

    Steve,
    Thanks for your comment. I agree on hype – it ends up creating unrealistic expectations and disappointment. PR and SM are high-value activities, there is no need to inflate the results.

  9. Lisa Golloher July 8, 2009 at 7:38 am #

    I find it strange that people would use traditional subscriber audience numbers of a static medium to measure the impact of a dynamic, live medium like twitter. Again marketers are having a hard time letting go of their archaic perspective, instead of embracing social media and evolving their thinking.

  10. bloggenblitzen July 8, 2009 at 2:08 pm #

    Good article. “Co-author” is not a verb.

  11. metricsman July 8, 2009 at 2:16 pm #

    Lisa,
    Thanks for the comment. One of the major issues now is the misapplication of traditional media metrics (like impressions/eyeballs) to social media. You are exactly right.

    Bloggenblitzen,
    Thanks for the comment and the grammatical correction.

  12. LKinoshita July 23, 2009 at 8:32 pm #

    Amen! Most Twitter apps also limit the amount of updates to 100 or 200 — so if I don’t go on Twitter for a couple of days, I’m not even going to have the opportunity to see 80% of the posts. The more people you follow, the more this becomes a problem.

    And I about fell off my chair when I saw a PRSA Anvil entry citing “an industry standard) of 3.5 for using AVE, plus some made-up numbers for how they calculated online impressions. Puh-lease! They actually had some link-farm ranked at over $100K of value — more than half of all total value!

    What is going on!?

    Clients need to challenge the PR firms and demand that anyone passing around AVE show the research that justifies it. They won’t be able to, of course.

  13. Jackie Chazan July 28, 2009 at 2:06 pm #

    Don:

    Your post should be required reading for everyone interested in marketing, public relations, advertising and communications.

    I have always told my clients that the true metric of a succesful campaign is the achievement of the objective: If the campaign is for a product, did it increase sales? If it is for an event, did it increase attendance? and so on.

    You say ‘The issue here, much as it is in traditional public relations, is that the audience figure that results from these sorts of calculations grossly overstates, by one or two orders of magnitude or more, the actual “audience” for these tweets. It is a hypothetical number that assumes everyone that possibly could see a tweet has in fact seen it, and everyone who sees it is relevant to you/your brand.’
    To which I would add that just because it is seen it doesn’t mean it was understood, it was remembered, and it moved the audience to action.

    Unless a message (or tweet) compells a person to do something, it is just noise.

  14. metricsman July 28, 2009 at 4:11 pm #

    LKinoshita,

    Right you are about simply missing 80% of your tweets unless you are glued to your tweetstream. This is a significant issue.

    I was a judge of a major PR contest a couple of years ago and was amazed by the lack of well-done research and measurement, particularly measurement. I immediately docked an entry 15% if they used AVEs in their measurement approach.

    Many people believe AVEs are the easy road to ROI, but of course they really tell us nothing of the true ROI of our programs. They are the easy road to nowhere.

  15. metricsman July 28, 2009 at 4:19 pm #

    Hi Jackie,
    Thanks for stopping by and for your kind words. Writing measurable objectives that are properly aligned with whatever business/marketing outcome is desired is fundamental to good measurement – I agree with the advice you give clients. I also agree, of course, that messages must be relevant, understood and remembered in order to be effective. I disagree slightly that all messages must motivate an action – some messages are merely intended to create awareness or change an opinion or attitude. The action step may occur later in the process and be motivated by subsequent messages. Thanks again!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 159 other followers

%d bloggers like this: