Digital Analytics – From Back Office to Front Page

Analytics have exploded into prominence in the past 15 months. What was once a mysterious statistical discipline understood by few has been elevated as the enabling technology that allows companies to unlock the potential of Big Data. Big Data was everywhere in 2012. There was a track devoted to it at the World Economic Forum in January, 2012. In October, the Harvard Business Review had a cover section on Big Data which characterized analytics as sexy and dubbed its leading practitioners Data Scientists. And at Ketchum, we made analytics training mandatory for ALL employees in 2012, a first for our industry.

The digitization of all forms of analog data is at the heart of the Big Data explosion.  From our click behavior online to purchases we make with a loyalty card to places we go in our vehicles, everything is captured as digital data potentially available for analysis. And with the accelerating use of cameras and sensors, the volume of data promises to keep rising for years to come.

For all the talk about Big Data, no one really wants Big Data. They want the insights hidden within the data that only digital analytics can unlock. Probably the hottest area within digital analytics is predictive analytics. Predictive analytics essentially predict how consumers will behave given certain conditions, assumptions and stimuli. This has powerful and tangible benefits to marketing. An Aberdeen Group study published in December 2011 found marketing organizations that applied data mining and statistical modeling to optimize marketing efforts saw as much as a 2X lift from marketing campaigns, a 76% higher click-through-rate and a 73% higher sales lift.

Marketing organizations are seeing the tremendous power and potential of predictive analytics across a broad spectrum of marketing activities. For example:

• Proactively optimizing marketing campaigns to improve engagement and conversions
• Identifying customers most likely to switch companies and targeting offers to them designed to keep them as customers
• Delivering customized offers designed to appeal to a prospect’s specific interests or life situation
• Predicting which ‘first visit’ customers are most likely to return or not return, and sending offers targeting the ‘not likely to return’ group while avoiding the costs of making offers to those likely to return anyway.

In 2013, we have seen analytics continue to be a mainstream news subject. Analytics is only in the early migration phase from early adopters to mainstream use. Expect this trend to continue and accelerate in the coming years as more marketers discover analytics are a clear path to improving marketing effectiveness, efficiency, and ultimately, the bottom line.

Three Fundamentals of Great Social Media Measurement

If you want to evaluate the robustness and effectiveness of your approach to social media measurement, ask yourself these three fundamental questions:

• Does the approach measure the ‘right’ things in order to show the business impact of the programs and initiatives? 
• Will stakeholders of the report receive the data and actionable insights required to make strategic decisions?
• Are the data and insights presented in a clear and concise manner that tells a story and makes it easy to understand and act upon?

Measuring the ‘Right’ Things

Social media metrics are derived from three primary sources:

Ideally, a robust social media measurement program will have a rich metrics set that contains metrics from all three areas. Metrics tied to program objectives allow for direct measurement of program success. Fundamentally, measurement is about assessing performance against objectives. It is surprising how often social program objectives are slanted toward channel-specific metrics (e.g. Likes or Followers) and not the specific outcomes desired for the program – what you hope to accomplish by implementing the program. Also, relying too heavily on channel metrics limits you to what you can measure rather than what you should measure. Business outcome metrics are used to connect the dots between social media programs and the business results they are designed to drive. Social programs that cannot answer, or at least address, the management question, “How is this impacting my business”, are more susceptible to resource allocation scrutiny (#pleasecutmybudget). Stated another way, if management asks how we’re doing in social media and we reply, “great, post virality is up 6.1% this month”, we make it difficult for that individual to understand how social media/business initiatives are helping move the business forward.

Getting to Data and Insights that Inform Strategic Decisions

Expectations for social media measurement and analysis have risen. In addition to sound analysis and reporting of performance against key metrics and KPIs, understanding audience dynamics and developing actionable insights are rapidly becoming de rigueur. Insights may be defined as synthesizing and interpreting data to provide actionable information and knowledge that informs strategic decisions. Too many social media measurement programs take a social-centric rather than a business-centric approach to insights. They often feature insights and recommendations that are tactical in nature – the best time of day or how many times to tweet, or what type of content seems to be most successful. Ideally, insights and recommendations in social measurement reports would be operating one level above this, informing strategic decisions about how social programs and conversations are impacting, or could impact, the business. To do this requires an understanding of the business function (e.g. marketing, customer service) impacted by the social program and an ability to ask the right questions prior to starting a social media analysis. 

For example, let’s say Company X plans to introduce a new video game. A social listening program has been implemented to analyze the early consumer reaction to the game. Based on the listening analysis, changes to the packaging, marketing or even the product itself are possible. If you are in charge of the marketing campaign for the game, what are the types of social media insights you need to make decisions about the game and the marketing campaign?

• What is the level of buzz about the game?  What is the overall sentiment? How does this compare to previous game launches?
• What are people talking about in social media – availability, cost, specific features of the game, packaging, marketing campaign?
• What features of the game do consumers seem to like most?  Least? Specifically, what do they like or dislike?
• What are the most influential gaming enthusiasts saying about the product?
• Who are the promoters and detractors? What is the ratio of promoters to detractors? How does this compare to promoters and detractors from previous game launches?
• How much social media conversation contains recommendations or expresses purchase intent?  How does this compare to previous launches?

Answering these types of questions provides actionable insights that provide context and can inform strategic marketing decisions.          

Presenting Results

Dashboards have gotten a bit of a bad rap – not because dashboards are not useful, but because some have used them as THE measurement report rather than just one aspect of a good report. I’m a dashboard proponent for a few reasons:

• Deciding which metrics to feature on a dashboard is a good strategic exercise requiring you to focus on the very most important and relevant metrics for the intended audience
• Online, dynamic dashboards are an effective user interface that can be used as a launching- off point for drilling into data to understand the underlying story
• Good dashboards present a snapshot of overall performance that is easily absorbed and understood.   

A dashboard-driven social media measurement report is versatile and effective in many situations. A typical report might consist of one of more dashboards and then a deeper dive on each of the key metrics featured on the dashboards, along with audience insights, strategic insights and recommendations. This format provides a quick snapshot (dashboard) of results, ideal for those stakeholders interested only in topline data, and provides sufficient depth to satisfy those more interested in the underlying drivers of the metric  

Social media measurement programs that are built around metrics tied to business outcomes and show how programs are performing against objectives are important. Reports that deliver clear insights that inform strategic decisions are important. And delivering those reports in a compelling format that enhances usability and effectiveness is important. How do your programs stack up?

Measurement 2020 and Other Fantasies

At the 3^rd European Summit on Measurement held in Lisbon in June 2011, standardization, education, ROI and measurement ubiquity emerged as the key themes in response to a call to set the Measurement Agenda 2020.  Delegates to the conference voted on 12 priorities they thought were most important to focus on in the period leading up to 2020.  The top four vote-getters became the Measurement Agenda 2020:

1. How to measure the return on investment of public relations (89%)
2. Create and adopt global standards for social media measurement (83%)
3. Measurement of PR campaigns and programs needs to become an intrinsic part of the PR toolkit (73%)
4. Institute a client education program such that clients insist on measurement of outputs, outcomes and business results from PR programs (61%)

For a very nice overview of the Lisbon session and the Barcelona Principles that came before, read this post from Dr. David Rockland of Ketchum who chaired the Barcelona and Lisbon sessions.  David pretty much said it all on these sessions, so I’ll just add a couple of comments and share a few thoughts on what I believe the future of measurement 2020 could be.

The rallying cry coming out of Barcelona has been focused and loud – death to AVEs!  Will there be a similar thematic coming out of Lisbon and what might it be?  My money is on standardization, borne out of cross-industry cooperation.  As David points out in his post, and in the words of AMEC Chairman Mike Daniels, “The Summit identified some significant challenges for the PR profession to address by 2020.  However, what we also accomplished in Lisbon beyond setting the priorities was to harness the commitment and energy of the industry to agree what we need to do together.”  The current cooperation and collaboration between industry groups – AMEC, Institute for Public Relations, PRSA and the Council of PR Firms is unprecedented in my time in this industry and is focused on tangible outcomes.  Cross-organization committees are already at work developing standard metrics for social media measurement for example.  The spirit of cooperation is uplifting.  While the outward thematic appears to be standardization, cooperation is the enabling force.  

I was also struck by the symmetry of the call to end AVEs in Barcelona and the call to codify ways to measure ROI in Lisbon.  One follows the other.  In my opinion the primary reason AVEs exist is because PR practitioners feel pressure to prove the value of what they do, and quite often they are asked to describe the impact in financial terms.  AVEs are perceived as a path of least resistance way to express financial value.  Except, as we all know, AVEs don’t really have anything to do with the impact public relations creates.  They are a misguided proxy for financial value.  Hence the need for research-based methods to determine true return on investment.

All of the priorities coming out of Lisbon are excellent goals for the industry.  And like David Rockland, I believe they will be achieved, and be achieved before 2020.  Here are three other items on my wish list for Measurement 2020:

Word of Mouth/Word of Mouse Integration: For those of us focused in social media and other digital technologies, we can’t allow our digital lens to color what is fundamentally an analog world.  Research studies suggest the majority of word of mouth happens in real life.  From an influence perspective, I don’t think too many would argue that word of mouth from a trusted friend or family member is more powerful than word of mouse from someone you follow on Twitter.  Digital cross-platform research is difficult enough, but when one huge platform is ‘real life’, we have significant challenges in measurement.  WOMMA and others have made early attempts to define measurement approaches for offline WOM, but much work remains.  We need ways to assess its impact and then we need to think about ways to attribute value to that impact.  Mobile is a wild card here as it becomes the preferred platform for online activity.  The need to triangulate online, mobile and ‘real life’ measurement presents significant challenges today, and may still by 2020.

Cookie Wars: We all know the measurement versus privacy showdown is coming, right?  The first shots have already been fired.  The collection of source-level personal data, enabled by cookies, is crucial to measurement and insights but has the potential for misuse or unintended disclosure.  Some sophisticated consumers have had their fill of cookies.  Although the broader issue might be framed as social sharing versus privacy control, how it plays out will have a direct impact on digital analytics and measurement.

Integrated Measurement across the Paid Earned Shared Owned (PESO) Spectrum: Measurement has increasingly become integrated.  It began with integrated traditional (Earned) and social media (Shared) measurement and then progressed rapidly to Earned, Owned and Shared, which is where most integrated measurement programs are today.  Many leading-edge integrated programs today also include advertising or Paid media.  By 2020, integrated measurement across the PESO spectrum will most likely be the norm and not the exception.  A key enabling element here in my view is some base level of agreement on how each area should be measured and standard metrics for each.  It will take significant cooperation between industry groups, vendors, agencies and major customers/clients for cross-discipline standardization to move forward effectively.  We are at the beginning of this movement in 2011.  By 2020, it will be fascinating to look back and see how all this plays out.

When looking ahead to 2020, I am reminded of a measurement discussion pulled together by PRWeek a couple of years ago.  Many of the Measurati attended.  In response to a question of where measurement will be in five years, David Rockland replied (paraphrasing here), ‘Who knows?  Five years ago who would have guessed we would all be focused on how to measure social media?’  So, there is a certain fantasy element to discussing 2020 challenges in measurement.  What are your measurement fantasies?

How Much Does a House Cost?

I don’t come from the “there are no dumb questions” school.  For example, in an academic environment, I would define a ‘dumb question’ as one in which the answer should be easily known had the student read the assignment or attended the previous class.  There are a lot of dumb questions asked all the time and social media gets more than its share of these.  Many of them are specific to social media measurement/ROI.  For example:

• Which has higher ROI, Twitter or Facebook?
• What ROI should I expect from Twitter?
• How do I measure the ROI of social media?

The flip answer to all these questions is, it depends.  All results are contextual.  Results are also specific.  While industry averages may be interesting, averages mask any real meaning for an individual brand or company.  They result in ‘one size fits none’ thinking.  Let’s go back to our house analogy and bring this to life.  The cost of a house depends on several factors:

• Where is the house located?  You’ll need to know the city and the specific neighborhood.  You may also want to know which block the house is on within a given neighborhood.
• How large is the house in terms of square feet?
• How large is the lot?
• Is the house new or previously owned?
• In what condition is the house?
• What is the level of finish-out?  For example, granite versus tile countertops.  High-end appliances or mid-range?
• What are the desirable or unique features of the house?

In social media measurement we have our own list of questions to ask before attempting to answer generally stated questions about measurement and ROI:

• What brand/company are we speaking about?  The answers for a well-established cult brand will be very different from those of a less well-established brand.  Answers for eCommerce companies will vary from those of B2B companies.  Answers will also vary by industry segment.
• How long has the brand/company been participating in social networks?
• How much investment in social media marketing – time and money – has the brand/company made?  What has been the level of effort?
• What other communications channels (e.g. advertising, direct, search, public relations) are being utilized in parallel with social marketing?
• What is our point of view on the role of social media in the marketing mix?  For example, is the role of social media primarily to drive exposure to content or is the program or initiative designed to drive conversion events through social channels?
• What were/are the specific objectives of the program or initiative?

This last question is especially important because measurement is fundamentally about assessing performance against stated objectives.  When someone asks you how to measure something in social media your first response should always be this question – What were the specific objectives of the program or initiative?

The question of when to expect a return on social media efforts is also an interesting one.  Brands often expect an immediate ROI on social media efforts.  Social media marketing is a process not an event.  Too often people forget about the ‘I’ aspect of ROI – you usually have to make an investment in resources and time before you can drive a return.  It is wise to listen to social conversations before engaging, and build your presence and trust before trying to drive conversion events.  Listen and learn and then convert.  I would argue the majority of social media efforts today are likely in the investment phase and not the return phase.  It is somewhat unfair in these cases for the social media effort to be held to an ROI standard in the short-term.  Measuring impact rather than ROI is advised.  Perhaps we can add another question to our list of dumb social media ROI questions – ‘What ROI should we expect in the first year of our social media initiative?’

If you are one of the prescient humans who has a crystal ball that enables you to answer the ‘how much does a house cost’ question, I have another question for you, ‘how long is a string?’

Social Media ROI Twitterchat

Yesterday I was the guest on Shonali Burke’s #measurepr twitterchat.  The subject was Social Media ROI.  The conversation was lively and engaging.  I found it exciting and stimulating, and appreciate Shonali letting me share with all of you.  Shonali did a great wrap-up of the session on her Waxing Unlyrical blog.  You can read it here. You can also download a transcript of the chat here:  #measurepr transcript 8.31

Shonali was kind enough to invite me back for another round.  So please join us for Social Media ROI Round II.  The date is September 14.  The time is 12:00 – 1:00PM (Eastern).  DM Shonali (@Shonali) or shoot her an email if you have a question you would like us to address.  You can sign-up for the chat here, or just join us on the 14th. using #measurepr.  Hope you can join us!  – @Donbart

Don’t Let the Tool Tail Wag the Measurement Dog

Social media listening and measurement tools are sexy.  Well, at least to those of us in research and measurement – it’s all relative right?  In the last three years or so there has been an explosion of social media tool vendors and platform choices.  Tools are sexy and important, but in the grand scheme of things are being overemphasized to some degree.  We are letting tools decide what we can measure without giving sufficient thought to what we should measure.  We are letting the tool tail wag the measurement dog.

There are several steps and decisions that should be addressed prior to selecting a tool or suite of tools.  Consider this diagram as a starting point to help you think through these interim considerations and decisions:

OBJECTIVES

Proper social media objectives should be measurable (indicate change in metric of interest and timeframe) and aligned with desired organizational outcomes.  Understanding the social media objectives will suggest broad parameters the measurement program, and ultimately the tool decision, must operate within.  For example, geographic coverage requirements, type of content to be considered and on-platform engagement capability may all be strongly suggested based on a review of social media objectives.

PROCESS

In addition to comprehending organizational or business outcomes, it is essential to understand the business process the social media program will address or drive.  If the program is marketing oriented, the sales funnel process (Awareness/Consideration/Preference/Sales/Loyalty) may be most appropriate.  For a brand-building campaign, the brand pyramid (Presence/Relevance/Performance/Advantage/Bonding) is what you want to measure your program impact against.  Other business processes that are commonly addressed by social media programs include customer service and support, CRM, corporate reputation and lead generation.

METRICS

Understanding the requisite business process the social media program is driving is crucial because each business process drives specific metrics.  For example, the sales funnel drives a specific metrics set:  percentage of unaided or aided awareness; percentage of the target audience who would consider the product/company; percentage who prefer the product/company; incremental sales revenues; percentage who would purchase the product again number or the number/amount of repeat purchases.  For B2B companies, the lead generation process would drive a different set of metrics: number of incoming leads; percentage/number of qualified leads; lead conversion rate; sales revenues generated.  In addition to the business process metric sets, there are other metrics areas like Exposure and Engagement we will want to address.  Reach/opportunities to see, share of positive discussion, comments/post ratio, number of @ mentions and RTs per 1000 followers are examples of ‘standard’ metrics that might be applicable for many social media programs.

Understanding how the social media program drives a specific business process is also important to our ability to describe the impact or, in some cases, return on investment the program has created.

DATA SETS

Each metric has data requirements, usually two pieces of data per metric – a numerator and a denominator.  Examine the set of metrics you have defined for your social media program.  Catalog all the specific pieces of data you need to compute the various metrics.  For example, the data needed to compute the basic sales funnel metrics and some ‘standard’ metrics might include:

• Number of individuals in the target audience
• Number of survey respondents
• Number of respondents ‘aware’ of the product/company
• Number of respondents who would consider/seriously consider purchasing the product/doing business with the company
• Number of respondents purchasing the product
• Amount of sales revenue directly attributable to the program
• Number of purchasers who purchased again
• Total branded mentions
• Volume of positive and negative mentions
• Number of posts
• Number of comments
• Number of RTs and @ mentions
• Number of followers

TOOLS

Armed with an understanding of all the data needed to calculate the metrics required to measure the social media program, you will be able to assess which tools or classes of tools best deliver the data you need.  Pick the best three to five tools for further evaluation.  You most likely will find no one tool can deliver the complete data set you need.  It is common to need two or more tools, e.g. web analytics package and social content analysis platform, in order to fully meet data requirements.  Budgetary constraints may also limit your ability to capture the entire data set required.

By addressing the interim steps leading up to tool selection, you will be able to make a more informed tool decision.  You also will have a much better chance of measuring what you should measure rather than settling for what you can measure.  No tool before its time.  Let the big dogs run.

Public Relations Measurement 2010: Five Things to Forget & Five Things to Learn

(This post is a re-purposing of a speech I gave to the FPRA/PRSA-Orlando on July 23, 2009.  You can download the slides here.)

Public relations measurement is at a crossroads.  Old techniques are no longer sufficient.  Old metrics are no longer applicable.  Old thinking must be replaced by new.  The need for accountability, and to prove the value of PR and social media programs, has never been greater.

As we look to the next year, here are five things to forget and five things to learn about public relations measurement in 2010.

Things to Forget in 2010

1. Media Relations Focus

A focus on media relations fails to capture several important aspects of PR – brand, reputation, crisis, employee communication and DTC to name a few.  Also, the importance of traditional media is declining.  Numerous studies have shown people don’t trust what they read in the media, they trust each other.  I believe it was Hauser and Katz who coined the term ‘you are what you measure’ in 1998.  If measurement is focused on media relations that is how the public relations function will be judged.

__________________________________________________________

2. Outputs

The need to put PR results in a business context has never been greater.  We need to be able to address the question – what are we doing to help drive the business?  If you are focused on output metrics like impressions or message delivery, you will always have a hard time explaining business impact.  Instead, we need to focus on outcomes and answer the question – what happened as a result of our program or coverage?  Understanding outputs has primary benefit as a diagnostic tool rather than a ‘scorecard’.

__________________________________________________________

3. Impressions (and Multipliers!)

The most common PR metric today is Impressions.  While it is a somewhat dubious metric for traditional media, it really loses meaning in social media where engagement not eyeballs is what we seek.  Impressions also (greatly) overstate actual relevant audience.  Generally only a fraction of any particular magazine or newspaper’s circulation meets your target audience demographics.  And impressions merely represent an opportunity to see, they do not attempt to estimate the (small) percentage of the potential audience that actually saw your content.  To compound the problems, many PR practitioners use a multiplier on impression numbers to account for pass-along readership or a mythical credibility advantage PR has over other communication tools.  The simple fact is there is no factual basis (e.g. research proof) that multipliers should be used in any case.

__________________________________________________________

4. Ad Equivalency (AVEs)
There are many reasons why using ad equivalency as a proxy for PR value is not advisable.  Here are five good reasons they should be avoided:

• AVE calculations vary and there are no standards.  Tonality, article length, competitive mentions and other factors are handled differently.
• AVE results can be misleading.  AVEs may be trending up while metrics like message communication, share of favorable positioning and share of positive press are falling.
• AVEs reduce PR to just the media dimension by only assigning a value in this area.
• AVEs only apply to traditional media.  What is the AVE of a positive conversation about your company on a leading blog?
• How much is it worth for a troubled company to not appear in the Wall Street Journal?  AVEs cannot address this.

__________________________________________________________

5. Return on (Engagement/Influence/etc.)
Not a day goes by on Twitter without someone declaring a new and improved metric for the acronym ROI, or stating that ROI does not apply in social networks.  Wrong and wrong.  Most of these folks either don’t understand ROI or don’t know how to obtain the data necessary to calculate it.   There is also a lot of confusion between creating value and ROI.  Generating awareness creates value, for example, but may not immediately result in demonstrable ROI.

__________________________________________________________

Things to Learn in 2010

1. Total Value of PR

The majority of current PR measurement efforts focus on marketing/sales and output metrics.  The Total Value Cube is a way to visualize and think about all the potential value your PR and social media efforts deliver.  Beyond marketing to include brand and reputation, beyond outputs to include engagement, influence and action, and beyond revenue generation to include cost savings and cost avoidance.

__________________________________________________________

2. A New Model for Measurement
Many public relations practitioners regularly get their Outputs confused with their Outtakes or Outcomes.  Outtakes is not often used in the U.S. – it seems much more prevalent in Europe.  The overall terminology is confusing and is defined in different ways by different practitioners.  Further compounding the confusion is the fact audiences we present our results to rarely understand the terms and have trouble relating to them.  In short, the terms are too much ‘inside baseball’.

What we need is a metrics taxonomy that is easier to understand and explain.  I like this one.

Exposure – to what degree have we created exposure to content and message?

Engagement – who, how and where are people interacting/engaging with our content?

Influence – the degree to which exposure and engagement have influenced perceptions and attitudes

Action – as a result of the PR/social media effort, what actions if any has the target taken?”

__________________________________________________________

3. Three Zones of Measurement

From the left, companies or brands control, own or manage websites  – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with.  This zone is measured primarily by web analytics.  In the middle are the actual social networks and conversations between individuals.   In this zone we are interested in data sets that cannot be gathered solely using web analytics packages.  How often is the brand being mentioned in conversation?  What is the sentiment of the comments?  How often is the brand being recommended and by whom?  Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone.  The third zone represents all the real-world, offline transactions that may be of interest.  Did someone visit the store or attend or event?  Did they buy a product?  Did they recommend the brand or product to a friend over coffee?  Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.

Your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones.  The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.

__________________________________________________________

4. New Metrics

_________________________________________________________

5. The Difference Between Impact/Value and ROI
ROI is a form of value/impact, but not all value takes form of ROI.  ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.  ROI lives on the income statement in business terms.

Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product.  Value creation occurs over time, not at a point in time.  Value creation is process-oriented.  Value lives on the balance sheet.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

Inflationary Twitter Audience Numbers Hurt Social Media Credibility

In yesterday’s New York Times, you may have read the article, Spinning the Web: P.R. in Silicon Valley, an interesting although not overly insightful piece.  From a social media measurement perspective, two items caught my eye.  The first, referring to Brian Solis, Principal of FutureWorks, about how he calculates social media audience figures:

“Instead of calculating the impressions an article gets by estimating a publication’s circulation and pass-along rate, Mr. Solis counts the number of people who tweeted about a company and their combined following, the number of retweets or clicks on links, as well as traffic from Facebook and other social networks.”

Toward the end of the article, we learn:

“By 6:30 p.m. on the day Wordnik went live, Brew’s staff calculated that 1.43 million people had seen tweets about it.”

Setting aside for a moment that the article and these sorts of audience metrics take a broadcast-oriented view of Twitter (Mr. Solis discusses the shortcomings of the NYT viewpoint here), the emerging view of audience measures for Twitter is to calculate the Followers of each person tweeting about the subject of interest, and then adding Follower numbers for each person retweeting the subject and so on.  The issue here, much as it is in traditional public relations, is that the audience figure that results from these sorts of calculations grossly overstates, by one or two orders of magnitude or more, the actual “audience” for these tweets.  It is a hypothetical number that assumes everyone that possibly could see a tweet has in fact seen it, and everyone who sees it is relevant to you/your brand.  This is fantasy of course.

On the issue of relevant audience, here is a quick example.  At the time I pulled these figures, the audited circulation of the New York Times was 4,974,000.  Most PR practitioners getting a ‘hit’ in the NYT would claim this as their audience.  However:

• If you were only trying to reach a C-Suite audience with your message, the actual audience reached would be 598,000 or 12% of the total circulation
• If you were trying to reach Women, your audience would be 1,937,000 or 39% of the total
• If you were trying to reach 25 – 54 year old Men, your potential audience would have been 2,930,000, or 59% of the total number.

There is a large difference between how many people theoretically can see a tweet, versus how many actually saw it/read it, versus how many of those seeing the tweet find it relevant to them, versus how many engaged with it by hitting a link or retweeting.  Part of my issue with this is the language we use to report the figures.  For the Brew staff to use these numbers to estimate 1.43 million people “had seen tweets about it” is wrong.  If they had said 1.43 million people had an opportunity to see the tweet, it would have been more realistic, although still greatly overstating actual relevant audience.

This problem of audience inflation has already been institutionalized in public relations.  The use of Impressions as an output metric does not mean a true impression in the branding sense, but rather an opportunity to see the content.  To make matters worse, many PR practitioners believe Impressions should be factored by either dubious pass-along readership figures and/or use of a multiplier to account for the mythical credibility advantage PR enjoys over impressions generated from advertising.  The simple fact is there is no research-supported, fact-based argument for using any adder or multiplier in public relations when calculating potential audience (here’s an IPR white paper on this subject I co-authored).

For Twitter and other social networks we lack demographics and data about tweet readership averages (i.e. what is the probability that any one tweet is actually read) that would allow for more precise audience estimates.  In the absence of data, believable assumptions should be used:

• Out of all the opportunities to see, how many actually read the tweet?  10%?
• Of those reading the tweet, how many find it relevant to them (or from the other perspective, how many of the readers are in your intended target audience)?  Maybe 10% again?

You can see how our audience estimate has already been reduced by a factor of 100.  This may well still be overstating the actual, relevant audience.  The issue here is that unrealistic and overstated audience figures have the potential to hurt credibility and call into question other data and metrics that may be more grounded in fact.  Actually the more meaningful metrics pertain to engagement or outcomes rather than exposure/outputs.  It is more meaningful that 40,000 visited the Wordnik website as a result of the campaign discussed in the NYT article than the overstated 1.43 million who were estimated to have seen the tweets.  40,000 is real.  1.43 million is fantasy.

A 30,000-Foot View of Social Media Measurement

Look back five years and the PR measurement field was full of challenges:

- Emphasis on media relations at the exclusion of other high-value PR activities, almost always
- Oriented toward outputs and not outcomes, consisting
- Primarily of media content analysis, with
- Little primary audience research, and
- No codified thinking on how to approach ROI determination.

Now add social media.  Old metrics like Impressions lose meaning.   It’s about engagement and not eyeballs.  Consumers have broad platforms to voice opinions about your brand.  Conversations are more effective than messages.  So needless to say, social media measurement is a highly fluid, and rapidly evolving field.  Lots of opinions, not much consensus.  Here is where I believe we are at a high level.

Early efforts to measure digital and social media focused almost exclusively on web analytics.  I would say the majority (80%?) of social media measurement in 2009 still focuses on web analytics, although many other forms of data and research are being used by leading organizations and practitioners.

Today, the frontier in social media measurement is evolving toward measuring the conversations and behavior patterns occurring within social networks.  The third area of interest is in tracking and connecting online and offline behavior and actions.  Here is a simple graphic (you may have a much better way of showing this) that shows these three primary interest areas, or zones, for social media measurement.

From the left, companies or brands control, own or manage websites  – corporate sites, FaceBook pages, Twitter accounts, LinkedIn pages and blogs by way of example – and create content that consumers may engage with.  This zone is measured primarily by web analytics.  In the middle are the actual social networks and conversations between individuals.   In this zone we are interested in data sets that cannot be gathered solely using web analytics packages.  How often is the brand being mentioned in conversation?  What is the sentiment of the comments?  How often is the brand being recommended and by whom?  Content and behavior analysis, including tracking technologies, are the primary measurement tools in this zone.  The third zone represents all the real-world, offline transactions that may be of interest.  Did someone visit the store or attend or event?  Did they buy a product?  Did they recommend the brand or product to a friend over coffee?  Primary audience research is necessary to address many of the questions, as well as scan or other purchase data in some cases.

Although I have attempted to define three distinct zones of measurement necessary to address the full spectrum of social media impact and ROI, your measurement strategy should be to take a holistic, integrated approach using methodologies, tools and data from all three zones.  The Holy Grail in many ways is to be able to track behavior of individuals across all three zones, cross-platform, understanding how online behavior impacts offline behavior and vice-versa.  It won’t take five years to get there.

The Difference Between Value and ROI

Social media and public relations programs create value and in some cases generate demonstrable ROI.  The two concepts are different in important ways.  They are related like the rectangle and the square.  Remember that silly distinction you learned in elementary school?  A square is a rectangle, but a rectangle is not a square.  ROI is a form of value, but not all value takes the form of ROI.

ROI is a financial metric – percentage of dollars returned for a given investment/cost.  The dollars may be revenue generated, dollars saved or spending avoided.  ROI is transactional.  ROI lives on the income statement in business terms.

Value is created when people become aware of us, engage with our content or brand ambassadors, are influenced by this engagement, and take some action like recommending to a friend or buying our product.  Value creation occurs over time, not at a point in time.  Value creation is process-oriented.  Value lives on the balance sheet.

From a sales process perspective, the ultimate value of a social media program may be in increasing the number of people who are likely to buy our products and services.  Other programs may be designed to improve or protect corporate reputation or to build and enhance brands.  Much of this value is said to be intangible.  It is goodwill that becomes tangible at the point in time a transaction occurs.   When buying decisions happen, your investments in marketing, brand and reputation work together.  They become tangible.  You can measure the ROI.

Many of the well-intentioned but misguided attempts to rename or reinvent what ROI means in social media – return on influence and return on engagement probably getting the most play – seem to be the result of an inability to distinguish value creation from ROI.  We know social networks have the ability to create value through customer engagement and community building.  However, ROI can only be measured by their ultimate impact on downstream metrics like sales, employee retention and customer loyalty/repeat purchase.

Your investments in social media or public relations remain an investment, creating additional value if done correctly, until which time they can be linked to a business outcome transaction that results in ROI.

Trying to get, keep or increase your budget for social tools, people and programs?   Estimate ROI where you can, but also try to articulate the value your programs will be creating, and how this value aligns with, and contributes toward achieving one or more desired business outcomes.  Propose metrics to track and assess progress in exposure, engagement and audience influence.  This is a better conversation to have than, “Let me tell you about Return on Influence…”

Make sense?