A carpenters' ruler with centimetre divisionsImage via Wikipedia

I’ve had the pleasure and honor of getting to know and working with a handful of social media measurement firms. For disclosure’s sake, Doe-Anderson presently uses Radian6 for one client as well as agency projects and my friends at Beam Global Spirits & Wine use Collective Intellect, which I have access to on their behalf. However, I’ve gotten to know and have become very familiar with a number of social media measurement firms and people over the last year including Nielsen Online, BuzzLogic, Cymfony, MotiveQuest and more. One of my favorite people to spar with on the issue is, of course, the measurement diva herself, K.D. Paine.

So in the course of conversation last week, I had an interesting back-and-forth with her that sparked some thoughts on social media measurement. It all started with a client’s question on whether or not we could measure social media impressions. I rhetorically asked Paine to chime in, knowing I’d get a passionate response (I did, more in a moment). My response to the client was, in essence, this:

It’s not possible to record impressions in social media. Social media is about conversations. No matter how much we market, if people want to talk about Barak Obama, gas prices or Janet Jackson’s nipples we may be 100-percent relevant and top of mind to 90% of the people, but we aren’t going to get any conversational love. So we then turn to eyeballs on our placements. But that’s impossible, too. The simple reason is that websites don’t have to share their traffic numbers. We can use projections like Alexa or Compete, but those projections are drawn from sample numbers of tech-savvy people smart enough to download their plugins and allow them to monitor one’s web behavior. The numbers are skewed to the geek community. We can nail placements. We can nail number of conversations (with a +/- error range as some sites set themselves to not be searchable). Of those conversations, we can nail sentiment and tone. We can plot it all out month-to-month, day-to-day, etc., and see how much better or worse we’ve gotten. We can also measure how much web traffic we’re getting, how many people are linking to us, not just mentioning us, etc. All of these are valid measurements of social media success. We need to prioritize which ones are success to us and focus on them.

Paine responded with some very solid points, most of which supported my response (shew!). The best was when she said, “The only way to accurate calculate impressions for blogs is to ask each blogger who’s site your brand appears on how many unique visitors he/she gets in a month. And even then, you’re assuming they’ll tell you the truth.”

As our discussion continued, I offered up this little number, partially to gauge a response, but also because I believe it to be fairly accurate:

“Nothing I’ve seen can’t be automated by a well-polished algorithm with a monthly report that takes one smart person about 3-4 hours to put together, glean a couple of insights and slap a few copy-paste quotes from some blogs and you’re done. And yet every firm I’ve seen wants a brand to pay tens, even hundreds of thousands of dollars for that? Give me a good software application developer and the API for del.icio.us and I’ll kick the big firms’ asses in a month.”

And yes, I linked to all those firms up there specifically so they would see and respond to that paragraph.

I went on to say:

“Big brands want deep-dive information without having to do any work. But they want deep-dive because there is not a hard, fast number we can circle and say, “1,547 here means we made an additional $46.78 there.” The measurement business is an illusion to them. They have no clue what the numbers mean, how they relate to success and, in the end, they can throw them all out and just say, ‘We did some cool sh*t on-line, too,” and their CMO/CEO will be happy, so long as the advertising campaigns lead to sales growth.”

Of course, Paine chimed back that she would argue that social media absolutely can draw a line to what impact it has. She pointed to the ASPCA and Mothers Against Drunk Driving and can point to how many new members and how many donations come in from what they do in social media. She also pointed to engagement, click thrus and sales as correlative support. My retort is that social media often happens off a brand’s website and can’t be quantified in direct numbers, sales or otherwise. My metaphor was that I talk about brands of bourbon in online conversations almost everyday. Does that mean every person who sees or participates in a discussion about them goes out and buys a bottle or a cocktail? Nope. Do I have any way of knowing how many do? Nope. And you can’t buy liquor from the brand online, so there goes some more of the argument.

In all, we had a great exchange on a hot topic and I’m indebted to her for her time and patience with my skepticism. I’ve met some great people who know a lot about measurement. K.D. Paine is certainly a prize contact.

And, yes, she smacked my hand pretty good on minimizing the work that goes into good social media analysis saying, “I doubt very much your developer can accomplish what yo have in mind in a week.” She’s right. And, for the record, I did include the line, “I’m not trying to minimize your work,” in my explanation.

Which brings me to what I think might be the best analogy we in social media can give to those of a more traditional advertising, marketing and public relations background when it comes to social media. Paine’s, “research geek,” Paul Kowalski said:

“There is a place where the measurement stops and, for branding efforts, it stops at measurements of brand equity. When an architect builds a building, what are her measures of success? Stock price? That would be ludicrous. Did she make a building? Yes. Can people work in it? Yes. Done. For brand architects, did you build the brand to spec? Are people who perceive your brand more likely to buy more stuff (or do whatever)? Stop. No further. Because anything past that, like calculating the impact on stock price of a building people love to work in, is what your friend so aptly called the illusion.”

Bravo!

For the record, K.D. Paine and Partners is one of the leading social media measurement firms in the world. They have an assortment of offerings for business of nearly any size and budget, including the DIY Gold Dashboard which is a powerful tool. They can be found online at http://www.kdpaine.com. Ms. Paine blogs herself at http://kdpaine.blogs.com and if you’re not following her on Twitter, you should. She’s @kdpaine.

Zemanta Pixie
Did you enjoy this blog post? If so, then why not:Leave Comment Below | Subscribe To This Blog | Sign Up For Our Newsletter |

About Jason Falls

Jason Falls

Jason Falls is a leading thinker, speaker and strategist in the world of digital marketing and is co-author of two books, No Bullshit Social Media: The All-Business, No-Hype Guide To Social Media Marketing and The Rebel's Guide To Email Marketing. By day, he leads digital strategy for Elasticity, one of the world's most innovative digital marketing and public relations firms. Follow him on Twitter (@JasonFalls).

Other posts by

Comments & Reactions

Comments Policy

Comments on Social Media Explorer are open to anyone. However, I will remove any comment that is disrespectful and not in the spirit of intelligent discourse. You are welcome to leave links to content relevant to the conversation, but I reserve the right to remove it if I don't see the relevancy. Be nice, have fun. Fair?

  • http://www.radian6.com Marcel LeBrun

    Hi Jason,
    I'm glad you covered the discussion here in this post. I've been getting to know Katie this year, as we have started doing some work together, and she certainly knows her stuff in this space. I have often heard her remind people of the “it depends” nature of measurement.

    The fact is, social media is increasingly being used for many different purposes. Some aspects are very measurable, depending on what you are trying to do, but some things are too tacit, indirect or chaotic to measure. I like the last quote you included from Paul Kowalsi which makes this point very well (as you do in your post).

    Some things, however, are easy to measure. For instance, if a brand intends to engage with customers using social media with a goal to answer inquiries or solve customer issues, then it is easy to measure. We can track exactly how many issues or inquiries were found, how many issues were resolved (or questions answered) and what it cost to give these customers excellent customer service (and easily compare the cost to other traditional channels like the company's telephone help desk, etc.). Now if you want to link that investment in social media customer engagement to stock price, well that's another story – and it is just as difficult as linking any traditional investment in customer service to stock price. Smart companies know that good customer service is critical, but it is difficult to quantify the direct cause/effect since there are so many factors involved.

    The thing that bugs me a bit about some of the conversation around “social media measurement” is that it seems to put social media in a box as if it is only a single purpose medium – as a kind of alternative to advertising. The truth is that the social web is much broader in its impact on business. Several business processes are being impacted: customer service, PR, advertising, sales/lead generation, etc.

    If you step back for a minute, it is easy to see how the social web is really becoming a new multi-purpose communications medium.

    While this might sound silly, when a business invests in a telephone system, it is used by the sales department, customer service department, PR and executives alike… all for different purposes. All of them are measured differently (and, yes, they are measured).

    The medium itself is generally not the primary focus of measurement, but rather the business function being performed using the medium.

    As the use of social media becomes more pervasive (across all business functions), we need to broaden the measurement discussion as well.

    Jason, as a PR and social media guy working an an advertising agency, do you run into this issue when talking to clients about social media measurement (i.e. one guy is thinking about it as an alternative form of advertising and, therefore, thinking impressions, etc,, when another is really talking about it as a way to build relationships with influencers, etc.)?

  • http://socialmediaexplorer.com JasonFalls

    Bravo, my good man. Excellent points here. (For those reading, Marcel is the CEO at Radian6 and a friend whose Ski-Doo I once crashed.)

    Yes, clients think about social media measurement differently and yes, it's hard to cover it all with them in a fashion where they see the light. But discussions like this always help me get closer.

    But I'm excited by your “sounds silly” part … it's not silly at all. Comparing social media to a telephone system so that people understand it's a communications medium, not a singular bucket that one department does is, frankly, brilliant. It presents a great way to explain it.

    Now, there will be some who say, “Yeah, but you want social media to be the responsibility of PR,” referring to my post of a couple weeks back. Yes, I do, but not siloed like everyone assumes. PR as the chief communications arm of an organization should be responsible for implementation of strategy, training, etc. — kind of like showing people how to use the phone and voice mail system properly, teaching them speed dial, replacing bad handsets, etc. — not the only department allowed to place calls.

    Excellent metaphore, Mr. Lebrun. Apparently all that hockey hasn't knocked the sense out of you yet. Thanks a million!

  • http://blog.techrigy.com MartinEdic

    I find myself agreeing with Marcel, a competitor of ours. The critical difference, which those seeking the Holy Grail of ROI, often fail to grasp is that social media is not static information like the first iteration of the web. It is a communication layer. Measurement is a lot more complex than tapping into a API and pulling out numbers. Sentiment, demographics, location, authority- these things don't come out of an API call. We pull up to 35 different pieces of data for each search result in SM2 and then we supply extensive tools for understanding how that fits together. You not only want to understand the conversation, you also need to understand who's doing the talking and who, in turn, is listening.
    There's another dynamic that measurement must deal with: the dynamism of popularity in social media. Tools like Buzzmetrics that ignore lower authority sources are a problem because those low authority sources can break a story or start a brushfire just as easily as the big media sites. With exponential, one to many communication, social measurement can get very complex, both to monitor and engage.

    • http://socialmediaexplorer.com JasonFalls

      Thanks Martin. Very good points to consider. Please don't hesitate to let us know who you're with, though. My apologies for not knowing off the top of my head. Relevant links to your service are welcome.

      • http://blog.techrigy.com MartinEdic

        Sorry Jason, I try not to get too overt in my engagement! I'm Director of
        Marketing for Techrigy, developers of SM2, our social media monitoring and
        measurement tool. We help people manage brands and reputation across the
        social media eco-system including blogs, wikis, forums, microblogs, social
        networks, user-generated content and more. We also provide a complete suite
        of analysis tools including sentiment, demographics (age, gender,
        geo-location), authority ranking, trend and theme analysis with extensive
        charting tools and reporting options. Users can configure and export over 30
        data fields from our results. There is a free, fully functional version
        (limited in the number of keywords and search results) available at
        http://sm2.techrigy.com.Does that answer your question? Glad to join the
        dialog.

        • http://socialmediaexplorer.com JasonFalls

          That clears it up. Thanks for letting us know about your tool. I'll have to dive in and check it out.

  • http://www.radian6.com Marcel LeBrun

    Hey Jason,
    I like your extension of the “phone” analogy regarding your view on PR owning social media. The unique characteristics of the social web as a communications medium definitely make it different from other media and I certainly agree that PR has the right skills to provide leadership here.

    For instance, customer service front line staff today are entrusted to be the voice of their brand in one-on-one conversations with customers who call in with questions, problems, etc. They are generally provided with training, guidelines, etc. But, the social medium is different. Conversations with customers on the social web happen in full public view and with a permanent record of the conversation. This does require a clear strategy, guidelines and training which PR is well positioned to provide to the rest of the organization.

    I really like the clarity though that PR shouldn't be the only one allowed to place calls, but rather be responsible for strategy, training, guidelines, etc…. showing people how to properly behave using the “social phone”.

  • http://www.stickyfigure.com Steve Woodruff

    If someone asked me to an ROI calculations on the various interactions I have in my father-son relationships, I'd be hard-pressed to come up with the data. This much I DO know, however: success is more likely by engaging in conversation and interaction with my kids. Failure is likely when it's neglected. That's good enough for me – in family relationships and in business networking.

    • http://socialmediaexplorer.com JasonFalls

      Bravo, Steve. Another great illustration of the crux we're in when talking about measurement. And on that note, I should go call (on the phone) a few folks to just say “hey” and check in. Thank you, sir.

  • http://blog.techrigy.com MartinEdic

    I was just commenting elsewhere (not Disqus, unfortunately) on the use of SM measurement tools by government agencies and several commenters brought up the 'big brother' issue. As Marcel notes you should remember that public participation in social media is just that: public (one to many). You don't have the protections that things like phone conversations (one to one). Companies like Techrigy and Radian6 are not spying, we are observing and collecting publicly available information. I suspect there is going to be a lot of discussion when government agencies start routinely using these tools.

  • http://www.24pagebooks.com MartinEdic

    I was just commenting elsewhere (not Disqus, unfortunately) on the use of SM measurement tools by government agencies and several commenters brought up the 'big brother' issue. As Marcel notes you should remember that public participation in social media is just that: public (one to many). You don't have the protections that things like phone conversations (one to one). Companies like Techrigy and Radian6 are not spying, we are observing and collecting publicly available information. I suspect there is going to be a lot of discussion when government agencies start routinely using these tools.

  • Pingback: Recent Links on Ma.gnolia at Fast Wonder Blog: Consulting, Online Communities, and Social Media

  • Pingback: Making an impression on social media measurement | Liberate Media - social media and online PR consultancy

  • Pingback: Impressions of and In Social Media Measurement « Media Bullseye – A New Media and Communications Magazine