Is social media measurement really meaningless?

I’ve been reading and re-reading this article over the past few days and trying to fully understand Amy Kean of the IAB’s point of view who argues the ‘Yes’ side of the debate that social media measurement is indeed meaningless. I can understand her point of view but I’m of the ilk that believe in a different kind of measurement and that adopting the traditional ideologies of past successes is akin to putting a square peg in a round hole. I’ve said before that although ROI will always have its place (everyone will always wants to get the most bang from their buck), ROA (Return on Attention) will grow in importance in social media. You’re still getting a return on investment but in a different way.

Back to the article, the busker analogy doesn’t really work for me when comparing thumbs up to positive buzz and great reviews for a company. They aren’t directly comparable. For someone to go out of their way to generate positive buzz on a brands behalf or give a product a great review, there is value in that. Time spent plus insights provided, that’s definite value. Research also states that their message often has more impact than paid for marketing and communications.

It may not be in the traditional marketing sense of what ‘value’ is but for me, social media requires new metrics and benchmarks than to what has been tried and tested in years previous. Success now looks different. Instead of looking purely from a numbers perspective and at additional sales, is there value in increased levels of brand perception and awareness, along with the conversations and connections made because of the company activity? It’s intangible yet it’s still valuable.

Amy says that most senior management simply don’t understand it. While this may be a sweeping generalization across the board, there is some truth in it. This for me is where the paradigm of business control is changing. This won’t be for everyone, only the ones who are open to change, evolution and welcome new blood taking their company forward and in a new direction. The MD’s, the board members, all the ladies and gents high up who have been used to knowing everything they need to in order to take their business forward, are now bringing in new talent to change the culture, operations and future of the company. The MD will still be the MD but he/she has realised that a new set of skills is required for the ever changing business environment.  Edit – I had previously mentioned about Jean Wyllie leaving PN but as Kerry notes in the comments below this was out of context, my apologies.

Back to the article again, social media isn’t about directly increasing sales over a finite period. It’s more longer term than that, investing in and developing relationships. Potentially fruitful relationships that are seen to have a direct effect maybe not in sales but in other benchmarks like buzz / word of mouth. Let’s take the recent collaboration between BitchBuzz and Magners UK.

If you hadn’t heard about it, Magners UK decided to give away a case of its pear cider every day for a week on BitchBuzz. The target audience and the product was a perfect fit. The competition was cleverly integrated with Facebook Connect and as a result of the week long campaign, delivered approx 100 new fans to the Magners UK Facebook page every day for the duration of the campaign. They achieved over 8 times the fan conversion they had when supporting the giveaways based on engagement on Twitter and Facebook alone. Their objectives were simple, to build the community. The competition accounted for almost 40% of total Facebook fans.

Can you put a traditional value on that? No. What was the ROI on that? I don’t know. Is it important? Yes.

Amy also says that part of the role of the IAB’s social media council is to educate the industry about the opportunities in this space. Yet she’s declaring measurement meaningless? Sounds more like discounting a discipline rather than educating about it.

Will, Robin, Stephen, David and Drew what do you think? Is social media measurement meaningless? They are all much more knowledgable and experienced in this world than I am, so would be great to get their take on it all. Oh and feel free to disagree entirely if you think otherwise, interested to hear from other perspectives.

39 thoughts on “Is social media measurement really meaningless?”

  1. Good post. Like real life the variation between depth of engagement from second to second is too hard to pragmatically measure within social media but as you’ve written a quality post I’ve started to make an attempt to write my digital media metrics post that has been languishing in my head for over a year or so. It’s all your fault I’m attempting to even write it. More soon :)

  2. First of all thank you for a most timely and stimulating post about an issue that concerns us all.

    Social media metrics are still in their infancy and I believe we are going to witness some tangible improvements in the short term. Having said that, the concept of ROA is not without value. This is what I think David Meerman Scott is at least partially referring to when he talks about the demise of ROI in the social media: it is not about selling units but about participating in the conversation and taking it to new ‘viral’ levels.

    To my mind traditional ROI still has a part to play, but not in the strict sense we have got used to in ‘traditional’ offline marketing and advertising. The same argument could of course be applied to traditional accounting, with the advent of the so-called ‘social’ and ‘environmental’ accounting.

    In the end, we are all ready and expectant for social media metrics to emerge as a distinct and credible new field that can add value beyond justifying investment on new platforms and channels.

  3. I think the pro-ROI argument is null and void. ROI in PR is a fabricated number (somewhere between 2 and 4) multiplied by an ad-value (we all know advertising is dying) So in essence traditional PR ROI is a total doesn’t actually exist. How can you then apply a false number from an old communication method, to a new communication method which is even tougher to measure? You can’t.

    So somehow we’re compelled to create our own mythical social media number to apply to the work we do to prove the value that we’re creating, all based on an advertising theory that was applied and skewed by PR 20-30 years ago. It makes no sense.

    I see elements of social media developing into a customer service role – what then is the value you can put on a brand helping out a customer? What is the value to a brand of having a call centre where people are actively talking to customers to solve their issues? Have any brands tried? No.

    With social media, we’re turning to providing the same sort of function – it’s a one to one transaction.

    ROA is an interesting alternative but i don’t think brands are ready to adopt anything like it because there’s no direct monetary attributable end product. It’s only when brands wake up and realise that there’s more to the service they’re providing than lining the pockets of their executives, such as offering first class customer service AND a good product, that we’ll see this ROI debate begin to diminish.

    I think this is why smaler brands are more confortable taking on social media and using it as a marketing tactic – they need to build relationships with their customers to retain business and grow, which brings another interesting question: Is Return on Attention the same as Return on Relationship?

  4. I admire the guts of someone who answers “Yes” to the IAB’s question, agreeing that social media measurement is useless. Even if it’s just to be provocative and get a bit of free PR it does seem a bit blinkered, especially from someone who represents an industry body. Anyway…

    The world of communications is rapidly changing. And the most simplistic way to think about social media measurement is this: 10 years ago, brands would have willingly paid good money for any analysis of what conversations took place about them in pubs/living rooms. Now they can get it for virtually free. That they could, and should now be measuring these online conversations is, in my humble opinion a given.

    BUT – the issue then becomes a) how to interpret the data, and b) how to present it back to CMOs/CFOs in a way which is relevant to their business objectives. That’s a whole different discusssion. It’s not about measurement, it’s then about analysis. And just because we haven’t nailed it yet across the board doesn’t mean we shouldn’t measure what we can and try and interpret it – be it via econometric modelling, NPS, dwell time, or growth in number of “followers” or whatever KPIs we think they’ll be most interested in.

    We’ll all have different interpretations of the relative importance of all those metrics depending on what the activity was designed to achieve in the first place, but hey – that’s no different from any comms campaign already isn’t it?

    I do get a bit uptight when (traditional) advertisers belittle social media measurement, given the volumes and data involved. Not quite social media measurement I know, but I recently worked on a campaign in Habbo Hotel which had over 30,000 demographically-spot-on respondents to a series of questions. 30,000! Compare that to the average research group sample size. Or the number of homes with Barb boxes in them. Or the numbers of people in an average tracker sample… I’m totally with Tim Whirledge on that one.

    And where I have been able to measure ROI of social media results against paid-for, from first awareness through to purchase, social media has won hands-down. Admittedly it didn’t got the volumes of paid-for, but I’m confident that will change in due course.

    Yes, there’s still work to do. Yes we need to harmonise how we present what we measure, but to suggest that social media measurement is useless is risible. I shall now jump off my high horse…

  5. No, social media measurement is not meaningless, and indeed i believe that if social media is to survive in the long term, these measurements are essential in order that the discipline can continue to increase it’s value share in the marketing/PR budgets.

    Perhaps there is an interpretation that says that now, whilst social media measurement is undefined, it can be rendered meaningless – if only by virtue of the fact that there are so many convoluted ways to measure KPIs and ROIs that the non-uniformity of views within social media mean there is no meaningFUL way to date. As a single set of measurement tools.

    Being a member of the IAB social media council i am actually heading up a sub group that will seek to set these measurement parameters (at least) in place. I am under no illusion that we can set a true definitive measurement prospect without the input of a wider social media community, and as such i would call for as many people to join me in this quest and contribute their views.

    Also, as an aside, i suspect that Amy presented a contrary POV because, quite frankly, she likes to incite dsicussion. I seke to prover her wrong… ;-)

  6. We shouldn’t be spending a penny of our clients money without being able to prove a return. Specify the objective, plan the campaign, and then build a measurement system to evaluate the ouput.

    Discussions about social media measurement always become bogged down in tactics. No one measurement system fits all, just as no one set of KPIs will suit all businesses.

  7. Good post Michael, but the IAB argument seems a little flawed to me. Seems they are overly simplifying things. And are they for real? “Do the attempts to measure and make sense of all those friends, fans, followers, pokes, posts and retweets really amount to anything,” they say.

    Key to this discussion I think is the definition of measurement and value. I’m yet to work on a campaign of some kind where there was no meaning or value placed in the output of that campaign.

    This is only my view, but I think it’s a common sense approach – I think you just need to find the right way to explain a campaign’s results to the business you are doing it for. Whether that’s a traditional value or the opposite, whatever that might be. It’s pretty simple, and without that, why do it in the first place. Just imho :)

  8. Hi mate. Firstly, thanks for the Magners plug (I should disclose to everyone, I work at Cake where I look after a lot of what Magners UK are doing within social media at the mo).

    Now on to more pressing matters; measurement. In my opinion, there are three key routes to unearthing a value to what we do; benchmarking, creative KPI setting and arguing.

    Benchmarking; four types; against competitor brands activity, against other marketing activity that claims to be able to answer the brief but in a different way, against similar activity that has been done by another brand, against previous activity.

    KPI setting; too many people over promise and under-deliver in social media agencies. We all know here that brands don’t ‘do’ social media- setting up a twitter feed is not an idea nor a strategy. We have to think longer and harder about what it is we are trying to do. This means we have to think beyond ‘traffic’ as a metric and number of followers etc we put in front of our client’s noses. Traffic is a means to an end (sales) but perhaps your activity can drive efficiencies in other ways which reduce costs for the business, which ultimately will affect profitability. e.g. because you’ve built a community of 1,000,000 people on twitter, DM spend could be reduced as your cost per contact on twitter has been reduced. Ok, bit crude and it’s late, but if it helps educate the client and shows progress; show it. I guess the point being is that there is a lot of stuff we CAN measure, we just need to think more creatively about how we use these metrics as KPIs.

    Thirdly, and perhaps most importantly, to answer the question in your title (the answer is no by the way!), every other marketing discipline has arrived at a commonly accepted set of processes and metrics which enable them to perpetuate their case for spend. TV advertisers use a little box on a few people’s telly to see what they like watching. OOH uses footfall therefore X many people saw our ad etc etc. DM uses cost per contact. Again, I’m being crude (and it’s even later), but my point is that no measurement system is going to be able to tell a client 100% that it was their activity that drove conversion and ultimately ROI. The phone number that was exclusively on the mailer for sure was the avenue the sale came through but it doens’t mean that it was the mailer that caused a change in behaviour and drove acquisition. He/She could have been motivated by the ATL or a forum post. We’re human beings. We get influenced a lot. By lots of things. And people. Offline and online.

    As a group of people who are passionate about the possibilities for socially enabling tools, I think we need to stop looking at the things we can’t measure (like influence for example= ooo, another can.worms.opened), and start looking at the things we can, assigning objectives and KPIs to the behaviours we can change and MEASURE which will ultimately depend on how your campaign looks to utilise social media tools.

    So social media measurement is not pointless. Of course, we have to keep asking the question, about creating industry standards, KPIs, metrics, benchmarking, ROI, etc. But ultimately, no measurement system in the history of marketing is flawless, so we may as well crack on.

  9. Michael,
    The bottom line is that (good or bad) for business, ROI and measurement is crucial. Everything is measured either for its contribution or its cost. I personally would rather be measured for my contribution to the company, not how much I’m costing the company.

  10. Interesting post, and I suspect the answer, as ever, lies somewhere between yes and no. Yes, it is useless because if you don’t/cant translate a meanginful percentage your thousands of Facebook fans into actual customers then it doesn’t matter how many ppl follow you online. No, it isn’t as ambient brand awareness and share of voice levels do seem to have an impact on bottom line figures.

    With regards to Jean Wyllie, my current MD, leaving PN being proof that the times they are a changing, I’m afraid you couldn’t be wider of the mark. Jean has possibly the greatest understanding of the importance of digital and social media for our industry than pretty much any PR person I know and has never shied away from bringing in new talent to help shape the future of the company. She’ll be sorely missed and not just by the digital team she created here at PN towers.

    BTW, I recommend this rather wonderful ppt that offers some concrete suggestions for tying buzz to business performance.

    http://www.slideshare.net/thebrandbuilder/olivier-blanchard-basics-of-social-media-roi?src=embed

  11. While analysis has it place – I like your ROA concept. Building relationships always has value – In push marketing it was possible to measure a campaign. In social web we have different metrics. Business should understand this – they go to Chamber meetings, play golf, attend charity events, etc. – all building relationships but value not easily measured.

Leave a Reply