The Problem with Klout as a Measure of Influence

Klout has quickly become the standard measure of social media influence. Bloggers and other social media “experts” are finding that their Klout scores are being used to determine everything from job opportunities to hospitality perks. On Tuesday, Huffington Post blogger @2morrowknight revealed he was rejected after an agency looked up his Klout score. Hotels and clubs in Las Vegas also admit to using Klout scores when putting together VIP lists.

Klout provides a score that is supposedly indicative of a Twitter and/or Facebook user’s influence across their social media networks. The score is calculated via an algorithm that takes into account factors such as Twitter retweets, @messages, follows, list inclusions, comments on Facebook, and Facebook likes. While these specific factors are used to determine “network influence”, Klout also looks at “True Reach” and “Amplification Probability ” to determine the size of a users’ engaged audience and “the likelihood that a users’ content will be acted upon”, respectively. Together, network influence, true reach and the amplification probability determine the Klout score, which is the “overall online influence.”

Klout has finally admitted that users’ profiles may not be accurate, causing discrepancies in everything from profile information to the “achievements” which directly impact the Klout score. In an email exchange with Klout representative Ashley, she conceded that “[t]here are a number of reasons that cause discrepancies and Klout is working very hard to fix them.” Users may see some profiles that underestimate a users’ actual unique mentions, retweets, retweeters, or list inclusions – just to name a few of the factors. The Klout algorithm also considers influencers, and within some Klout profiles this information is completely missing, or severely out of date. Prior to contacting Klout, I lacked “achievements” for retweets, mentions, and lists, as well as a module for other users I influence. Even after Klout recently manually recalculated my profile, I am considered an influencer of Twitter users of only those I have not engaged with in several months – and I am still lacking “achievements” such as list inclusions and reach. It’s not like I need a Klout score to validate my existence, but if this is the metric against which we are being judged, someone needs to call out the inaccuracy of the measuring stick.

In November 2010, Klout switched their application to automatically reprocess each profile daily, removing the function that allowed users to push for a manual update.  But when my Klout profile was clearly out of date by several weeks, and was lacking pertinent information crucial to the algorithm, I began to question the accuracy of any Klout score. As of this writing, the Klout profile of fellow girl geek @sarahaustin, who covers the industry for Forbes, has over 3,000 followers and is listed on over 200 lists, does not include any achievements or influence. As a result, Sarah’s score, compared to her actual influence in the industry, is shockingly low.

Klout scores always seem arbitrary, but when obvious information is missing or understated, how valid is the entire concept?

Article Written by

  • Sandy

    Yup; Klout’s info on me is either out-of-date or inaccurate; I stopped caring about Klout a while ago.

  • http://twitter.com/JoeFernandez Joe Fernandez

    Hey Kelly,

    I am one of the founders and the CEO here at Klout. The accuracy of our data is the most important thing to us here at Klout. The challenge here is we are dealing with extremely “dirty” data coming from Twitter, Facebook etc. To increase the fidelity on this data we have even had to slow down processing on our influencer of/influenced by calculations. Those calculations don’t impact your overall score and are primarily meant to be discovery mechanisms on our site.

    We continue to work on reducing the time between each data point on Klout profiles are processed across our whole user base. There are metrics that don’t change as much or don’t impact the score in a huge way that we update less frequently to make sure that the key metrics are always fresh. We should definitely be doing a better job on our site explaining this.

    This is definitely a work in progress and we appreciate this feedback.

    Thanks!
    Joe

  • http://twitter.com/JoeFernandez Joe Fernandez

    Hey Kelly,

    I am one of the founders and the CEO here at Klout. The accuracy of our data is the most important thing to us here at Klout. The challenge here is we are dealing with extremely “dirty” data coming from Twitter, Facebook etc. To increase the fidelity on this data we have even had to slow down processing on our influencer of/influenced by calculations. Those calculations don’t impact your overall score and are primarily meant to be discovery mechanisms on our site.

    We continue to work on reducing the time between each data point on Klout profiles are processed across our whole user base. There are metrics that don’t change as much or don’t impact the score in a huge way that we update less frequently to make sure that the key metrics are always fresh. We should definitely be doing a better job on our site explaining this.

    This is definitely a work in progress and we appreciate this feedback.

    Thanks!
    Joe

  • http://twitter.com/shelbyhealy Shelby Healy

    This is very interesting. I was wondering if some sort of change happened in the last couple of weeks. I was at a 57, now I’m at 38. How could that drop so low in such a short period of time?

  • http://twitter.com/shelbyhealy Shelby Healy

    This is very interesting. I was wondering if some sort of change happened in the last couple of weeks. I was at a 57, now I’m at 38. How could that drop so low in such a short period of time?

  • Kk+

    Klout score is a good relative measure of influence but a shitty absolute measure of influence. It’s one of many factors used to determined WTF is goin on out there on the internets. :)

  • Kk+

    Klout score is a good relative measure of influence but a shitty absolute measure of influence. It’s one of many factors used to determined WTF is goin on out there on the internets. :)

  • http://thesocialjoint.com/ Lucretia M Pruitt

    Um, Shelly? I’m the first person to admit that I’ve had my own issues with Klout – but I followed both of your links above and the first one links strictly to @2morrowknights twitter feed, but not to anything indicating that he has said anything like what you quoted he had ‘on Tuesday’, while the second one links to a PR announcement about Peter Shankman’s NYC party – but nothing indicating that “Hotels and clubs in Las Vegas” have admitted any such thing.
    Could you please actually link the sources or your information? Those allegations hold some pretty heavy weight if not just rumors.
    Thanks!

    • http://www.kelly-clay.com Kelly Clay

      In addition to VIP events, the Palms is building out the “Klout Klub” which “”will allow high-ranking influencers to experience Palms’ impressive set of amenities in hopes that these influencers will want to communicate their positive experience to their followers.”

      Virgin America has also offered free flights to those with high Klout scores.

      More information here: http://adage.com/digitalnext/post?article_id=146189

    • http://www.kelly-clay.com Kelly Clay

      In addition to VIP events, the Palms is building out the “Klout Klub” which “”will allow high-ranking influencers to experience Palms’ impressive set of amenities in hopes that these influencers will want to communicate their positive experience to their followers.”

      Virgin America has also offered free flights to those with high Klout scores.

      More information here: http://adage.com/digitalnext/post?article_id=146189

  • http://thesocialjoint.com/ Lucretia M Pruitt

    Um, Shelly? I’m the first person to admit that I’ve had my own issues with Klout – but I followed both of your links above and the first one links strictly to @2morrowknights twitter feed, but not to anything indicating that he has said anything like what you quoted he had ‘on Tuesday’, while the second one links to a PR announcement about Peter Shankman’s NYC party – but nothing indicating that “Hotels and clubs in Las Vegas” have admitted any such thing.
    Could you please actually link the sources or your information? Those allegations hold some pretty heavy weight if not just rumors.
    Thanks!

    • Anonymous

      In addition to VIP events, the Palms is building out the “Klout Klub” which “”will allow high-ranking influencers to experience Palms’ impressive set of amenities in hopes that these influencers will want to communicate their positive experience to their followers.”

      Virgin America has also offered free flights to those with high Klout scores.

      More information here: http://adage.com/digitalnext/post?article_id=146189

  • Anonymous

    I think Klout has a ton of potential but I feel like too many are using it as a catch-all silver bullet. There needs to be a “Klout U” on what it is and isn’t.

    No measuring system is going to ever be perfect when we’re talking about “influence.” What is influence? On what topics? There’s a lot of stuff at play here. But it helps to have a guide.

    That said, I’m very nervous that there’s agencies who are basing their hiring decisions (I’ve heard of other cases, as well) on personal Klout scores. I know a lot of good strategists who aren’t Twitterati; people who have put together solid stuff but they’re not the “cool kids” personally. For gosh sakes, the person in question has a score of 73! That’s pretty high, and if that’s not enough, what is?

    The challenges of navigating a brand and an organization are quite different than being popular or influential on a personal account. There’s a lot more to the story, a lot more moving parts. I’ve seen a lot of popular people fail spectacularly when they get to implementing within the constructs of an organization.

    One of the things Klout does very well and why it’s been adopted so fast, as I was having a discussion with a corporate colleague, is make C-Suites and execs happy – they’re data, number driven. A number is something they can understand, regardless of what’s contained within it.. So they lean on it.

  • Anonymous

    I think Klout has a ton of potential but I feel like too many are using it as a catch-all silver bullet. There needs to be a “Klout U” on what it is and isn’t.

    No measuring system is going to ever be perfect when we’re talking about “influence.” What is influence? On what topics? There’s a lot of stuff at play here. But it helps to have a guide.

    That said, I’m very nervous that there’s agencies who are basing their hiring decisions (I’ve heard of other cases, as well) on personal Klout scores. I know a lot of good strategists who aren’t Twitterati; people who have put together solid stuff but they’re not the “cool kids” personally. For gosh sakes, the person in question has a score of 73! That’s pretty high, and if that’s not enough, what is?

    The challenges of navigating a brand and an organization are quite different than being popular or influential on a personal account. There’s a lot more to the story, a lot more moving parts. I’ve seen a lot of popular people fail spectacularly when they get to implementing within the constructs of an organization.

    One of the things Klout does very well and why it’s been adopted so fast, as I was having a discussion with a corporate colleague, is make C-Suites and execs happy – they’re data, number driven. A number is something they can understand, regardless of what’s contained within it.. So they lean on it.

  • http://twitter.com/mikekujawski Mike Kujawski

    Patience people, patience. As Joe mentioned, the folks at Klout are obviously working hard to improve their algorithm and process. Like EVERY analytic tool out there, it’s only a tool. A tool that relies on data. The fact that the data is not perfect is not the tool’s fault. And even with perfectly accurate data, you still need a data analyst to put it in context. At present time, most people blindly take the numbers at face value and jump to conclusions. That’s the bigger problem here.

  • http://twitter.com/mikekujawski Mike Kujawski

    Patience people, patience. As Joe mentioned, the folks at Klout are obviously working hard to improve their algorithm and process. Like EVERY analytic tool out there, it’s only a tool. A tool that relies on data. The fact that the data is not perfect is not the tool’s fault. And even with perfectly accurate data, you still need a data analyst to put it in context. At present time, most people blindly take the numbers at face value and jump to conclusions. That’s the bigger problem here.

  • Anonymous

    I use KeePass and DropBox to do the same thing. Syncs with my Android phone as well. Even though the KeePass archive is in the “cloud” on DropBox, I control the access to the archive.

  • Christopher DeMero

    If you worried about your Google account, you can always use the two step verification method.

  • http://twitter.com/FrugalGeek The Frugal Geek

    Thanks, Ryan.

  • http://about.me/sandmaxprime Lionel Faleiro

    I normally:

    1. Download and play the demo – If I like it, I will keep it on a Too Buy List.
    2. Purchase Later – If the game has released, has gotten good reviews and impressed me in the demo, I will keep it aside and buy it Later. This tactic is especially useful for console games because after 3 months, the games get released as platinum releases where their prices are 50% of the price when it was first launched

  • Anonymous

    Well, as a mmorpg player, it gets expensive to pay the monthly fees up to 15 $us per month, after 1year, if you don’t pay, all that money means nothing and all that hard work is gone because you can not access the game to play, this is world of Warcraft model, but other games like world of tanks or global agenda are mature, solid mmorpg player games that have in game purchases that help to pay for the game, but free to play when you don’t have any money, this makes sense, because, you can pay for things you like to enhance the game experience, like tanks, you can buy some tanks, gold amo, convert experience to do upgrades, but if you don’t have any more money, you can still play the things you previously purchased. This is akin to purchasing some hobbies at home, you put it in the draw, if you don’t pay, the draw is locked, but in the real world, you paid for it, so you can enjoy it, want add on, extra features? Special jet pack what ever, then you can buy them and use them for ever, now I don’t feel conned, can save money and still have loads of fun in clans and teams.