Do Social Media Monitoring Services Leave Brands Blind?

Valuevine released a study today that indicates keyword based social media monitoring platforms may leave as much as 82 percent of the online content about a brand out of the data it reports back to brands. The information was collected by analyzing what CEO Neil Crist told me was “following best practices” for keyword-based social media monitoring solutions like Radian6, Sysomos and the like, and comparing it to the data his tool collects from location- and local-focused check-in, ratings and review sites.

Valuevine focuses its analytics product, Valuevine Connect, on source-based searches. So instead of plugging keywords in and indexing sites to find mentions of your brand, it instead identifies the location-specific pages on sites like Yelp, CitySearch, Foursquare, Twitter, Facebook Places and more, and reports user information gleaned from those pages.

Essentially, Valuevine focuses on your known quantity of online conversations. Keyword search engine platforms like Radian6 (and frankly most social media monitoring platforms) just search for those keywords, regardless of source.

While Valuevine’s data is shocking — one brand they tested in a keyword search only revealed 18 percent of the data Valuevine derived from source-based searching — brands using keyword-based tools need to be upset … perhaps.

First, let’s level-set expectations.

“Some of these sites, specifically Yelp, have legal issues with crawling them,” Nilesh Bansal, Chief Technical Officer and Co-Founder of Sysomos told me. “Their copyright explicitly prohibits such crawing. Also, there are technical limitation with some platforms that choose not to provide APIs.”

What this means is that social media monitoring platforms can’t legally index content on many of these sites.

Crist explained how Valuevine does it by saying this:

“There are two levels of data granularity that we can bring to into the view for our clients. The baseline is that we use a search engine indexing partner that works with many sources in accordance with standard search engine policies. In turn, we follow standard search engine display behavior. For instance, we we only show snippets and ultimately drive activity back to the source pages. In that scenario those services win with the eyeballs of merchants. For some partners, we have deeper access and partnerships.”

Regardless of whether or not Valuevine is collecting information in ways other services aren’t, the simple fact remains that for brands, this is likely the first time they’ve been alerted to the fact that the security blanket they thought social media monitoring services were providing is mighty small.

Think about it. If I check-in on Facebook Places and say, “I really love this restaurant,” it likely doesn’t trigger any keyword searches the brand has in place in its monitoring software. For that matter, volumes of information could be shared by consumers about a company without alerting any of the keyword scans.

For Valuevine’s scan of Starbucks, what that meant was that 68 percent of online conversations — mostly tied to brick-and-mortar locations where the consumer is at the point-of-sale — are potentially not being seen. This information is perhaps more critical to the brand than that being found by keyword searches because it’s 100 percent relevant to specific stores and brands. You could essentially develop an entire quality assurance effort around this data.

“We do source-based listening,” Radian6 CEO Marcel Lebrun assured me. “If I want every post on Dell.com regardless of what they say, I can get that in addition to the keyword searches with Radian6. In fact, most listening platforms do both.”

Sysomos confirmed for me they offer the same functionality. They label it “Domain Filtering,” but it operates the same way.

But can these systems do location page filtering — focusing on just a single location’s UrbanSpoon page, for instance — rather than a domain-wide search? No one really wanted to jump at the opportunity to say yay or nay because of the terms of service issues with respective sites. My guess is that they can do it, but don’t want to get into trouble.

So that brings us back to Valuevine’s data. Crist said they applied best practices to do a keyword search but wasn’t specific on which keyword engine they used. If it was internal, then comparing it to a robust and several-year-old platform like Radian6 would be unfair. He did indicate that of the clients and agency partners Valuevine worked with on the analysis, none of them seemed to either be aware their keyword-based solutions could do source-based monitoring, or they didn’t know how to use it.

The bottom line is that the huge gaps Valuevine’s data shows are probably mostly driven by the fact the social media monitoring platforms don’t index ratings and review sites. If they did, the gap would likely close. But where does this leave brands?

Beating up their social media monitoring platforms to do what Valuevine does or using both.

To his credit, I asked Crist if he thought brands should question their use of keyword-based tools.

No way!” he responded. “They absolutely need it. It’s fundamental for all brands. But for the subset of brands that have a brick and mortar location, keyword-based solutions are just one set of solutions you need. You also need to monitor the location-specific information keyword tools don’t see.”

That I agree with. And I’ve always been impressed with Valuevine’s offering, even before they became an analytics solution rather than a franchise management tool. It aggregates location-specific information into a nice, franchise-wide data set, but then allows you to drill down by location to get store-specific insights and metrics.

Another point to consider, as Vladimir Oane of uberVu deftly pointed out, is that working with location-based data is not the same as looking at topic-based streams. So adding it to keyword search data is apples-to-oranges.

“The data is structured differently, displayed differently and you interact with it a different way,” he told me. “Not to mention you need different metrics because old ones don’t apply.”

Sounds reasonable. You don’t count check-ins, mayorships and four-out-of-five star ratings when pulling information into your reporting from Twitter or some random blog. But that’s limiting your concern to the analytics. The real-time review of what people are saying on your very own brand pages being left out of monitoring services just smells wrong, doesn’t it?

The Valuevine research is, of course, accompanied by a nifty marketing play. They’re offering up a free calculator for companies who want to see how much of their local data is being missed by keyword tools. All you have to do to use it is plug in the keywords you’re searching for with your monitoring service (which may not be easy for most people), identify your business or brand and within a few hours, maybe even a day since the tool will be over-used for a few days, you’ll get a scorecard back.

But be warned: The data simply can’t be considered chapter and verse. Haphazzardly plugging in keywords to an online calculator isn’t efficient and we don’t know how Valuevine’s keyword searches compare to real keyword-based monitoring solutions. Plus, how Valuevine collects its data is — at a minimum — now going to come under scrutiny from the sites it pulls from.

For more on the Valuevine research, check out their full report on the Valuevine website.

But tell me what you think about the possibility that brands are now realizing their monitoring solutions were missing so much? Will this affect the way you look at your monitoring solution provider? Should it? Aren’t ratings and review site and location-based information sites perhaps more important than other data collected from random sites around the web?

I’m very interested to know your thoughts on this potential blind spot for brands. And what other solutions exist to ensure we’re getting what we expect from social media monitoring. The comments are yours.