Understanding Natural Language Processing For Social Media Monitoring
Understanding Natural Language Processing For Social Media Monitoring
by

In our explorations of social media monitoring firms, one differentiating factor between services is often the ability to parse out sentiment and tone of online conversations found about your brand. This task is performed by sophisticated computer functionality called natural language processing (NLP). It’s not the sexiest topic in the world, but it’s a very important one for public relations professionals, marketers and brand managers to understand.

Fortunately, I caught up with Jeffrey Catlin, the CEO of Lexalytics, an Amherst, Massachusetts-based NLP firm, recently to dive into the subject matter a bit. In this episode of SME-TV we learn what drives natural language processing, what the types of accuracy rates should be expected when using tools that provide the functionality and that Lexalytics has some consumer-facing products on the horizon we can be on the lookout for.

Exploring Natural Language Processing With Lexalytics CEO Jeff Catlin from Jason Falls on Vimeo.

As a follow-up to the chat with Catlin, I spoke with a couple of my friends in the monitoring space. A couple of them agreed with most of Catlin’s assessments of the accuracy question. One, however, noted that their service does use a vendor like Lexalytics to provide a foundation of NLP for their tool, but has built an ever-learning algorithm atop that technology. Because they can train the tool to understand the dynamics of conversations around a particular client and build upon that learning in an ongoing fashion, they can produce accuracies in the low- to mid-80s over time and are even around 83% on some clients now.

Another NLP hang up is that tools like Lexalytics were trained to interpret press releases and more formal journalistic pieces. Social media is chock full-o-typos and other truly natural human language quirks that the machines can learn, but only if they’re told to do so. Plus, there are dozens of sentiments around brands, not just positive, negative and neutral, so NLP has its limitations. Still, as Catlin said, with a large amount of information, it can give you a broad brush stroke of look at what’s out there.

Regardless of the tit-for-tat debate over who is more accurate or if they even are, at least we know understand the process and environment better. Hopefully, this helps you make better decisions in social media monitoring solutions for your company or brand.

Be sure to keep an eye on Lexalytics in the coming weeks, however. Their free tools, which I’ve seen demonstrations of, are simple, intuitive and most useful for those of us who have data (perhaps blog content, customer surveys or focus group interview transcripts) and need a tool to process and analyze it without chewing up a lot of budget. I’m excited to see how you can leverage those tools.

And, if you have more questions about natural language processing and its application in the social media monitoring and measurement space, please throw them out in the comments. If I can’t answer them, I’ll do my very best to find someone who can. (Plus, I’ll bet the Lexalytics and social media monitoring folks will be reading with interest.)

Reblog this post [with Zemanta]

  • Tan_yong_tai

    Interesting. Thank you.

  • Because they can train the tool to understand the dynamics of conversations around a particular client and build upon that learning in an ongoing fashion, they can produce accuracies in the low- to mid-80s over time and are even around 83% on some clients now.

  • Because they can train the tool to understand the dynamics of conversations around a particular client and build upon that learning in an ongoing fashion, they can produce accuracies in the low- to mid-80s over time and are even around 83% on some clients now.

  • Pingback: Why You Shouldn’t Trust Automated Sentiment Scoring | BrettMBell.com()

  • Pingback: Why You Shouldn't Trust Automated Sentiment Scoring()

  • You had a great post, but I can't relate in it.

  • Wow! You have a great point in your topic. Well I came to realize of what you pointing out. Thanks for sharing this great great idea to us.

  • This is really a great article. Thanks for posting.

  • jtorras

    Thanks for this article Jason!
    At Inbenta we are using our “semantic clustering” technology to group conversations that have a similar topic. As you told, they are not necessarily good or bad, but it's interesting to see patterns related to semantic meaning of those conversations

    • You're welcome. Thanks for the comment!

  • winter123

    Louis Vuitton Matsuya Ginza Store done renewal and displays their “speedy” archives over past years tiffany jewellery online shop sells varieties of tiffany jewelry products, including tiffany bracelet, tiffany necklace, tiffany ring, tiffany earrings, tiffany pendant and various other tiffany replica jewelries. We are one of the biggest suppliers of tiffany jewellery sale
    ,Louis Vuitton has been welcomed by many louis vitton handbag lovers. I think there is no cheaper louis vuitton handbags than you in the world. “speedy” is known as Louis Vuitton’s iconic line. Every arranged bags were very interesting.

    Louis Vuitton Matsuya Ginza Store done renewal and displays their “speedy” archives over past years,Louis Vuitton has been welcomed by many Louis Vuitton Handbags lovers. I think there is no cheaper LV Handbags than you in the world. “speedy” is known as Louis Vuitton’s iconic line. Every arranged bags were very interesting.
    Louis Vuitton Matsuya Ginza Store done renewal and displays their “speedy” archives over past years,Louis Vuitton has been welcomed by many Louis Vuitton Handbag lovers. I think there is no cheaper louis vuitton sale than you in the world. “speedy” is known as Louis Vuitton’s iconic line. Every arranged bags were very interesting.

  • winter123

    Louis Vuitton Matsuya Ginza Store done renewal and displays their “speedy” archives over past years tiffany jewellery online shop sells varieties of tiffany jewelry products, including tiffany bracelet, tiffany necklace, tiffany ring, tiffany earrings, tiffany pendant and various other tiffany replica jewelries. We are one of the biggest suppliers of tiffany jewellery sale
    ,Louis Vuitton has been welcomed by many louis vitton handbag lovers. I think there is no cheaper louis vuitton handbags than you in the world. “speedy” is known as Louis Vuitton’s iconic line. Every arranged bags were very interesting.

    Louis Vuitton Matsuya Ginza Store done renewal and displays their “speedy” archives over past years,Louis Vuitton has been welcomed by many Louis Vuitton Handbags lovers. I think there is no cheaper LV Handbags than you in the world. “speedy” is known as Louis Vuitton’s iconic line. Every arranged bags were very interesting.
    Louis Vuitton Matsuya Ginza Store done renewal and displays their “speedy” archives over past years,Louis Vuitton has been welcomed by many Louis Vuitton Handbag lovers. I think there is no cheaper louis vuitton sale than you in the world. “speedy” is known as Louis Vuitton’s iconic line. Every arranged bags were very interesting.

  • Thanks, Darian. Great to hear from you!

  • Jason, great article. I couldn't agree more with the Jeff's comments on accuracy of sentiment and it's usefulness in aggregate. We developed a hybrid sentiment engine that handles poor grammar and typos so I understand first-hand the difficulty in handling social media. Sarcasm is a huge challenge. Thanks!

    — Darian Shimy, VP of Technology Biz360.com

  • Jason, great article. I couldn't agree more with the Jeff's comments on accuracy of sentiment and it's usefulness in aggregate. We developed a hybrid sentiment engine that handles poor grammar and typos so I understand first-hand the difficulty in handling social media. Sarcasm is a huge challenge. Thanks!

    — Darian Shimy, VP of Technology Biz360.com

    • Thanks, Darian. Great to hear from you!

    • Yes, I agree to you that this is really a great article.

  • Great video

  • Really great interview. I agree that what makes automated sentiment classification valuable is the ability to uncover trends and make sense of enormous data sets that human vetting just can't accomplish. The rates of accuracy described here ring true; it's still a great overview or “broad brush stroke” tool, as you said.
    It becomes vitally important then to make sure you integrate that data into a usable toolset. A virtualization can give you a visual representation that's easier to gain insight from than simple pie charts and graphs.

    Eric Melin – @ Spiral16

  • Really great interview. I agree that what makes automated sentiment classification valuable is the ability to uncover trends and make sense of enormous data sets that human vetting just can't accomplish. The rates of accuracy described here ring true; it's still a great overview or “broad brush stroke” tool, as you said.

    It becomes vitally important then to make sure you integrate that data into a usable toolset. A virtualization can give you a visual representation that's easier to gain insight from than simple pie charts and graphs.

  • Thanks for the time, then and now, Jeff. Great to hear how NLP works
    from someone who knows. Can't wait to see the new consumer products as
    well.

  • jeffcatlin

    Jason, Thanks for taking the time with me out in the wind storm at Gillette. Seems I'm already late to the commenting party, but I did want to comment one the person that said it's possible to achieve mid-80s levels of accuracies on sentiment. This is certainly true, if you focus in on a vertical market or particular type of content. We ourselves did a project to tune our engine for Hotel Review content and managed to achieve about 84% accuracy on that data, so yes it's possible but does require some added work.

    Thanks to those that liked the “simplified” terminology I used… Nothing worse than people that explain technology using big nasty words :-)

  • jeffcatlin

    Jason, Thanks for taking the time with me out in the wind storm at Gillette. Seems I'm already late to the commenting party, but I did want to comment one the person that said it's possible to achieve mid-80s levels of accuracies on sentiment. This is certainly true, if you focus in on a vertical market or particular type of content. We ourselves did a project to tune our engine for Hotel Review content and managed to achieve about 84% accuracy on that data, so yes it's possible but does require some added work.

    Thanks to those that liked the “simplified” terminology I used… Nothing worse than people that explain technology using big nasty words :-)

    • Thanks for the time, then and now, Jeff. Great to hear how NLP works
      from someone who knows. Can't wait to see the new consumer products as
      well.

    • Jeff is spot on here… especially with regard to application of NLP within vertical industries… we find that, with our customers, that success is driven by developing “rules” within very specific verticals and the context within those verticals (e.g. ongoing “corporate reputation” issues, known consumer issues/needs, etc.).

      Furthermore, by enabling domain experts across customer touch-points (marketing, PR, customer service) to influence how rules get “taught”, accuracy rates can approach the 80% range… any social media measurement / NLP efforts must be seen as a journey…

  • It's a deal. Give me a week or so and we'll take a look under the hood.

  • nilesh_bansal

    Jason, not sure if you have had a chance to look at Sysomos MAP. If not, we would love to show you a demo.

  • You're welcome, Amy. Thanks for the comment.

  • Thanks for chiming in Nilesh. Glad to have your perspective here.

  • Thanks Michael. Glad to see you here.

  • researchgoddess

    Jason, thanks for posting this! Semantic search is becoming a hot topic in the recruitment sourcing world as well; this was a very good explanation of how to determine sentiment and I am surprised at even the human error here. Great interview!

  • researchgoddess

    Jason, thanks for posting this! Semantic search is becoming a hot topic in the recruitment sourcing world as well; this was a very good explanation of how to determine sentiment and I am surprised at even the human error here. Great interview!

    • You're welcome, Amy. Thanks for the comment.

  • nilesh_bansal

    Hi Jason,
    I agree with you. Language processing is a very important component when you go beyond the world of social media monitoring to the space of analytics. There are two main parts to it:

    – Summarizing the text to automatically create condensed visual summary of what is going on, without having to read through the data.

    – Finding the sentiment. While it is a difficult problem, based on our work here at Sysomos, it is doable. The trick is to make the engine understand the context not only within well written news articles but also in more colloquial social media language.

    NLP however goes beyond just sentiment or summarization. We use it in all modules of the Sysomos platform, such as in cleaning spam and removing junk.

    Nilesh Bansal
    http://www.sysomos.com

  • nilesh_bansal

    Hi Jason,
    I agree with you. Language processing is a very important component when you go beyond the world of social media monitoring to the space of analytics. There are two main parts to it:

    – Summarizing the text to automatically create condensed visual summary of what is going on, without having to read through the data.

    – Finding the sentiment. While it is a difficult problem, based on our work here at Sysomos, it is doable. The trick is to make the engine understand the context not only within well written news articles but also in more colloquial social media language.

    NLP however goes beyond just sentiment or summarization. We use it in all modules of the Sysomos platform, such as in cleaning spam and removing junk.

    Nilesh Bansal
    http://www.sysomos.com

    • Thanks for chiming in Nilesh. Glad to have your perspective here.

      • nilesh_bansal

        Jason, not sure if you have had a chance to look at Sysomos MAP. If not, we would love to show you a demo.

        • It's a deal. Give me a week or so and we'll take a look under the hood.

  • Michael Fraietta

    Wow Jeff, thanks for the awesome, very clear explanation.
    I will be quoting some of your phrases (particularly, “Humans are only 85-90%”) when discussing sentiment with our clients. Great post Jason!

    Michael Fraietta
    Filtrbox
    @Michael Fraietta
    http://fbx.bz/E

  • Thanks for the interview. I'll really be interested on how the consumer service and plug-in take off, since that is a way to evangelize capabilities and value (especially since so many may not get what it does/can do).

  • Thanks for the interview. I'll really be interested on how the consumer service and plug-in take off, since that is a way to evangelize capabilities and value (especially since so many may not get what it does/can do).

    • vandolet

      Wow Jeff, thanks for the awesome, very clear explanation.
      I will be quoting some of your phrases (particularly, “Humans are only 85-90%”) when discussing sentiment with our clients. Great post Jason!

      Michael Fraietta
      Filtrbox
      @Michael Fraietta

      • Thanks Michael. Glad to see you here.