Debunking Natural Language Processing (NLP)

By: Guest Blogger

August 3, 2017

Clarabridge Analytics
Text Analytics
Artificial Intelligence
Machine Learning
Natural Language Processing
Natural Language Understanding

By: Ellen Falci, Product Manager, NLP/Enrichment, Clarabridge


When I started working at Clarabridge five years ago, NLP and text analytics were very technical sounding terms. My parents and peers were baffled when I tried to explain the field that I worked in. In those five years though, the term has had a rebirth as marketers have tried to capitalize on the perceived sexiness and sophistication of related technologies. While it still preserves a very specific meaning in academia and research organizations, the term NLP has been watered down in the CX space and often just refers to lists of words and topics.

This change in semantics is somewhat unsurprising. As NLP toolkits like Stanford NLP, NLTK, spaCy, IBM’s Watson and Google’s SyntaxNet become more consumable, they are quickly divorced from their technical origins. No longer does one need to be a software developer or a computational linguist to use NLP technologies. These toolkits “blackbox” a lot of their complexity so that the user can focus on the outputs, not on the algorithms. However, as a result, many users of these systems only leverage a small fraction of their capabilities by focusing on outputs that are easy to consume (ie: word clouds). In an attempt to recapture the complexities of NLP, some individuals have introduced the term Natural Language Understanding (NLU) to refer to computational approaches to understanding meaning behind language, instead of just abstract patterns.

Let’s be honest, just grouping data into topics or categories isn’t enough anymore. For a while, it was exciting and novel to be able to categorize your data into topics with a few keywords and phrases. It streamlined old, subjective processes, such as manually reading through and categorizing customer feedback data, and added consistency and efficiencies into CX workflows. Some tools (including Clarabridge) offer features that help you uncover new topics which can then be added into your reporting but over time there are fewer and fewer truly new categories to add. We’ve reached the long tail. It’s useful to monitor these categories but there is a sure ceiling as to what they can truly accomplish for you. We are at the point where simply analyzing volumes and sentiment for distinct topics is insufficient to find actionable and meaningful insights.

So now you’re asking, Ellen, what’s an analyst to do?

Glad you asked!

In order to get to NLU, a system must have mature NLP. At Clarabridge, our 10 years developing our proprietary NLP engine has afforded us an extremely strong linguistic foundation upon which we can build. You may have noticed a shift in our attention over the past 2-3 years to spend more time developing features that help you understand your data in new ways, with less effort. We are bullish about the value in leveraging semantic analysis, emotion analysis and more on top of topic analysis, not in place of it. We advocate for layering new types of language analysis to expose new kinds of insights. Imagine a world, for example, where you could automatically find all suggestions from your customers, isolate every time a customer praised an employee or even find all instances of high effort.

Over the next few months, I will be highlighting several features that help Clarabridge users go beyond simple topic analysis. This series will show you how those new types of analysis aren’t so far-fetched, after all. Look out for a new post every two weeks!