One struggle that seems common among most companies that gather feedback from their customers is how to best understand and use unstructured feedback (often called verbatim comments). On one hand, these comments are often the best way to truly understand an individual situation, as some customers will provide paragraphs of information describing their issues, concerns, and improvement suggestions. People love these comments. However, sometimes users of feedback will latch onto one or two comments and consider those representative of the entire customer universe, which can be dangerous, and flat out incorrect.
The best use of feedback is a balanced approach using both quantitative feedback to understand trends, strengths, weaknesses, and areas driving future behaviors, while leveraging the qualitative feedback to drill deeper once the focus has been narrowed. Traditionally that has required sifting through comments to categorize them by theme, and then identify meaningful ones based on the kind of customer, their experiences, and the prominence of their voice in the eyes of executives. However, we’re entering an age where much more can be done with customer commentary.
Technologies have emerged recently, with some initial human-based teaching and support, will automatically categorize customer comments. What’s more, software can also convert the comment into a quantitative form of feedback based on the tone of the comment (positive, neutral, or negative). Linking this to other information about customers (revenue, profitability, loyalty, and other items), may potentially open many doors that have not been visible in the past.
The one shortcoming of these technologies to this point has been the ability to properly handle sarcasm, a common tactic that humans use, particularly when displeased with something to add even more of a negative tone to a comment. Most humans can pick up and interpret quickly, while machines have been prone to taking the comments at face value, and thus misinterpreting them.
So, a comment like:
“Great idea, now try again with a real product development team!”
... would likely be determined by a computer program as a positive comment, but you and I know this is a slam on a company’s product development team.
Fortunately, a team of researchers at the Hebrew University in Jerusalem appears to be on their way to address this potential issue with automated review of commentary. Called SASI (Semi-supervised Algorithm for Sarcasm Identification), the team has been able to recognize sarcasm among online product reviews with 77% accuracy, which is probably better than many humans could identify. Incorporating this into our ability to review, understand, and act upon comments, both individually and in aggregate, will surely help us capitalize on the feedback being provided by customers.
I’ll be watching closely to see how this technology progresses, and will continue to comment on the evolution of this in the future. Are there other technologies that you’re using to help take advantage of the feedback your customers provide? If so, I’d love to hear about them.
Vice President, Consulting Services