Modeling and computational characterization of twitter customer service conversations

Abstract

Given the increasing popularity of customer service dialogue on Twitter, analysis of conversation data is essential to understanding trends in customer and agent behavior for the purpose of automating customer service interactions. In this work, we develop a novel taxonomy of fine-grained "dialogue acts" frequently observed in customer service, showcasing acts that are more suited to the domain than the more generic existing taxonomies. Using a sequential SVM-HMM model, we model conversation flow, predicting the dialogue act of a given turn in real time, and showcase this using our "PredDial" portal. We characterize differences between customer and agent behavior in Twitter customer service conversations and investigate the effect of testing our system on different customer service industries. Finally, we use a data-driven approach to predict important conversation outcomes: customer satisfaction, customer frustration, and overall problem resolution. We show that the type and location of certain dialogue acts in a conversation have a significant effect on the probability of desirable and undesirable outcomes and present actionable rules based on our findings. We explore the correlations between different dialogue acts and the outcome of the conversations in detail using an actionable-rule discovery task by leveraging a state-of-the-art sequential rule mining algorithm while modeling a set of conversations as a set of sequences. The patterns and rules we derive can be used as guidelines for outcome-driven automated customer service platforms.