Applying Data Mining Techniques to Direct Marketing: Challenges and Solutions

Abstract

In this paper, we discuss one of the most common tasks faced by marketers when faced with resource and time constraints, namely, consumer prioritization with the objective of optimizing one or more marketing key performance indicators such as consumer conversion. A key element in building predictive models is the ability to introduce features that capture historical user behaviors in an effective manner so as to differentiate between those consumers who are most likely to convert without nurturing, those who are likely to convert with nurturing, and those who are unlikely to convert irrespective of the marketing campaign and channel to which they may be subjected. Towards this effort, we propose to use a set of dynamic features to capture consumers' engagement behaviors. Moreover, we have also applied the non-negative matrix factorization (NMF) to identify certain hidden customer behavior patterns and use them as additional features. Various sampling techniques are then explored and compared to address the following three most common challenges faced in dealing with marketing data: 1) severely unbalanced historical ground truth, 2) sparsity in the relevant data due to low historical response rates, and 3) high dimensional due to the large variety of campaign channels, programs or themes. To validate our approach, we conducted some preliminary experiments using real-world campaign data. The evaluation shows that the random oversampling approach has the best performance giving the largest area under the curve (AUC) and an up to 160% improvement in the lift index. Finally, we conducted a pilot with one of our clients over a period of five weeks, which aims to help prioritize the campaign contacts to its consumers in Latin America. As a result, the pilot achieved 74% and 132% of lift in terms of customer responses to phone calls and emails in the test group, as compared to those from the control group.