AI trust in business processes: The need for process-aware explanations

Abstract

Business processes underpin a large number of enterprise operations including processing loan applications, managing invoices, and insurance claims. There is a large opportunity for infusing AI to reduce cost or provide better customer experience and the BPM literature is rich in machine learning solutions. More recently, deep learning models have been applied to process predictions. Unfortunately, companies have applied or adopted very few of these innovations. We assert that a reason for this lack of adoption is that business users are risk-averse and do not implicitly trust AI models. We challenge the BPM community to build on the AI interpretability literature, and the AI Trust community to understand what it means to take advantage of business process artifacts in order to provide business level explanations.