ImpactBot: Chatbot Leveraging Language Models to Automate Feedback and Promote Critical Thinking Around Impact Statements

Abstract

Impact statements articulate the impacts of a research project with concise and unambiguous statements about problems addressed, actions to resolve, and explanations of any impacts. Researchers and technologists often rely on impact statements as means to provoke introspective critical thinking around the impacts of technology being developed. However, due to factors such as technocentrism, positivity bias, marketization, or hyperinflation of impact statements, the claims presented in these statements do not cover all important aspects when creating technology – for instance, negative and delayed impacts. This work contributes to the development of a chatbot called ImpactBot to promote critical thinking while researchers create impact statements for research projects or scientific papers. The proposed chatbot leverages two fine-tuned state-of-the-art RoBERTa models for sequence classification and was assessed in this case study with 5 researchers from a large information technology company and 7 university engineering research scientists or students. This approach may be reused as part of content management or a paper submission system, for instance, to dialogue with researchers and promote critical thinking about negative impacts and how to mitigate them (if any) while creating impact statements for their projects or scientific papers.