About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
NeurIPS 2020
Workshop paper
The Design and Development of Games with a Purpose for AI Systems
Abstract
Collecting training data is often time-consuming, expensive and imposes a bottleneck on many machine learning tasks. Much of training data used to train ML systems is a result of the work of crowdworkers who are paid to do routinizable mundane tasks. Games with a purpose leverage game mechanics to use the perceptive capacities of users to collect data in a way that is more enjoyable to crowdworkers. Different machine learning tasks require different types of training data. In this paper, we discuss the design and development of building two games with a purpose: Guess the Word and Fool the AI, designed to collect data from both crowdworkers and domain experts for two very different machine learning problems. To make these games enjoyable and interactive, a team of engineers, research scientists and designers create new games with a purpose around various machine learning tasks. In this paper, we describe the design of these games, how we incorporate game mechanics within these games to make the collection of annotation tasks more efficient but also enjoyable.