Gamifying the Human-in-the-Loop: Toward Increased Motivation for Training AI in Customer Service

Abstract

In this paper, we contribute to research on human-AI collaboration in the scope of Hybrid Intelligence Systems, which enable mutual augmentation and collaborative learning of both human and AI. Thereby, we address a research gap focusing on the continuance intention of customer service employees to teach AI during their work task. So far, the human-in-the-loop (HITL) approach is commonly applied to directly involve the human user in Machine Learning (ML) to actively advance AI. However, there is only little consideration of users’ motivation regarding the extra effort of teaching AI during their work. To address this gap, we combine gamification and ML toward increased motivation to participate in HITL learning. Therefore, we follow the Design Science Research process and align to a framework for gamifying collaboration processes. Thus, we identify meta-requirements toward intended outcomes of gamified collaboration based on expert interviews, consequently derive design principles to gamify the process interactions and determine matching gamification elements. For demonstration, we implement the design principles with the according gamification elements in a prototype enabling customer service employees to provide feedback to an AI. Eventually, we evaluate the design principles with the prototype in user test simulations. The results reveal their successful implementation as well as the relevance of mixed gamification elements to trigger various motivation types. Additionally, we show applicability of the framework to gamify human-AI collaboration.