Multisensed Emotions as Adaptation Controllers in Human-to-Serious NeuroGames Communication

Emotion is defined (from Britannica) as a complex combination of experiences of consciousness, bodily sensation and behavior, reflecting personal significance of externally and/or internally triggered changes. As an indispensable part of human communication, emotions could be represented within the digital world as means that could control the human interaction with the digital artifacts. When this is combined with gamification, a more engaging communication for the player could be established. In this article, we explore how multisensed emotions of players could be used as adaptation controllers within the emotion-aware serious neurogames (SNGs). We first briefly describe the emotion sensing and recognition landscape, examining the emotion-source sensing and emotion inferring approaches. We then discuss the SNGs design framework and the way human-to-SNG communication is set. Additionally, we describe the way the adaptation of the SNG characteristics can be driven by the inferred emotions, towards the achievement of the desired SNG goals. In addition, we demonstrate the feasibility and design characteristics of such multisensed emotion-aware SNGs, by presenting a related application case, namely EmoSense, and evaluate its acceptance as a technological solution in the field of neurofeedback. Finally, we discuss challenges and opportunities in the field.