loading page

Multimodal Temporal Modeling of Emotion using Physiological Signals
  • +1
  • Saurabh Hinduja ,
  • Maneesh Bilalpur ,
  • Liza Jivnani ,
  • Shaun Canavan
Saurabh Hinduja
University of Pittsburgh

Corresponding Author:[email protected]

Author Profile
Maneesh Bilalpur
Author Profile
Liza Jivnani
Author Profile
Shaun Canavan
Author Profile


Modeling of human emotion is a challenging problem that can require multiple signals types, as well as contextual information that has been obtained over time. Considering this, in this paper we present our approach, based on physiological signals, to the Emotion Physiology and Experience Collaboration (EPIC) challenge at Affective Computing & Intelligent Interaction (ACII), 2023. In total there are four scenarios that we model: 1) across time; 2) across subjects; 3) across elicitor; and 4) across version. To tackle this challenge, we propose to use a physiological fusion-based approach to solve each scenario. Along with this, we give a detailed analysis of the evaluated physiological signals and personalized predictions for each subject are shown. Our proposed approach shows encouraging results with the lowest root mean square error achieved for scenario 4 (across version) for both valence and arousal on the challenge test set.