Modeling of human emotion is a challenging problem that can require multiple signals types, as well as contextual information that has been obtained over time. Considering this, in this paper we present our approach, based on physiological signals, to the Emotion Physiology and Experience Collaboration (EPIC) challenge at Affective Computing & Intelligent Interaction (ACII), 2023. In total there are four scenarios that we model: 1) across time; 2) across subjects; 3) across elicitor; and 4) across version. To tackle this challenge, we propose to use a physiological fusion-based approach to solve each scenario. Along with this, we give a detailed analysis of the evaluated physiological signals and personalized predictions for each subject are shown. Our proposed approach shows encouraging results with the lowest root mean square error achieved for scenario 4 (across version) for both valence and arousal on the challenge test set.