Deep Neural Networks system for Automatic Emotion Recognition

Research in the field of automatic human emotion recognition has shown that changes in emotions can lead to physiological responses in humans. Typically, sensors placed on the body are used to collect physiological data for automatic emotion recognition. However, recent studies have demonstrated that heart rate can be estimated from human face videos by extracting photoplethysmographic (PPG) signals based on the RGB color changes in the face during the cardiac cycle. This study proposes a novel framework for classifying human emotions using contactless PPG signals. The study evaluated PPG signal extraction methods using a popular emotional database, and for classification, a deep learning architecture combining a 1D convolutional neural network (1DCNN) and a long short-term memory (LSTM) was adopted after normalization and signal segmentation. The proposed method achieved a recognition rate of 68.9\% and 54.2\% for binary classification of valence and arousal, respectively, with a signal segmentation of 5 seconds. It is important to note that the main aim of this study is to introduce a new approach to automatic emotion recognition using deep learning techniques based on contactless sensing of physiological signals.