Emotion analysis through biological signal processing

Abstract

In our incessant need to make machines intelligent, a tendency to try to make them interact with human beings is arising, including both decision making and emotional behavior analysis. Emotional intelligence is composed of several components, the main ones being the expression of emotions, and the interpretation of the emotions perceived. In this thesis we pretend to study the design of automatic systems able to interpret and recognize emotions emitted by a human. Human beings externalize their emotions in several conscious and unconscious ways. In speech we find different markers that can guide us to the analysis of the emotion experienced by the subject when it comes to speaking, but almost all the physiological signals generated by the human body are also affected by emotions. When the subject experiences an emotional state, the hear rate, the breathing, the brain and the skin inevitably response and change in consequence, and physiological signals such as the electrocardiogram, the thoracic impedance, or the electrodermal activity, are affected by these changes. In the present thesis, an attempt is made to investigate the way of extracting the most valuable information regarding the emotion activity from multiple biological process. For this purpose, the speech, the electrocardiogram, the thoracic impedance and the electrodermal activity, recorded in both the hand and arm, have been studied and analyzed. Several systems have been proposed, and a final real-time smartphonebased implementation with physiological signals has been implemented and tested. In order to characterize each of the used signals, we present an important bibliographic review with those features that contain information about emotion, activity and stress. Each identified feature has been tested and implemented, both offline and in real-time, and a comparative study has been carried out of those that contain the most relevant information. In order to know the importance of the features used, evolutive-based feature selection techniques have been proposed. Concerning the methods of classification used to analyze emotion, activity and stress, we have compared the behavior of several different classifiers, with the aim of determining which type of classifier offers the best performance in our application. The classifiers evaluated are the least squares linear classifier, the least squares quadratic classifier, support vector machines and multilayer perceptron neural networks. Numerous problems and difficulties have been found in obtaining information from emotions. Due to the difficulty of capturing physiological signals at the moment when the subject really feels a particular emotion is elicited, the available databases are composed of a very low number of subjects, which leads to generalization problems. To mitigate this problem we propose several solutions, one of them consisting in virtually enlarging the number of subjects in the design data, and other consisting in a novel way to initialize the learning process of artificial neural networks. In the thesis we have analyzed the relevancy of the features and signals under study, as well as that the classifier that renders a better performance overcoming the generalization problems in each case. Results demonstrate the suitability of the proposals, offering a trade-off between computational complexity and performance in terms of error probability.