Improving Real Time Emotion Recognition Accuracy by Comparing CNN-Based Fusion of Physiology and Speech Signals with MFCC Based Speech Models

The objective of this study is to design a CNN-based multimodal emotion recognition system using speech and ECG signals to improve recognition accuracy. Group 1 represents the existing speech-based emotion recognition system using MFCC features with a CNN model. Group 2 represents the proposed multimodal system combining MFCC speech features and ECG signals using CNN-based fusion. The evaluation of the system was done using Accuracy, Precision, Recall, and F1-score measures. The multimodal system's accuracy was higher, i.e., 92.8%, than the existing system, which is based on speech alone, with an accuracy of 78.5%. The results proved that the system using speech and ECG is more accurate and reliable, making it appropriate for emotion recognition in real-time applications.