A Hybrid Approach to Speech Emotion Recognition with Audio TextAlignment скачать в хорошем качестве

A Hybrid Approach to Speech Emotion Recognition with Audio TextAlignment

Speaker: Shahane Tigranyan (CAST) Topic: A Hybrid Approach to Speech Emotion Recognition with Audio TextAlignment DataFest Yerevan 2025, https://datafest.am/ Abstract: Speech emotion recognition with a multimodal approach is an important part of affective computing, enabling machines to understand and respond to human emotions more effectively. Due to the scarcity of labeled emotional datasets, maximizing the extraction of relevant information from available data remains a significant challenge. To address this issue, we propose ATENet, a bimodal neural network designed to enhance SER by processing information at both the sentence-level and the word-level. In addition to the bimodal scenario, ATENet introduces a novel alignment branch with two interconnected components: one processes aligned audio segments, while the other handles corresponding word tokens. The addition of the alignment branch enhances model performance compared to the standard bimodal scenario, highlighting its contribution to better speech-text feature integration for SER.