Multiple feature fusion for unimodal emotion recognition

doi:10.19682/j.cnki.1005-8885.2019.1003

中国邮电高校学报(英文) ›› 2019, Vol. 26 ›› Issue (2): 17-29.doi: 10.19682/j.cnki.1005-8885.2019.1003

• Artificial Intelligence • 上一篇下一篇

Multiple feature fusion for unimodal emotion recognition

Yang Lingzhi, Ban Xiaojuan, Michele Mukeshimana, Chen Zhe

1. School of Computer and Communication Engineering, Beijing Key Laboratory of Knowledge Engineering for Materials Science, University of Science and Technology Beijing, Beijing 10083, China
2. Citic Pacific Special Steel Holdings Qingdao Special Iron and Steel Company Limited
3. Faculity of Engineering Sciences, University of Burundi, Bujumbura P. O. Box 1550 Bujumbura, Burundi
4. Qingdao Hisense Group Company Limited, Qingdao 266000, China

收稿日期:2018-08-07 修回日期:2019-04-04 出版日期:2019-04-30 发布日期:2019-06-14
通讯作者: Corresponding author: Yang Lingzhi, E-mail: yanglingzhi@citicsteel.com E-mail:yanglingzhi@citicsteel.com
作者简介:Corresponding author: Yang Lingzhi, E-mail: yanglingzhi@citicsteel.com
基金资助:
This work was supported by the National Key Research and Development Program of China (2016YFB1001404), and the National Natural Science Foundation of China (61873299, 61702036, 61572075).

Multiple feature fusion for unimodal emotion recognition

Yang Lingzhi, Ban Xiaojuan, Michele Mukeshimana, Chen Zhe

1. School of Computer and Communication Engineering, Beijing Key Laboratory of Knowledge Engineering for Materials Science, University of Science and Technology Beijing, Beijing 10083, China
2. Citic Pacific Special Steel Holdings Qingdao Special Iron and Steel Company Limited
3. Faculity of Engineering Sciences, University of Burundi, Bujumbura P. O. Box 1550 Bujumbura, Burundi
4. Qingdao Hisense Group Company Limited, Qingdao 266000, China

Received:2018-08-07 Revised:2019-04-04 Online:2019-04-30 Published:2019-06-14
Contact: Corresponding author: Yang Lingzhi, E-mail: yanglingzhi@citicsteel.com E-mail:yanglingzhi@citicsteel.com
About author:Corresponding author: Yang Lingzhi, E-mail: yanglingzhi@citicsteel.com
Supported by:
This work was supported by the National Key Research and Development Program of China (2016YFB1001404), and the National Natural Science Foundation of China (61873299, 61702036, 61572075).

摘要/Abstract

摘要： A new semi-serial fusion method of multiple feature based on learning using privileged information (LUPI) model was put forward. The exploitation of LUPI paradigm permits the improvement of the learning accuracy and its stability, by additional information and computations using optimization methods. The execution time is also reduced, by sparsity and dimension of testing feature. The essence of improvements obtained using multiple features types for the emotion recognition (speech expression recognition), is particularly applicable when there is only one modality but still need to improve the recognition. The results show that the LUPI in unimodal case is effective when the size of the feature is considerable. In comparison to other methods using one type of features or combining them in a concatenated way, this new method outperforms others in recognition accuracy, execution reduction, and stability.

关键词: multiple feature, LUPI, emotion recognition, semi-serial fusion method

Abstract: A new semi-serial fusion method of multiple feature based on learning using privileged information (LUPI) model was put forward. The exploitation of LUPI paradigm permits the improvement of the learning accuracy and its stability, by additional information and computations using optimization methods. The execution time is also reduced, by sparsity and dimension of testing feature. The essence of improvements obtained using multiple features types for the emotion recognition (speech expression recognition), is particularly applicable when there is only one modality but still need to improve the recognition. The results show that the LUPI in unimodal case is effective when the size of the feature is considerable. In comparison to other methods using one type of features or combining them in a concatenated way, this new method outperforms others in recognition accuracy, execution reduction, and stability.

Key words: multiple feature, LUPI, emotion recognition, semi-serial fusion method

中图分类号:

TP391

Yang Lingzhi, Ban Xiaojuan, Michele Mukeshimana, Chen Zhe. Multiple feature fusion for unimodal emotion recognition[J]. The Journal of China Universities of Posts and Telecommunications, 2019, 26(2): 17-29.

参考文献

References
1. Mehrabian A, Epstein N. A measure of emotional empathy. Journal of Personality, 2006, 40(4): 525 -543
2. Valdez P, Mehrabian A. Effects of color on emotions. Journal of Experimental Psychology: General, 1994, 123(4): 394 -409
3. Tian L, Moore J, Lai C. Recognizing emotions in spoken dialogue with hierarchically fused acoustic and lexical features. Spoken Language Technology Workshop, 2017
4. Voyer D, Myles D. The effect of unimodal affective priming on dichotic emotion recognition. Laterality: Asymmetries of Body, Brain and Cognition, 2017: 1 -21
5. Gharavian D, Bejani M, Sheikhan M. Audio-visual emotion recognition using FCBF feature selection method and particle swarm optimization for fuzzy ARTMAP neural networks. Multimedia Tools and Applications, 2017, 76(2): 2331 -2352
6. Miura N, Nagasaka A, Miyatake T. Feature extraction of finger vein patterns based on iterative line tracking and its application to personal identification. Systems and Computers in Japan, 2004, 35(7): 61 -71
7. Faustino N B, Rocio G M, Belen P G M, et al. Acoustic voice analysis using the PRAAT programme: comparative study with the Dr. speech programme. Acta Otorrinolaringologica Espanola, 2014
8. Koolagudi R. Real life emotion classification using VOP and pitch based spectral features. India Conference. IEEE, 2011
9. Kim S, Eriksson T, Kang H G, et al. A pitch synchronous feature extraction method for speaker recognition. 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing. IEEE Computer Society, 2004
10. Abuelquran A R, Goubran R A. Pitch-based feature extraction for audio classification. IEEE Internatioal Workshop on Haptic, Audio and Visual Environments and Their Applications. IEEE, 2003
11. Roh Y W, Kim D J, Lee W S, et al. Novel acoustic features for speech emotion recognition. Science in China Series E: Technological Sciences, 2009, 52(7): 1838 -1848

[1]	Guo Xiangbo, Wang Jian, Huang Mengjie, Wang Minghui, Yang Jian, Yu Yongtao. Deep knowledge tracking algorithm based on forgetting law[J]. 中国邮电高校学报(英文版), 2023, 30(1): 17-27.
[2]	Wang Xianlun, Wang Guangyu, Cui Yuxia. Facial expression recognition based on improved ResNet[J]. 中国邮电高校学报(英文版), 2023, 30(1): 28-38.
[3]	Wu Hongxin, Lin Zhijian, Chen Pingping, Chen Feng. Joint partial computation offloading and resource allocation in MEC-enable networks[J]. 中国邮电高校学报(英文版), 2023, 30(1): 80-86.
[4]	Kong Chao, Ou Weihua, Gong Xiaofeng, Li Weian, Han Jie, Yao Yi, Xiong Jiahao. Face anti-spoofing based on multi-modal and multi-scale features fusion [J]. 中国邮电高校学报(英文版), 2022, 29(6): 73-82.
[5]	张函井音吉赵永利. FS-LSTM: sales forecasting in e-commerce on feature selection [J]. 中国邮电高校学报(英文版), 2022, 29(5): 92-98.
[6]	Shi Jinjing, Wang Wenxuan, Xiao Zimeng, Mu Shuai, Li Qin. Quantum classifier with parameterized quantum circuit based on the isolated quantum system[J]. 中国邮电高校学报(英文版), 2022, 29(4): 21-31.
[7]	Liu Hailing, Zhang Jie, Qin Sujuan, Gao Fei. Quantum algorithm for soft margin support vector machine with hinge loss function[J]. 中国邮电高校学报(英文版), 2022, 29(4): 32-41.
[8]	Wang Jian, Qiao Kuoyuan, Yuan Yanlei, Liu Xiaole, Yang Jian. Adaptive learning path recommendation model for examination-oriented education[J]. 中国邮电高校学报(英文版), 2022, 29(4): 77-88.
[9]	Meng Wei, Wang Liting, Lu Meng. Summary of research on recommendation system based on serendipity[J]. 中国邮电高校学报(英文版), 2022, 29(4): 89-105.
[10]	Jia Wei, Gong Chao. Precise and efficient Chinese license plate recognition in the real monitoring scene of intelligent transportation system[J]. 中国邮电高校学报(英文版), 2022, 29(3): 1-14.
[11]	Song Yue, Wu Chengmao, Tian Xiaoping, Song Qiuyu. Enhanced kernel-based fuzzy local information clustering integrating neighborhood membership [J]. 中国邮电高校学报(英文版), 2021, 28(6): 65-81.
[12]	Xue Chenzi, Wei Yifei, Zhang Yong. Performance optimization for smart grid blockchain integrated with fog computing using DDQN[J]. 中国邮电高校学报(英文版), 2021, 28(2): 68-78.
[13]	Guo Hairu, Meng Xueyao, Liu Yongli, Liu Shen. Improved HHO algorithm based on good point set and nonlinear convergence formula[J]. 中国邮电高校学报(英文版), 2021, 28(2): 48-67.
[14]	吴成茂曹卓. Entropy-like distance driven fuzzy clustering with local information constraints for image segmentation [J]. 中国邮电高校学报(英文版), 2021, 28(1): 24-40.
[15]	山蕊蒋林邓军勇崔朋飞张玉婷吴皓月谢晓燕. Parallel design of convolutional neural networks for remote sensing images object recognition based on data-driven array processor [J]. 中国邮电高校学报(英文版), 2020, 27(6): 87-100.

Multiple feature fusion for unimodal emotion recognition

Multiple feature fusion for unimodal emotion recognition

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价