中国邮电高校学报(英文版) ›› 2019, Vol. 26 ›› Issue (2): 30-42.doi: 10.19682/j.cnki.1005-8885.2019.1004

• Artificial Intelligence • 上一篇    下一篇

Research on unified recognition model and algorithm for multi-modal gestures

Guo Xiaopei, Feng Zhiquan, Sun Kaiyun, Liu Hong, Xie Wei, Bi Jianping   

  1. 1. School of Information Science and Engineering, University of Jinan, Jinan 250022, China
    2. Shandong Provincial Key Laboratory of Network-based Intelligent Computing, Jinan 250022, China
    3. School of Information Science and Engineering, Shandong Normal University, Jinan 250014, China
    4. School of Information and Electrical Engineering, Harbin Institute of Technology at Weihai, Weihai 264209, China
    5. Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan 250022, China
  • 收稿日期:2018-08-07 修回日期:2019-04-04 出版日期:2019-04-30 发布日期:2019-06-14
  • 通讯作者: Corresponding author: Feng Zhiquan, E-mail: ise_fengzq@ujn.edu.cn E-mail:ise_fengzq@ujn.edu.cn
  • 作者简介:Corresponding author: Feng Zhiquan, E-mail: ise_fengzq@ujn.edu.cn
  • 基金资助:
    This work was supported by the National Key R&D Program of China (2018YFB1004901) and the National Natural Science Foundation of China (61472163, 61472232).

Research on unified recognition model and algorithm for multi-modal gestures

Guo Xiaopei, Feng Zhiquan, Sun Kaiyun, Liu Hong, Xie Wei, Bi Jianping   

  1. 1. School of Information Science and Engineering, University of Jinan, Jinan 250022, China
    2. Shandong Provincial Key Laboratory of Network-based Intelligent Computing, Jinan 250022, China
    3. School of Information Science and Engineering, Shandong Normal University, Jinan 250014, China
    4. School of Information and Electrical Engineering, Harbin Institute of Technology at Weihai, Weihai 264209, China
    5. Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan 250022, China
  • Received:2018-08-07 Revised:2019-04-04 Online:2019-04-30 Published:2019-06-14
  • Contact: Corresponding author: Feng Zhiquan, E-mail: ise_fengzq@ujn.edu.cn E-mail:ise_fengzq@ujn.edu.cn
  • About author:Corresponding author: Feng Zhiquan, E-mail: ise_fengzq@ujn.edu.cn
  • Supported by:
    This work was supported by the National Key R&D Program of China (2018YFB1004901) and the National Natural Science Foundation of China (61472163, 61472232).

摘要: In gesture recognition, static gestures, dynamic gestures and trajectory gestures are collectively known as multi-modal gestures. To solve the existing problem in different recognition methods for different modal gestures, a unified recognition algorithm is proposed. The angle change data of the finger joints and the movement of the centroid of the hand were acquired respectively by data glove and Kinect. Through the preprocessing of the multi-source heterogeneous data, all hand gestures were considered as curves while solving hand shaking, and a uniform hand gesture recognition algorithm was established to calculate the Pearson correlation coefficient between hand gestures for gesture recognition. In this way, complex gesture recognition was transformed into the problem of a simple comparison of curves similarities. The main innovations: 1) Aiming at solving the problem of multi-modal gesture recognition, an unified recognition model and a new algorithm is proposed; 2) The Pearson correlation coefficient for the first time to construct the gesture similarity operator is improved. By testing 50 kinds of gestures, the experimental results showed that the method presented could cope with intricate gesture interaction with the 97.7% recognition rate.

关键词: Kinect, data glove, multi-modal gesture, gesture interaction

Abstract: In gesture recognition, static gestures, dynamic gestures and trajectory gestures are collectively known as multi-modal gestures. To solve the existing problem in different recognition methods for different modal gestures, a unified recognition algorithm is proposed. The angle change data of the finger joints and the movement of the centroid of the hand were acquired respectively by data glove and Kinect. Through the preprocessing of the multi-source heterogeneous data, all hand gestures were considered as curves while solving hand shaking, and a uniform hand gesture recognition algorithm was established to calculate the Pearson correlation coefficient between hand gestures for gesture recognition. In this way, complex gesture recognition was transformed into the problem of a simple comparison of curves similarities. The main innovations: 1) Aiming at solving the problem of multi-modal gesture recognition, an unified recognition model and a new algorithm is proposed; 2) The Pearson correlation coefficient for the first time to construct the gesture similarity operator is improved. By testing 50 kinds of gestures, the experimental results showed that the method presented could cope with intricate gesture interaction with the 97.7% recognition rate.

Key words: Kinect, data glove, multi-modal gesture, gesture interaction

中图分类号: