The Journal of China Universities of Posts and Telecommunications ›› 2019, Vol. 26 ›› Issue (2): 30-42.doi: 10.19682/j.cnki.1005-8885.2019.1004

• Artificial intelligence • Previous Articles     Next Articles

Research on unified recognition model and algorithm for multi-modal gestures

Guo Xiaopei, Feng Zhiquan, Sun Kaiyun, Liu Hong, Xie Wei, Bi Jianping   

  1. 1. School of Information Science and Engineering, University of Jinan, Jinan 250022, China
    2. Shandong Provincial Key Laboratory of Network-based Intelligent Computing, Jinan 250022, China
    3. School of Information Science and Engineering, Shandong Normal University, Jinan 250014, China
    4. School of Information and Electrical Engineering, Harbin Institute of Technology at Weihai, Weihai 264209, China
    5. Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan 250022, China
  • Received:2018-08-07 Revised:2019-04-04 Online:2019-04-30 Published:2019-06-14
  • Contact: Corresponding author: Feng Zhiquan, E-mail:
  • About author:Corresponding author: Feng Zhiquan, E-mail:
  • Supported by:
    This work was supported by the National Key R&D Program of China (2018YFB1004901) and the National Natural Science Foundation of China (61472163, 61472232).

Abstract: In gesture recognition, static gestures, dynamic gestures and trajectory gestures are collectively known as multi-modal gestures. To solve the existing problem in different recognition methods for different modal gestures, a unified recognition algorithm is proposed. The angle change data of the finger joints and the movement of the centroid of the hand were acquired respectively by data glove and Kinect. Through the preprocessing of the multi-source heterogeneous data, all hand gestures were considered as curves while solving hand shaking, and a uniform hand gesture recognition algorithm was established to calculate the Pearson correlation coefficient between hand gestures for gesture recognition. In this way, complex gesture recognition was transformed into the problem of a simple comparison of curves similarities. The main innovations: 1) Aiming at solving the problem of multi-modal gesture recognition, an unified recognition model and a new algorithm is proposed; 2) The Pearson correlation coefficient for the first time to construct the gesture similarity operator is improved. By testing 50 kinds of gestures, the experimental results showed that the method presented could cope with intricate gesture interaction with the 97.7% recognition rate.

Key words: Kinect, data glove, multi-modal gesture, gesture interaction

CLC Number: