Abstract: In gesture recognition, static gestures, dynamic gestures and trajectory gestures are collectively known as multi-modal gestures. To solve the existing problem in different recognition methods for different modal gestures, a unified recognition algorithm is proposed. The angle change data of the finger joints and the movement of the centroid of the hand were acquired respectively by data glove and Kinect. Through the preprocessing of the multi-source heterogeneous data, all hand gestures were considered as curves while solving hand shaking, and a uniform hand gesture recognition algorithm was established to calculate the Pearson correlation coefficient between hand gestures for gesture recognition. In this way, complex gesture recognition was transformed into the problem of a simple comparison of curves similarities. The main innovations: 1) Aiming at solving the problem of multi-modal gesture recognition, an unified recognition model and a new algorithm is proposed; 2) The Pearson correlation coefficient for the first time to construct the gesture similarity operator is improved. By testing 50 kinds of gestures, the experimental results showed that the method presented could cope with intricate gesture interaction with the 97.7% recognition rate.

Key words: Kinect, data glove, multi-modal gesture, gesture interaction