Early work of automatic modeling of emotional gestures depends largely on hand-crafted features [1]}, [2]}, [3]}. Recently, numerous neural networks have been introduced for gesture/action recognition. Among them, supervised learning is the predominate technique for which labeled data are utilized to train the models. The earliest attempts utilized a 2D convolutional neural network (CNN) [4]}, [5]}, [6]}, [7]}, [8]}, [9]} to extract spatial features from the selected frames, and the temporal aggregation is considered by an additional stream of optical flow or the temporal pooling layers. The 3D CNNs [10]}, [11]}, [12]}, [13]}, [14]}, [15]}, [16]} can jointly capture spatial-temporal semantics, where the filters are designed in a 3D manner. Compared to the 2D CNNs, 3D ones can process the temporal information hierarchically throughout the whole network. Also, some models like the Slow-fast [17]} considered a joint implementation of both two streams (fast and slow) and 3D CNN. The RNN is also commonly used for temporal integration. Specifically, the long short-term memories (LSTMs) [18]}, [19]}, [20]}, [21]}, [22]}, [23]}, [24]}, [25]} have demonstrated their strength on learning sequential data. Recently, the skeleton data is gaining increasingly popularity because of their invariance to background dynamics. Current work on skeleton based action recognition can mainly be categorized into two types: one is RNN based methods [19]}, [24]}, [28]}, [25]}, [30]}, [31]} which directly process gesture skeletons as time series, and the other one is graph convolutional network (GCN) [32]}, [33]}, [34]}, [35]}, [36]}, [37]}, [38]} based methods which reorganize the skeleton data as a graph.
<FIGURE>