Spatio Temporal LSTM with Aggregated Feature Learning for Human Action Recognition
Human action recognition has received much research attention in recent years. Discriminative features learning is important for modeling the spatial and temporal evolutions of different actions. Recent works attempted to use RNN/LSTM to learn spatio temporal features of joints. However, the features of joints are not fully explored in these conventional LSTMs, where each joint only receives context information from itself from previous frames. In this paper, we propose a LSTM network model by using aggregated feature learning strategy to extend feature representation in both spatial and temporal domains. In this model, the features of a joint are aggregated not only from all joints at previous moments, but also at current moment. Therefore, action related features within the human skeleton sequences can be better represented and analyzed. To further optimize the network, we present a reparameterization that involves batch normalization and sparsemax classifier for the proposed LSTM network. We evaluate our approach on NTU RGB+D benchmark dataset. Experiment results demonstrate the effectiveness of the proposed approaches.
2019 IEEE International Conference on Real time Computing and Robotics (RCAR)