Skeleton-Based Action Recognition with Gated Convolutional Neural Networks

For skeleton-based action recognition, most of the existing work used recurrent neural networks. Using convolutional neural networks is another attractive solution in consideration of their advantages in parallelization, effectiveness in feature learning and model base sufficiency. Besides these, skeleton data are low-dimensional features. It is natural to arrange a sequence of skeleton features chronologically into an image which retains the original information. Therefore, we solve the sequence learning problem as an image classification task using convolutional neural networks. For better learning ability, we build a classification network with stacked residual blocks and having a special design called linear skip gated connection which can benefit information propagation across multiple residual blocks. When arranging the coordinates of body joints in one frame into a skeleton feature, we systematically investigate the performance of part-based, chain-based and traversal-based orders. Furthermore, a fully-convolutional permutation network is designed to learn an optimized order for data rearrangement. Without any bells and whistles, our proposed model achieves state-of-the-art performance on two challenging benchmark datasets, outperforming existing methods significantly.