Learning representations from quadrilateral based geometric features for skeleton-based action recognition using LSTM networks

Abstract

With the recent developments in sensor technology and pose estimation algorithms, skeleton based action recognition has become popular. Classical machine learning methods based on hand-crafted features fail on large scale datasets due to their limited representation power. Recently, recurrent neural networks (RNN) based methods focus on the temporal evolution of body joints and neglect the geometric relations between them. In this paper, we propose eleven quadrilaterals to capture the geometric relations among joints for action recognition. An end-to-end 3-layer Bi-LSTM network is designed as Base-Net to learn robust representations. We propose two subnets based on the Base-Net to extract discriminative spatio temporal features. Specifically, the first subnet (SQuadNet) uses four spatial features and the second one (TQuadNet) uses two temporal features. The empirical results on two benchmark datasets, NTU RGB $+$ D and UTD MHAD, show how our method achieves state of the art performance when compared to recent methods in the literature.