Slow feature subspace: A video representation based on slow feature analysis for action recognition

Abstract: This paper proposes a new video representation for subspace-based action recognition. Traditional subspace-based methods represent a video as a subspace by applying principal component analysis (PCA) to its frames. However, this subspace might lead to an imprecise representation of actions, as PCA loses the temporal information of frames. Therefore, we introduce the slow feature subspace based on the slow feature analysis (SFA). SFA extracts a set of slow features of an input video by projecting the video frames onto the weight vectors that minimize the data variance over time. Motivated by these properties, several methods based on SFA were proposed for action recognition. However, they do not explicitly consider the distribution of the slow features, which represents essential action information. Therefore, our key idea is to capture this distribution through a low-dimensional subspace called slow feature subspace. Our subspace is generated by applying PCA to several weight vectors corresponding to the slowest components obtained by SFA. Our framework replaces the subspaces of several traditional mutual subspace methods with our slow feature subspace to improve their results in action recognition. This approach transforms the comparison between two videos into the comparison between two slow feature subspaces using the canonical angles between them, avoiding vector concatenation and data aggregation. The effectiveness of our framework is demonstrated through extensive experiments with various datasets. Our results show that our slow feature subspace can improve the traditional subspace-based methods and achieve competitive performance compared to different methods, including state-of-the-art neural networks.

Download paper here

Recommended citation: Beleza, S. R. A., Shimomoto, E. K., Souza, L. S., & Fukui, K. (2023). Slow feature subspace: A video representation based on slow feature analysis for action recognition. Machine Learning with Applications, 14, 100493.