Hits:

Title of Paper:Multi-Scale Adaptive Skeleton Transformer foraction recognition

Journal:Computer Vision and Image Understanding

Key Words:Skeleton-based action recognition, Transformer, Position encoding, Multi-scale representation

Summary:Transformer has demonstrated remarkable performance in various computer vision tasks. However, its potential is not fully explored in skeleton-based action recognition. On one hand, existing methods primarily utilize fixed function or pre-learned matrix to encode position information, while overlooking the sample-specific position information. On the other hand, these approaches focus on single-scale spatial relationships, while neglecting the discriminative fine-grained and coarse-grained spatial features. To address these issues, we propose a Multi-Scale Adaptive Skeleton Transformer (MSAST), including Adaptive Skeleton Position Encoding Module (ASPEM), Multi-Scale Embedding Module (MSEM), and Adaptive Relative Location Module (ARLM). ASPEM decouples spatial–temporal information in the position encoding procedure, which acquires inherent dependencies of skeleton sequences. ASPEM is also designed to be dependent on input tokens, which can learn sample-specific position information. The MSEM employs multi-scale pooling to generate multi-scale tokens that contain multi-grained features. Then, the spatial transformer captures multi-scale relations to address the subtle differences between various actions. Another contribution of this paper is that ARLM is presented to mine suitable location information for better recognition performance. Extensive experiments conducted on three benchmark datasets demonstrate that the proposed model achieves Top-1 accuracy of 94.9%/97.5% on NTU-60 C-Sub/C-View, 88.7%/91.6% on NTU-120 X-Sub/X-Set and 97.4% on NW-UCLA, respectively.

All the Authors:Kai Chen,Zhifu Zhao,Guangming Shi,Xuemei Xie,Xiang Jiang,Yifan Yang

First Author:Xiaotian Wang

Correspondence Author:Zhifu Zhao

Document Code:104229

Document Type:J

Volume:250

Issue:Q2, 三区，IF=4.8

Translation or Not:No

Date of Publication:2024-11-08

Date:2025-12-16

Impact Factor:4.8