Pang Yunsheng ; Ke Qiuhong ; Rahmani Hossein ; Bailey James ; Liu Jun

Abstract

Human interaction recognition is very important in many applications. Onecrucial cue in recognizing an interaction is the interactive body parts. Inthis work, we propose a novel Interaction Graph Transformer (IGFormer) networkfor skeleton-based interaction recognition via modeling the interactive bodyparts as graphs. More specifically, the proposed IGFormer constructsinteraction graphs according to the semantic and distance correlations betweenthe interactive body parts, and enhances the representation of each person byaggregating the information of the interactive body parts based on the learnedgraphs. Furthermore, we propose a Semantic Partition Module to transform eachhuman skeleton sequence into a Body-Part-Time sequence to better capture thespatial and temporal information of the skeleton sequence for learning thegraphs. Extensive experiments on three benchmark datasets demonstrate that ourmodel outperforms the state-of-the-art with a significant margin.