Diaz-Arias Alec ; Shin Dmitriy

Abstract

Recently, fully-transformer architectures have replaced the defactoconvolutional architecture for the 3D human pose estimation task. In this paperwe propose \textbf{\textit{ConvFormer}}, a novel convolutional transformer thatleverages a new \textbf{\textit{dynamic multi-headed convolutionalself-attention}} mechanism for monocular 3D human pose estimation. We designeda spatial and temporal convolutional transformer to comprehensively model humanjoint relations within individual frames and globally across the motionsequence. Moreover, we introduce a novel notion of \textbf{\textit{temporaljoints profile}} for our temporal ConvFormer that fuses complete temporalinformation immediately for a local neighborhood of joint features. We havequantitatively and qualitatively validated our method on three common benchmarkdatasets: Human3.6M, MPI-INF-3DHP, and HumanEva. Extensive experiments havebeen conducted to identify the optimal hyper-parameter set. These experimentsdemonstrated that we achieved a \textbf{significant parameter reductionrelative to prior transformer models} while attaining State-of-the-Art (SOTA)or near SOTA on all three datasets. Additionally, we achieved SOTA for ProtocolIII on H36M for both GT and CPN detection inputs. Finally, we obtained SOTA onall three metrics for the MPI-INF-3DHP dataset and for all three subjects onHumanEva under Protocol II.