Zheng Jinkai ; Liu Xinchen ; Liu Wu ; He Lingxiao ; Yan Chenggang ; Mei Tao

Abstract

Existing studies for gait recognition are dominated by 2D representationslike the silhouette or skeleton of the human body in constrained scenes.However, humans live and walk in the unconstrained 3D space, so projecting the3D human body onto the 2D plane will discard a lot of crucial information likethe viewpoint, shape, and dynamics for gait recognition. Therefore, this paperaims to explore dense 3D representations for gait recognition in the wild,which is a practical yet neglected problem. In particular, we propose a novelframework to explore the 3D Skinned Multi-Person Linear (SMPL) model of thehuman body for gait recognition, named SMPLGait. Our framework has twoelaborately-designed branches of which one extracts appearance features fromsilhouettes, the other learns knowledge of 3D viewpoints and shapes from the 3DSMPL model. In addition, due to the lack of suitable datasets, we build thefirst large-scale 3D representation-based gait recognition dataset, namedGait3D. It contains 4,000 subjects and over 25,000 sequences extracted from 39cameras in an unconstrained indoor scene. More importantly, it provides 3D SMPLmodels recovered from video frames which can provide dense 3D information ofbody shape, viewpoint, and dynamics. Based on Gait3D, we comprehensivelycompare our method with existing gait recognition approaches, which reflectsthe superior performance of our framework and the potential of 3Drepresentations for gait recognition in the wild. The code and dataset areavailable at https://gait3d.github.io.