
摘要
人体骨骼的动态特征为动作识别提供了重要信息。传统的骨骼建模方法通常依赖于手工设计的部件划分或遍历规则,因而表达能力有限,且难以实现良好的泛化性能。本文提出一种新型动态骨骼建模方法——时空图卷积网络(Spatial-Temporal Graph Convolutional Networks, ST-GCN),通过从数据中自动学习空间与时间模式,突破了以往方法的局限性。该方法不仅显著提升了模型的表达能力,还增强了泛化性能。在Kinetics和NTU-RGBD两个大规模数据集上,该方法均显著优于主流方法。
代码仓库
nntanaka/Fourier-Analysis-for-Skeleton-based-Action-Recognition
pytorch
GitHub 中提及
XinzeWu/st-GCN
pytorch
GitHub 中提及
yysijie/st-gcn
官方
pytorch
GitHub 中提及
hariharannatesh/Action-Recognition-using-Pytorch-Geometric
pytorch
GitHub 中提及
ken724049/action-recognition
GitHub 中提及
ZhangNYG/ST-GCN
pytorch
GitHub 中提及
KrisLee512/ST-GCN
pytorch
GitHub 中提及
ericksiavichay/cs230-final-project
pytorch
GitHub 中提及
open-mmlab/mmskeleton
pytorch
GitHub 中提及
github-zbx/ST-GCN
pytorch
GitHub 中提及
eliasboughosn/Spatial-Temporal-Graph-Convolutional-Networks
pytorch
GitHub 中提及
kennymckormick/pyskl
pytorch
GitHub 中提及
l13025816/PGCN
pytorch
GitHub 中提及
AbiterVX/ST-GCN
pytorch
GitHub 中提及
DixinFan/st-gcn
pytorch
GitHub 中提及
1zgh/st-gcn
pytorch
GitHub 中提及
antoniolq/st-gcn
pytorch
GitHub 中提及
stillarrow/S2VT_ACT
pytorch
GitHub 中提及
TaatiTeam/stgcn_parkinsonism_prediction
pytorch
GitHub 中提及
Powercoder64/TAA-GCN
pytorch
GitHub 中提及
GeyuanZhang/st-gcn-master
pytorch
GitHub 中提及
Tudouu/stgcn_light_op
pytorch
GitHub 中提及
metrics-lab/st-fmri
pytorch
GitHub 中提及
基准测试
| 基准 | 方法 | 指标 |
|---|---|---|
| 3d-human-pose-estimation-on-human36m | ST-GCN | Average MPJPE (mm): 57.4 |
| action-recognition-in-videos-on-icvl-4 | ST-GCN | Accuracy: 80.23% |
| action-recognition-in-videos-on-ird | ST-GCN | Accuracy: 74.03% |
| action-recognition-on-h2o-2-hands-and-objects | ST-GCN | Actions Top-1: 73.86 Hand Pose: 3D Object Label: No Object Pose: Yes RGB: No |
| multimodal-activity-recognition-on-ev-action | ST-GCN (Skeleton Kinect) | Accuracy: 79.6 |
| multimodal-activity-recognition-on-ev-action | ST-GCN (Skeleton Vicon) | Accuracy: 50.7 |
| skeleton-based-action-recognition-on-ntu-rgbd | ST-GCN [PYSKL, 3D Skeleton] | Accuracy (CS): 90.7 Accuracy (CV): 96.5 |
| skeleton-based-action-recognition-on-ntu-rgbd | ST-GCN [Vanilla, 2D Skeleton] | Accuracy (CS): 90.1 Accuracy (CV): 95.1 |
| skeleton-based-action-recognition-on-ntu-rgbd | ST-GCN | Accuracy (CS): 81.5 Accuracy (CV): 88.3 |
| skeleton-based-action-recognition-on-ntu-rgbd | ST-GCN [Vanilla, 3D Skeleton] | Accuracy (CS): 86.6 Accuracy (CV): 93.2 |
| skeleton-based-action-recognition-on-ntu-rgbd-1 | ST-GCN [PYSKL, 3D Skeleton] | Accuracy (Cross-Setup): 88.4 Accuracy (Cross-Subject): 86.2 |
| skeleton-based-action-recognition-on-ntu-rgbd-1 | ST-GCN [PYSKL, 2D Skeleton] | Accuracy (Cross-Setup): 89.0 Accuracy (Cross-Subject): 84.7 |
| skeleton-based-action-recognition-on-uav | ST-GCN | CSv1(%): 30.25 CSv2(%): 56.14 |
| skeleton-based-action-recognition-on-varying | ST-GCN | Accuracy (AV I): 53% Accuracy (AV II): 43% Accuracy (CS): 71% Accuracy (CV I): 25% Accuracy (CV II): 56% |