Li Kun ; Guo Dan ; Chen Guoliang ; Fan Chunxiao ; Xu Jingyuan ; Wu Zhiliang ; Fan Hehe ; Wang Meng

Abstract

Micro-Action Recognition (MAR) has gained increasing attention due to itscrucial role as a form of non-verbal communication in social interactions, withpromising potential for applications in human communication and emotionanalysis. However, current approaches often overlook the inherent ambiguity inmicro-actions, which arises from the wide category range and subtle visualdifferences between categories. This oversight hampers the accuracy ofmicro-action recognition. In this paper, we propose a novel PrototypicalCalibrating Ambiguous Network (PCAN) to unleash and mitigate the ambiguity ofMAR. Firstly, we employ a hierarchical action-tree to identify the ambiguoussample, categorizing them into distinct sets of ambiguous samples of falsenegatives and false positives, considering both body- and action-levelcategories. Secondly, we implement an ambiguous contrastive refinement moduleto calibrate these ambiguous samples by regulating the distance betweenambiguous samples and their corresponding prototypes. This calibration processaims to pull false negative (FN) samples closer to their respective prototypesand push false positive (FP) samples apart from their affiliated prototypes. Inaddition, we propose a new prototypical diversity amplification loss tostrengthen the model's capacity by amplifying the differences between differentprototypes. Finally, we propose a prototype-guided rectification to rectifyprediction by incorporating the representability of prototypes. Extensiveexperiments conducted on the benchmark dataset demonstrate the superiorperformance of our method compared to existing approaches. The code isavailable at https://github.com/kunli-cs/PCAN.