Sultani Waqas Chen Chen Shah Mubarak

Abstract

Surveillance videos are able to capture a variety of realistic anomalies. Inthis paper, we propose to learn anomalies by exploiting both normal andanomalous videos. To avoid annotating the anomalous segments or clips intraining videos, which is very time consuming, we propose to learn anomalythrough the deep multiple instance ranking framework by leveraging weaklylabeled training videos, i.e. the training labels (anomalous or normal) are atvideo-level instead of clip-level. In our approach, we consider normal andanomalous videos as bags and video segments as instances in multiple instancelearning (MIL), and automatically learn a deep anomaly ranking model thatpredicts high anomaly scores for anomalous video segments. Furthermore, weintroduce sparsity and temporal smoothness constraints in the ranking lossfunction to better localize anomaly during training. We also introduce a newlarge-scale first of its kind dataset of 128 hours of videos. It consists of1900 long and untrimmed real-world surveillance videos, with 13 realisticanomalies such as fighting, road accident, burglary, robbery, etc. as well asnormal activities. This dataset can be used for two tasks. First, generalanomaly detection considering all anomalies in one group and all normalactivities in another group. Second, for recognizing each of 13 anomalousactivities. Our experimental results show that our MIL method for anomalydetection achieves significant improvement on anomaly detection performance ascompared to the state-of-the-art approaches. We provide the results of severalrecent deep learning baselines on anomalous activity recognition. The lowrecognition performance of these baselines reveals that our dataset is verychallenging and opens more opportunities for future work. The dataset isavailable at: https://webpages.uncc.edu/cchen62/dataset.html