Tang Shitao ; Feng Litong ; Kuang Zhangkui ; Chen Yimin ; Zhang Wei

Abstract

Detection of video shot transition is a crucial pre-processing step in videoanalysis. Previous studies are restricted on detecting sudden content changesbetween frames through similarity measurement and multi-scale operations arewidely utilized to deal with transitions of various lengths. However,localization of gradual transitions are still under-explored due to the highvisual similarity between adjacent frames. Cut shot transitions are abruptsemantic breaks while gradual shot transitions contain low-levelspatial-temporal patterns caused by video effects in addition to the gradualsemantic breaks, e.g. dissolve. In order to address the problem, we propose astructured network which is able to detect these two shot transitions usingtargeted models separately. Considering speed performance trade-offs, we designa smart framework. With one TITAN GPU, the proposed method can achieve a30(\times) real-time speed. Experiments on public TRECVID07 and RAI databasesshow that our method outperforms the state-of-the-art methods. In order totrain a high-performance shot transition detector, we contribute a new databaseClipShots, which contains 128636 cut transitions and 38120 gradual transitionsfrom 4039 online videos. ClipShots intentionally collect short videos for morehard cases caused by hand-held camera vibrations, large object motions, andocclusion.