Existing state-of-the-art salient object detection networks rely onaggregating multi-level features of pre-trained convolutional neural networks(CNNs). Compared to high-level features, low-level features contribute less toperformance but cost more computations because of their larger spatialresolutions. In this paper, we propose a novel Cascaded Partial Decoder (CPD)framework for fast and accurate salient object detection. On the one hand, theframework constructs partial decoder which discards larger resolution featuresof shallower layers for acceleration. On the other hand, we observe thatintegrating features of deeper layers obtain relatively precise saliency map.Therefore we directly utilize generated saliency map to refine the features ofbackbone network. This strategy efficiently suppresses distractors in thefeatures and significantly improves their representation ability. Experimentsconducted on five benchmark datasets exhibit that the proposed model not onlyachieves state-of-the-art performance but also runs much faster than existingmodels. Besides, the proposed framework is further applied to improve existingmulti-level feature aggregation models and significantly improve theirefficiency and accuracy.