Kong Shu Shen Xiaohui Lin Zhe Mech Radomir Fowlkes Charless

Abstract

Real-world applications could benefit from the ability to automaticallygenerate a fine-grained ranking of photo aesthetics. However, previous methodsfor image aesthetics analysis have primarily focused on the coarse, binarycategorization of images into high- or low-aesthetic categories. In this work,we propose to learn a deep convolutional neural network to rank photoaesthetics in which the relative ranking of photo aesthetics are directlymodeled in the loss function. Our model incorporates joint learning ofmeaningful photographic attributes and image content information which can helpregularize the complicated photo aesthetics rating problem. To train and analyze this model, we have assembled a new aesthetics andattributes database (AADB) which contains aesthetic scores and meaningfulattributes assigned to each image by multiple human raters. Anonymized rateridentities are recorded across images allowing us to exploit intra-raterconsistency using a novel sampling strategy when computing the ranking loss oftraining image pairs. We show the proposed sampling strategy is very effectiveand robust in face of subjective judgement of image aesthetics by individualswith different aesthetic tastes. Experiments demonstrate that our unified modelcan generate aesthetic rankings that are more consistent with human ratings. Tofurther validate our model, we show that by simply thresholding the estimatedaesthetic scores, we are able to achieve state-or-the-art classificationperformance on the existing AVA dataset benchmark.