Li Jie ; Han Kai ; Wang Peng ; Liu Yu ; Yuan Xia

Abstract

As a voxel-wise labeling task, semantic scene completion (SSC) tries tosimultaneously infer the occupancy and semantic labels for a scene from asingle depth and/or RGB image. The key challenge for SSC is how to effectivelytake advantage of the 3D context to model various objects or stuffs with severevariations in shapes, layouts and visibility. To handle such variations, wepropose a novel module called anisotropic convolution, which properties withflexibility and power impossible for the competing methods such as standard 3Dconvolution and some of its variations. In contrast to the standard 3Dconvolution that is limited to a fixed 3D receptive field, our module iscapable of modeling the dimensional anisotropy voxel-wisely. The basic idea isto enable anisotropic 3D receptive field by decomposing a 3D convolution intothree consecutive 1D convolutions, and the kernel size for each such 1Dconvolution is adaptively determined on the fly. By stacking multiple suchanisotropic convolution modules, the voxel-wise modeling capability can befurther enhanced while maintaining a controllable amount of model parameters.Extensive experiments on two SSC benchmarks, NYU-Depth-v2 and NYUCAD, show thesuperior performance of the proposed method. Our code is available athttps://waterljwant.github.io/SSC/