We propose an algorithm to predict room layout from a single image thatgeneralizes across panoramas and perspective images, cuboid layouts and moregeneral layouts (e.g. L-shape room). Our method operates directly on thepanoramic image, rather than decomposing into perspective images as do recentworks. Our network architecture is similar to that of RoomNet, but we showimprovements due to aligning the image based on vanishing points, predictingmultiple layout elements (corners, boundaries, size and translation), andfitting a constrained Manhattan layout to the resulting predictions. Our methodcompares well in speed and accuracy to other existing work on panoramas,achieves among the best accuracy for perspective images, and can handle bothcuboid-shaped and more general Manhattan layouts.