We propose an algorithm to predict room layout from a single image that generalizes across panoramas and perspective images, cuboid layouts and more general layouts (e.g. L-shape room). Our method operates directly on the panoramic image, rather than decomposing into perspective images as do recent works. Our network architecture is similar to that of RoomNet, but we show improvements due to aligning the image based on vanishing points, predicting multiple layout elements (corners, boundaries, size and translation), and fitting a constrained Manhattan layout to the resulting predictions. Our method compares well in speed and accuracy to other existing work on panoramas, achieves among the best accuracy for perspective images, and can handle both cuboid-shaped and more general Manhattan layouts.
Single-view room layout estimation has been an active topic of research for the past ten years. Delage al @cite_13 fit floor wall boundaries in a perspective image taken by a level camera to create a 3D model under Manhattan world'' assumptions @cite_16 . The Manhattan world assumptions are that all walls are at right angles to each other and perpendicular to the floor. A special case is the cuboid model, in which four walls, ceiling, and floor enclose the room. Lee al @cite_6 produce Orientation Maps, generate layout hypotheses based on detected line segments, and select a best-fitting layout from among them. Hedau al @cite_0 recover cuboid layouts by solving for three vanishing points, sampling layouts consistent with those vanishing points, and selecting the best layout based on edge and Geometric Context @cite_24 consistencies. Subsequent works follow a similar approach, with improvements to layout generation @cite_8 @cite_33 @cite_4 , features for scoring layouts @cite_33 @cite_4 , and incorporation of object hypotheses @cite_25 @cite_9 @cite_34 @cite_17 @cite_26 or other context. The most recent methods train deep network features to classify pixels into layout surfaces (walls, floor, ceiling) @cite_15 @cite_2 , boundaries @cite_31 , corners @cite_27 , or a combination @cite_7 .
@cite_31 @cite_26 @cite_4 @cite_33 @cite_7 @cite_8 @cite_9 @cite_6 @cite_34 @cite_0 @cite_24 @cite_27 @cite_2 @cite_15 @cite_16 @cite_13 @cite_25 @cite_17
2598108580 2962717701 2794800539 2964180491
This paper focuses on the task of room layout estimation from a monocular RGB image. Prior works break the problem into two sub-tasks: semantic segmentation of floor, walls, ceiling to produce layout hypotheses, followed by an iterative optimization step to rank these hypotheses. In contrast, we adopt a more direct formulation of this problem as one of estimating an ordered set of room layout keypoints. The room layout and the corresponding segmentation is completely specified given the locations of these ordered keypoints. We predict the locations of the room layout keypoints using RoomNet, an end-to-end trainable encoder-decoder network. On the challenging benchmark datasets Hedau and LSUN, we achieve state-of-the-art performance along with 200× to 600× speedup compared to the most recent work. Additionally, we present optional extensions to the RoomNet architecture such as including recurrent computations and memory units to refine the keypoint locations under the same parametric capacity. We propose an algorithm to predict room layout from a single image that generalizes across panoramas and perspective images, cuboid layouts and more general layouts (e.g. "L"-shape room). Our method operates directly on the panoramic image, rather than decomposing into perspective images as do recent works. Our network architecture is similar to that of RoomNet [15], but we show improvements due to aligning the image based on vanishing points, predicting multiple layout elements (corners, boundaries, size and translation), and fitting a constrained Manhattan layout to the resulting predictions. Our method compares well in speed and accuracy to other existing work on panoramas, achieves among the best accuracy for perspective images, and can handle both cuboid-shaped and more general Manhattan layouts. We propose an algorithm to predict room layout from a single image that generalizes across panoramas and perspective images, cuboid layouts and more general layouts (e.g. L-shape room). Our method operates directly on the panoramic image, rather than decomposing into perspective images as do recent works. Our network architecture is similar to that of RoomNet, but we show improvements due to aligning the image based on vanishing points, predicting multiple layout elements (corners, boundaries, size and translation), and fitting a constrained Manhattan layout to the resulting predictions. Our method compares well in speed and accuracy to other existing work on panoramas, achieves among the best accuracy for perspective images, and can handle both cuboid-shaped and more general Manhattan layouts. This paper presents a method of estimating the geometry of a room and the 3D pose of objects from a single 360 panorama image. Assuming ManhattanWorld geometry, we formulate the task as an inference problem in which we estimate positions and orientations of walls and objects. The method combines surface normal estimation, 2D object detection and 3D object pose estimation. Quantitative results are presented on a dataset of synthetically generated 3D rooms containing objects, as well as on a subset of handlabeled images from the public SUN360 dataset.