This paper describes a method for inferring three-dimensional (3D) plant branch structures that are hidden under leaves from multi-view observations. Unlike previous geometric approaches that heavily rely on the visibility of the branches or use parametric branching models, our method makes statistical inferences of branch structures in a probabilistic framework. By inferring the probability of branch existence using a Bayesian extension of image-to-image translation applied to each of multi-view images, our method generates a probabilistic plant 3D model, which represents the 3D branching pattern that cannot be directly observed. Experiments demonstrate the usefulness of the proposed approach in generating convincing branch structures in comparison to prior approaches.
On the other hands, an approach based on observations of real-world trees that uses photographs @cite_31 or 3D scans @cite_14 @cite_32 is shown promising for automatic tree modeling. Several tree modeling approaches using multi-view images have been proposed @cite_1 @cite_7 @cite_35 . These approaches have been further extended to single-image based methods @cite_26 @cite_18 for better applicability. The major focus of the most image-based modeling approaches has been to generate 3D tree models that well fit the silhouette or volumes, not necessarily aiming at recovering branch structures. Using multi-view images of bare trees, previous approaches achieve geometric reconstruction of a branch structure @cite_16 @cite_0 @cite_34 ; however, leaves of plants or trees make 3D reconstruction considerably difficult due to occlusions. The scope of plant modeling now goes beyond computer graphics and is becoming an important application of computer vision, , analysis of plant shapes and growth for vision-assisted cultivation and plant phenotyping. Our method aims to recover branch structures via a multi-view approach with an emphasis on dealing with uncertainty due to severe occlusions by leaves.
@cite_35 @cite_18 @cite_14 @cite_26 @cite_7 @cite_1 @cite_32 @cite_34 @cite_0 @cite_31 @cite_16
2035804958 2332779444 2208368362 2055686029
We present a novel image-based technique for modeling complex unfoliaged trees. Existing tree modeling tools either require capturing a large number of views for dense 3D reconstruction or rely on user inputs and botanic rules to synthesize natural-looking tree geometry. In this paper, we focus on faithfully recovering real instead of realistically-looking tree geometry from a sparse set of images. Our solution directly integrates 2D 3D tree topology as shape priors into the modeling process. For each input view, we first estimate a 2D skeleton graph from its matte image and then find a 2D skeleton tree from the graph by imposing tree topology. We develop a simple but effective technique for computing the optimal 3D skeleton tree most consistent with the 2D skeletons. For each edge in the 3D skeleton tree, we further apply volumetric reconstruction to recover its corresponding curved branch. Finally, we use piecewise cylinders to approximate each branch from the volumetric results. We demonstrate our framework on a variety of trees to illustrate the robustness and usefulness of our technique. State-of-the-art approaches for tree reconstruction either put limiting constraints on the input side (requiring multiple photographs, a scanned point cloud or intensive user input) or provide a representation only suitable for front views of the tree. In this paper we present a complete pipeline for synthesizing and rendering detailed trees from a single photograph with minimal user effort. Since the overall shape and appearance of each tree is recovered from a single photograph of the tree crown, artists can benefit from georeferenced images to populate landscapes with native tree species. A key element of our approach is a compact representation of dense tree crowns through a radial distance map. Our first contribution is an automatic algorithm for generating such representations from a single exemplar image of a tree. We create a rough estimate of the crown shape by solving a thin-plate energy minimization problem, and then add detail through a simplified shape-from-shading approach. The use of seamless texture synthesis results in an image-based representation that can be rendered from arbitrary view directions at different levels of detail. Distant trees benefit from an output-sensitive algorithm inspired on relief mapping. For close-up trees we use a billboard cloud where leaflets are distributed inside the crown shape through a space colonization algorithm. In both cases our representation ensures efficient preservation of the crown shape. Major benefits of our approach include: it recovers the overall shape from a single tree image, involves no tree modeling knowledge and minimal authoring effort, and the associated image-based representation is easy to compress and thus suitable for network streaming. Graphical abstractDisplay Omitted HighlightsAutomatic algorithm for generating crown representations from a single tree image.Compact image-based representation of dense tree crowns suitable for network streaming.Output-sensitive rendering for distant trees.Automatic generation of billboard clouds for detailed trees. We introduce a novel 3D tree skeletonization method from multiple images.Affine transformation and iterative tracking are added to PyrLK optical flow.Parallel voxel flooding is proposed for determining point cloud of single branch.L-System extraction algorithm for tree skeletons is proposed. Display Omitted 3D tree models are widely applied to construct large-scale virtual scenes. However, converting real trees into computer representation faces two main problems. One is the low quality of reconstructed point cloud. The other is that the skeletonization produces inaccurate results due to the complex structure of trees. We propose a novel pipeline to reconstruct automatically accurate 3D tree skeletons from sequential tree images. It involves three steps: (1) Pre-processing: an optical flow based feature-matching algorithm is proposed to acquire high-quality point clouds. (2) Shape-based parallel skeletonization: tree shape is taken into consideration to detect specific structure features of trees during parallel flooding. (3) Post-processing: an L-System extraction algorithm is applied on the skeletons and the lightweight representation is obtained to meet various requirements. The effectiveness of our approach is demonstrated through experiments. We propose a probabilistic formulation of joint silhouette extraction and 3D reconstruction given a series of calibrated 2D images. Instead of segmenting each image separately in order to construct a 3D surface consistent with the estimated silhouettes, we compute the most probable 3D shape that gives rise to the observed color information. The probabilistic framework, based on Bayesian inference, enables robust 3D reconstruction by optimally taking into account the contribution of all views. We solve the arising maximum a posteriori shape inference in a globally optimal manner by convex relaxation techniques in a spatially continuous representation. For an interactively provided user input in the form of scribbles specifying foreground and background regions, we build corresponding color distributions as multivariate Gaussians and find a volume occupancy that best fits to this data in a variational sense. Compared to classical methods for silhouette-based multiview reconstruction, the proposed approach does not depend on initialization and enjoys significant resilience to violations of the model assumptions due to background clutter, specular reflections, and camera sensor perturbations. In experiments on several real-world data sets, we show that exploiting a silhouette coherency criterion in a multiview setting allows for dramatic improvements of silhouette quality over independent 2D segmentations without any significant increase of computational efforts. This results in more accurate visual hull estimation, needed by a multitude of image-based modeling approaches. We made use of recent advances in parallel computing with a GPU implementation of the proposed method generating reconstructions on volume grids of more than 20 million voxels in up to 4.41 seconds.