Demands on capturing dynamic scenes of underwater environments are rapidly growing. Passive stereo is applicable to capture dynamic scenes, however the shape with textureless surfaces or irregular reflections cannot be recovered by the technique. In our system, we add a pattern projector to the stereo camera pair so that artificial textures are augmented on the objects. To use the system at underwater environments, several problems should be compensated, i.e., refraction, disturbance by fluctuation and bubbles. Further, since surface of the objects are interfered by the bubbles, projected patterns, etc., those noises and patterns should be removed from captured images to recover original texture. To solve these problems, we propose three approaches; a depth-dependent calibration, Convolutional Neural Network(CNN)-stereo method and CNN-based texture recovery method. A depth-dependent calibration is our analysis to find the acceptable depth range for approximation by center projection to find the certain target depth for calibration. In terms of CNN stereo, unlike common CNNbased stereo methods which do not consider strong disturbances like refraction or bubbles, we designed a novel CNN architecture for stereo matching using multi-scale information, which is intended to be robust against such disturbances. Finally, we propose a multi-scale method for bubble and a projected-pattern removal method using CNNs to recover original textures. Experimental results are shown to prove the effectiveness of our method compared with the state of the art techniques. Furthermore, reconstruction of a live swimming fish is demonstrated to confirm the feasibility of our techniques.
Passive active stereo can solve such problems. It projects a pattern, but the correspondence between feature points on pattern and captured images are neither used in calibration nor reconstruction. Therefore, refraction on projected pattern can be remained, and rich" pattern can be used. For example, Konolige investigated how passive active stereo works @cite_22 . They tried several types of patterns and showed optimized pattern works better than not optimized ones. For the above reasons, we took passive active stereo method.
@cite_22
1967061262 2131563332 2135064677 2122985851
Passive stereo vision is widely used as a range sensing technology in robots, but suffers from dropouts: areas of low texture where stereo matching fails. By supplementing a stereo system with a strong texture projector, dropouts can be eliminated or reduced. This paper develops a practical stereo projector system, first by finding good patterns to project in the ideal case, then by analyzing the effects of system blur and phase noise on these patterns, and finally by designing a compact projector that is capable of good performance out to 3m in indoor scenes. The system has been implemented and has excellent depth precision and resolution, especially in the range out to 1.5m. We present a novel 3D scanning system combining stereo and active illumination based on phase-shift for robust and accurate scene reconstruction. Stereo overcomes the traditional phase discontinuity problem and allows for the reconstruction of complex scenes containing multiple objects. Due to the sequential recording of three patterns, motion will introduce artifacts in the reconstruction. We develop a closed-form expression for the motion error in order to apply motion compensation on a pixel level. The resulting scanning system can capture accurate depth maps of complex dynamic scenes at 17 fps and can cope with both rigid and deformable objects. Is the real problem in resolving correspondence using current stereo algorithms the lack of the "right" matching criterion? In studying the related task of reconstructing three-dimensional space curves from their projections in multiple views, we suggest that the problem is more basic: matching and reconstruction are coupled, and so reconstruction algorithms should exploit this rather than assuming that matching can be successfully performed before reconstruction. To realize this coupling, a generative model of curves is introduced which has two key components: (i) a prior distribution of general space curves and (ii) an image formation model which describes how 3D curves are projected onto the image plane. A novel aspect of the image formation model is that it uses an exact description of the gradient field of a piecewise constant image. Based on this forward model, a fully automatic algorithm for solving the inverse problem is developed for an arbitrary number of views. The resulting algorithm is robust to partial occlusion, deficiencies in image curve extraction and it does not rely on photometric information. The relative motion of the cameras is assumed to be given. Several experiments are carried out on various realistic scenarios. In particular, we focus on scenes where traditional correlation-based methods would fail. In this paper, we address the stereo matching problem in the presence of reflections and translucency, where image formation can be modeled as the additive superposition of layers at different depth. The presence of such effects violates the Lambertian assumption underlying traditional stereo vision algorithms, making it impossible to recover component depths using direct color matching based methods. We develop several techniques to estimate both depths and colors of the component layers. Depth hypotheses are enumerated in pairs, one from each layer, in a nested plane sweep. For each pair of depth hypotheses, we compute a component-color-independent matching error per pixel, using a spatial-temporal differencing technique. We then use graph cut optimization to solve for the depths of both layers. This is followed by an iterative color update algorithm whose convergence is proven in our paper. We show convincing results of depth and color estimates for both synthetic and real image sequences.