Dense 3D shape acquisition of swimming human or live fish is an important research topic for sports, biological science and so on. For this purpose, active stereo sensor is usually used in the air, however it cannot be applied to the underwater environment because of refraction, strong light attenuation and severe interference of bubbles. Passive stereo is a simple solution for capturing dynamic scenes at underwater environment, however the shape with textureless surfaces or irregular reflections cannot be recovered. Recently, the stereo camera pair with a pattern projector for adding artificial textures on the objects is proposed. However, to use the system for underwater environment, several problems should be compensated, i.e., disturbance by fluctuation and bubbles. Simple solution is to use convolutional neural network for stereo to cancel the effects of bubbles and or water fluctuation. Since it is not easy to train CNN with small size of database with large variation, we develop a special bubble generation device to efficiently create real bubble database of multiple size and density. In addition, we propose a transfer learning technique for multi-scale CNN to effectively remove bubbles and projected-patterns on the object. Further, we develop a real system and actually captured live swimming human, which has not been done before. Experiments are conducted to show the effectiveness of our method compared with the state of the art techniques.
Abstract. In-situ calibration of structured light scanners in underwater environments is time-consuming and complicated. This paper presents a self-calibrating line laser scanning system, which enables the creation of dense 3D models with a single fixed camera and a freely moving hand-held cross line laser projector. The proposed approach exploits geometric constraints, such as coplanarities, to recover the depth information and is applicable without any prior knowledge of the position and orientation of the laser projector. By employing an off-the-shelf underwater camera and a waterproof housing with high power line lasers an affordable 3D scanning solution can be built. In experiments the performance of the proposed technique is studied and compared with 3D reconstruction using explicit calibration. We demonstrate that the scanning system can be applied to above-the-water as well as underwater scenes. We consider the problem of deliberately manipulating the direct and indirect light flowing through a time-varying, general scene in order to simplify its visual analysis. Our approach rests on a crucial link between stereo geometry and light transport: while direct light always obeys the epipolar geometry of a projector-camera pair, indirect light overwhelmingly does not. We show that it is possible to turn this observation into an imaging method that analyzes light transport in real time in the optical domain, prior to acquisition. This yields three key abilities that we demonstrate in an experimental camera prototype: (1) producing a live indirect-only video stream for any scene, regardless of geometric or photometric complexity; (2) capturing images that make existing structured-light shape recovery algorithms robust to indirect transport; and (3) turning them into one-shot methods for dynamic 3D shape capture.  Underwater 3D shape scanning technique becomes popular because ofseveral rising research topics, such as map making ofsubmarine topography for autonomous underwater vehicle (UAV), shape measurement of live fish, motion capture of swimming human, etc. Structured light systems (SLS) based active 3D scanning systems are widely used in the air and also promising to apply underwater environment. When SLS is used in the air, the stereo correspondences can be efficiently retrieved by epipolar constraint. However, in the underwater environment, the camera and projector are usually set in special housings and refraction occurs at the interfaces between water glass and glass air, resulting in invalid conditions for epipolar constraint which severely deteriorates the correspondence search process. In this paper, we propose an efficient technique to calibrate the underwater SLS systems as well as robust 3D shape acquisition technique. In order to avoid the calculation complexity, we approximate the system with central projection model. Although such an approximation produces an inevitable errors in the system, such errors are diminished by a combination of grid based SLS technique and a bundle adjustment algorithm. We tested our method with a real underwater SLS, consisting ofcustom-made laser pattern projector and underwater housings, showing the validity ofour method. A Laser-based Structured Light System (LbSLS) has been designed to perform underwater close-range 3D reconstructions even with high turbidity conditions and outperform conventional systems. The system uses a camera and a 532 nm green laser projector. The optical technique used is based on the projection of a pattern obtained placing a Diffractive Optical Element (DOE) in front of the laser beam. In the experiments described in this manuscript, the DOE used diffracts the laser beam in 25 parallel lines providing enough information in a single camera frame to perform a 3D reconstruction. Commercial RGB-D cameras provide the possibility of fast, accurate, and cost-effective 3-D scanning solution in a single package. These economical depth cameras provide several advantages over conventional depth sensors, such as sonars and lidars, in specific usage scenarios. In this paper, we analyze the performance of Kinect v2 time-of-flight camera while operating fully submerged underwater in a customized waterproof housing. Camera calibration has been performed for Kinect’s RGB and NIR cameras, and the effect of calibration on the generated 3-D mesh is discussed in detail. To overcome the effect of refraction of light due to the sensor housing and water, we propose a time-of-flight correction method and a fast, accurate and intuitive refraction correction method that can be applied to the acquired depth images, during 3-D mesh generation. Experimental results show that the Kinect v2 can acquire point cloud data up to 650 mm. The reconstruction results have been analyzed qualitatively and quantitatively, and confirm that the 3-D reconstruction of submerged objects at small distances is possible without the requirement of any external NIR light source. The proposed algorithms successfully generated 3-D mesh with a mean error of ±6 mm at a frame rate of nearly 10 fps. We acquired a large data set of RGB, IR and depth data from a submerged Kinect v2. The data set covers a large variety of objects scanned underwater and is publicly available for further use, along with the Kinect waterproof housing design and correction filter codes. The research is aimed toward small-scale research activities and economical solution for 3-D scanning underwater. Applications such as coral reef mapping and underwater SLAM in shallow waters for ROV’s can be a viable application area that can benefit from results achieved. We propose a new method to analyze light transport in homogeneous scattering media. The incident light undergoes multiple bounces in translucent objects, and produces a complex light field. Our method analyzes the light transport in two steps. First, single and multiple scattering are separated by projecting high-frequency stripe patterns. Then, multiple scattering is decomposed into each bounce component based on the light transport equation. The light field for each bounce is recursively estimated. Experimental results show that light transport in scattering media can be decomposed and visualized for each bounce.