This paper presents a new deep-learning based method to simultaneously calibrate the intrinsic parameters of fisheye lens and rectify the distorted images. Assuming that the distorted lines generated by fisheye projection should be straight after rectification, we propose a novel deep neural network to impose explicit geometry constraints onto processes of the fisheye lens calibration and the distorted image rectification. In addition, considering the nonlinearity of distortion distribution in fisheye images, the proposed network fully exploits multi-scale perception to equalize the rectification effects on the whole image. To train and evaluate the proposed model, we also create a new largescale dataset labeled with corresponding distortion parameters and well-annotated distorted lines. Compared with the state-of-the-art methods, our model achieves the best published rectification quality and the most accurate estimation of distortion parameters on a large set of synthetic and real fisheye images.
Another topic closely related to our work is distorted lines extraction in fisheye images. Various arc detection methods and optimizing strategies have been utilized in the calibrating process @cite_25 @cite_27 @cite_22 @cite_35 @cite_15 , but they are not robust to detect arcs especially in the environments with noises or texture absence. Although recent deep learning based methods @cite_26 @cite_3 @cite_34 @cite_12 @cite_0 show a promising performance on edge detection, none of them is well qualified to deal with ditorted lines in fisheye images.
@cite_35 @cite_26 @cite_22 @cite_3 @cite_0 @cite_27 @cite_15 @cite_34 @cite_25 @cite_12
1910129379 2105060035 2564804471 2017428553
Fisheye image rectification and estimation of intrinsic parameters for real scenes have been addressed in the literature by using line information on the distorted images. In this paper, we propose an easily implemented fisheye image rectification algorithm with line constrains in the undistorted perspective image plane. A novel Multi-Label Energy Optimization (MLEO) method is adopted to merge short circular arcs sharing the same or the approximately same circular parameters and select long circular arcs for camera rectification. Further we propose an efficient method to estimate intrinsic parameters of the fisheye camera by automatically selecting three properly arranged long circular arcs from previously obtained circular arcs in the calibration procedure. Experimental results on a number of real images and simulated data show that the proposed method can achieve good results and outperforms the existing approaches and the commercial software in most cases. The use of super-wide angle and fish-eye lenses causes strong distortions in the resulting images. A methodology for the correction of distortions in these cases using only single images and linearity of imaged objects is presented. Contrary to most former algorithms, the algorithm discussed here does not depend on information about the real world co-ordinates of matching points. Moreover reference points determination and camera calibration is not required in this case. The algorithm is based on circle fitting. It requires only the possibility of the extraction of distorted image points from straight lines in the 3D scene. Further, the actual distortion must approximately fit the chosen distortion model. For most fish-eye lenses appropriate distortion correction results can be obtained. We present a direct visual odometry algorithm for a fisheye-stereo camera. Our algorithm performs simultaneous camera motion estimation and semi-dense reconstruction. The pipeline consists of two threads: a tracking thread and a mapping thread. In the tracking thread, we estimate the camera pose via semi-dense direct image alignment. To have a wider field of view (FoV) which is important for robotic perception, we use fisheye images directly without converting them to conventional pinhole images which come with a limited FoV. To address the epipolar curve problem, plane-sweeping stereo is used for stereo matching and depth initialization. Multiple depth hypotheses are tracked for selected pixels to better capture the uncertainty characteristics of stereo matching. Temporal motion stereo is then used to refine the depth and remove false positive depth hypotheses. Our implementation runs at an average of 20 Hz on a low-end PC. We run experiments in outdoor environments to validate our algorithm, and discuss the experimental results. We experimentally show that we are able to estimate 6D poses with low drift, and at the same time, do semi-dense 3D reconstruction with high accuracy. To the best of our knowledge, there is no other existing semi-dense direct visual odometry algorithm for a fisheye-stereo camera. This paper presents a hierarchical approach for fast and robust ellipse extraction from images. At the lowest level, the image is described as a set of edge pixels, from which line segments are extracted. Then, line segments that are potential candidates of elliptic arcs are linked to form arc segments according to connectivity and curvature relations. After that, arc segments that belong to the same ellipse are grouped together. Finally, a robust statistical method, namely RANSAC, is applied to fit ellipses. This method does not need a high dimensional parameter space like Hough transform based algorithms, and so it reduces the computation and memory requirements. Experiments on both synthetic and real images demonstrate that the proposed method has excellent performance in handling occlusion and overlapping ellipses.