This work addresses the problem of semantic scene understanding under fog. Although marked progress has been made in semantic scene understanding, it is mainly concentrated on clear-weather scenes. Extending semantic segmentation methods to adverse weather conditions such as fog is crucial for outdoor applications. In this paper, we propose a novel method, named Curriculum Model Adaptation (CMAda), which gradually adapts a semantic segmentation model from light synthetic fog to dense real fog in multiple steps, using both labeled synthetic foggy data and unlabeled real foggy data. The method is based on the fact that the results of semantic segmentation in moderately adverse conditions (light fog) can be bootstrapped to solve the same problem in highly adverse conditions (dense fog). CMAda is extensible to other adverse conditions and provides a new paradigm for learning with synthetic data and unlabeled real data. In addition, we present three other main stand-alone contributions: 1) a novel method to add synthetic fog to real, clear-weather scenes using semantic input; 2) a new fog density estimator; 3) a novel fog densification method to densify the fog in real foggy scenes without using depth; and 4) the Foggy Zurich dataset comprising 3808 real foggy images, with pixel-level semantic annotations for 40 images under dense fog. Our experiments show that 1) our fog simulation and fog density estimator outperform their state-of-the-art counterparts with respect to the task of semantic foggy scene understanding (SFSU); 2) CMAda improves the performance of state-of-the-art models for SFSU significantly, benefiting both from our synthetic and real foggy data. The datasets and code are available at the project website.
Fog fades the color of observed objects and reduces their contrast. Extensive research has been conducted on image defogging (dehazing) to increase the visibility of foggy scenes @cite_35 @cite_45 @cite_49 @cite_25 @cite_69 @cite_2 @cite_52 . Certain works focus particularly on enhancing foggy road scenes @cite_60 @cite_5 . Recent approaches also rely on trainable architectures @cite_36 , which have evolved to end-to-end models @cite_12 @cite_19 . For a comprehensive overview of defogging dehazing algorithms, we point the reader to @cite_41 @cite_53 . Our work is complementary and focuses on semantic foggy scene understanding. This work investigates the usefulness of image dehazing for semantic scene understanding under fog.
@cite_35 @cite_69 @cite_60 @cite_36 @cite_41 @cite_53 @cite_52 @cite_19 @cite_45 @cite_49 @cite_2 @cite_5 @cite_25 @cite_12
2095543923 2099015219 1552073401 2748021867 2058843488 2518979500 2536722097 2293894573 1923499759 1494869093 1980939919 2963074253 2963780738 2698984883
One source of accidents when driving a vehicle is the presence of homogeneous and heterogeneous fog. Fog fades the colors and reduces the contrast of the observed objects with respect to their distances. Various camera-based Advanced Driver Assistance Systems (ADAS) can be improved if efficient algorithms are designed for visibility enhancement of road images. The visibility enhancement algorithm proposed in [1] is not dedicated to road images and thus it leads to limited quality results on images of this kind. In this paper, we interpret the algorithm in [1] as the inference of the local atmospheric veil subject to two constraints. From this interpretation, we propose an extended algorithm which better handles road images by taking into account that a large part of the image can be assumed to be a planar road. The advantages of the proposed local algorithm are its speed, the possibility to handle both color images or gray-level images, and its small number of parameters. A comparative study and quantitative evaluation with other state-of-the-art algorithms is proposed on synthetic images with several types of generated fog. This evaluation demonstrates that the new algorithm produces similar quality results with homogeneous fog and that it is able to better deal with the presence of heterogeneous fog. One source of accidents when driving a vehicle is the presence of fog. Fog fades the colors and reduces the contrasts in the scene with respect to their distances from the driver. Various camera-based Advanced Driver Assistance Systems (ADAS) can be improved if efficient algorithms are designed for visibility enhancement in road images. The visibility enhancement algorithm proposed in [1] is not optimized for road images. In this paper, we reformulate the problem as the inference of the local atmospheric veil from constraints. The algorithm in [1] thus becomes a particular case. From this new derivation, we propose to better handle road images by introducing an extra constraint taking into account that a large part of the image can be assumed to be a planar road. The advantages of the proposed local algorithm are the speed, the possibility to handle both color and gray-level images, and the small number of parameters. A new scheme is proposed for rating visibility enhancement algorithms based on the addition of several types of generated fog on synthetic and camera images. A comparative study and quantitative evaluation with other state-of-the-art algorithms is thus proposed. This evaluation demonstrates that the new algorithm produces better results with homogeneous fog and that it is able to deal better with the presence of heterogeneous fog. Finally, we also propose a model allowing to evaluate the potential safety benefit of an ADAS based on the display of defogged images. We propose a referenceless perceptual fog density prediction model based on natural scene statistics (NSS) and fog aware statistical features. The proposed model, called Fog Aware Density Evaluator (FADE), predicts the visibility of a foggy scene from a single image without reference to a corresponding fog-free image, without dependence on salient objects in a scene, without side geographical camera information, without estimating a depth-dependent transmission map, and without training on human-rated judgments. FADE only makes use of measurable deviations from statistical regularities observed in natural foggy and fog-free images. Fog aware statistical features that define the perceptual fog density index derive from a space domain NSS model and the observed characteristics of foggy images. FADE not only predicts perceptual fog density for the entire image, but also provides a local fog density index for each patch. The predicted fog density using FADE correlates well with human judgments of fog density taken in a subjective study on a large foggy image database. As applications, FADE not only accurately assesses the performance of defogging algorithms designed to enhance the visibility of foggy images, but also is well suited for image defogging. A new FADE-based referenceless perceptual image defogging, dubbed DEnsity of Fog Assessment-based DEfogger (DEFADE) achieves better results for darker, denser foggy images as well as on standard foggy images than the state of the art defogging methods. A software release of FADE and DEFADE is available online for public use: http: live.ece.utexas.edu research fog index.html . This work addresses the problem of semantic foggy scene understanding (SFSU). Although extensive research has been performed on image dehazing and on semantic scene understanding with clear-weather images, little attention has been paid to SFSU. Due to the difficulty of collecting and annotating foggy images, we choose to generate synthetic fog on real images that depict clear-weather outdoor scenes, and then leverage these partially synthetic data for SFSU by employing state-of-the-art convolutional neural networks (CNN). In particular, a complete pipeline to add synthetic fog to real, clear-weather images using incomplete depth information is developed. We apply our fog synthesis on the Cityscapes dataset and generate Foggy Cityscapes with 20,550 images. SFSU is tackled in two ways: (1) with typical supervised learning, and (2) with a novel type of semi-supervised learning, which combines (1) with an unsupervised supervision transfer from clear-weather images to their synthetic foggy counterparts. In addition, we carefully study the usefulness of image dehazing for SFSU. For evaluation, we present Foggy Driving, a dataset with 101 real-world images depicting foggy driving scenes, which come with ground truth annotations for semantic segmentation and object detection. Extensive experiments show that (1) supervised learning with our synthetic data significantly improves the performance of state-of-the-art CNN for SFSU on Foggy Driving; (2) our semi-supervised learning strategy further improves performance; and (3) image dehazing marginally advances SFSU with our learning strategy. The datasets, models and code are made publicly available. In the frog and haze climatic condition, the captured picture will become blurred and the color is partial gray and white, due to the effect of atmospheric scattering. This situation brings a great deal of inconvenience to the video surveillance system, so the study of defogging algorithm in this weather is of great importance. This paper deeply analyzes the physical process of imaging in foggy weather. After full study on the haze removal algorithm of single image over the last decade, we propo se a fast haze removal algorithm which based on a fast bilateral filtering combined with dark colors prior. This algorithm starts with the atmospheric scattering model, derives a estimated transmission map by using dark channel prior, and then combines with grayscale to extract refined transmission map by using the fast bilateral filter. This algorithm has a fast execution speed and greatly improves the original algorithm which is morre time-consuming. On this basis, we analyzed the reasons why the image is dim after the haze removal using dark channel prior, and proposed the improved transmission map formula. Experimental-results show that this algorithm is feasible which effectively restores the contrast and color of the scene, significantly improves the visual effects of the image. Those image with large area of sky usually prone to distortion when using the dark channel prior, Therefore we propose a method of weakening the sky region, aims to improve the adaptability of the algorithm. Most existing image dehazing methods tend to boost local image contrast for regions with heavy haze. Without special treatment, these methods may significantly amplify existing image artifacts such as noise, color aliasing and blocking, which are mostly invisible in the input images but are visually intruding in the results. This is especially the case for low quality cellphone shots or compressed video frames. The recent work of (2014) addresses blocking artifacts for dehazing, but is insufficient to handle other artifacts. In this paper, we propose a new method for reliable suppression of different types of visual artifacts in image and video dehazing. Our method makes contributions in both the haze estimation step and the image recovery step. Firstly, an image-guided, depth-edge-aware smoothing algorithm is proposed to refine the initial atmosphere transmission map generated by local priors. In the image recovery process, we propose Gradient Residual Minimization (GRM) for jointly recovering the haze-free image while explicitly minimizing possible visual artifacts in it. Our evaluation suggests that the proposed method can generate results with much less visual artifacts than previous approaches for lower quality inputs such as compressed video clips. One source of difficulties when processing outdoor images is the presence of haze, fog or smoke which fades the colors and reduces the contrast of the observed objects. We introduce a novel algorithm and variants for visibility restoration from a single image. The main advantage of the proposed algorithm compared with other is its speed: its complexity is a linear function of the number of image pixels only. This speed allows visibility restoration to be applied for the first time within real-time processing applications such as sign, lane-marking and obstacle detection from an in-vehicle camera. Another advantage is the possibility to handle both color images or gray level images since the ambiguity between the presence of fog and the objects with low color saturation is solved by assuming only small objects can have colors with low saturation. The algorithm is controlled only by a few parameters and consists in: atmospheric veil inference, image restoration and smoothing, tone mapping. A comparative study and quantitative evaluation is proposed with a few other state of the art algorithms which demonstrates that similar or better quality results are obtained. Finally, an application is presented to lane-marking extraction in gray level images, illustrating the interest of the approach. Video and images acquired by a visual system are seriously degraded under hazy and foggy weather, which will affect the detection, tracking, and recognition of targets. Thus, restoring the true scene from such a foggy video or image is of significance. The main goal of this paper was to summarize current video and image defogging algorithms. We first presented a review of the detection and classification method of a foggy image. Then, we summarized existing image defogging algorithms, including image restoration algorithms, image contrast enhancement algorithms, and fusion-based defogging algorithms. We also presented current video defogging algorithms. We summarized objective image quality assessment methods that have been widely used for the comparison of different defogging algorithms, followed by an experimental comparison of various classical image defogging algorithms. Finally, we presented the problems of video and image defogging which need to be further studied. The code of all algorithms will be available at http: www.yongxu.org lunwen.html . We present a method to jointly estimate scene depth and recover the clear latent image from a foggy video sequence. In our formulation, the depth cues from stereo matching and fog information reinforce each other, and produce superior results than conventional stereo or defogging algorithms. We first improve the photo-consistency term to explicitly model the appearance change due to the scattering effects. The prior matting Laplacian constraint on fog transmission imposes a detail-preserving smoothness constraint on the scene depth. We further enforce the ordering consistency between scene depth and fog transmission at neighboring points. These novel constraints are formulated together in an MRF framework, which is optimized iteratively by introducing auxiliary variables. The experiment results on real videos demonstrate the strength of our method. In this paper we introduce a novel approach to restore a single image degraded by atmospheric phenomena such as fog or haze. The presented algorithm allows for fast identification of hazy regions of an image, without making use of expensive optimization and refinement procedures. By applying a single per pixel operation on the original image, we produce a 'semi-inverse' of the image. Based on the hue disparity between the original image and its semi-inverse, we are then able to identify hazy regions on a per pixel basis. This enables for a simple estimation of the airlight constant and the transmission map. Our approach is based on an extensive study on a large data set of images, and validated based on a metric that measures the contrast but also the structural changes. The algorithm is straightforward and performs faster than existing strategies while yielding comparative and even better results. We also provide a comparative evaluation against other recent single image dehazing methods, demonstrating the efficiency and utility of our approach. In this paper, we propose a novel learning-based approach for single image dehazing. The proposed approach is mostly inspired by the observation that the color of the objects fades gradually along with the increment of the scene depth. We regard the RGB values of the pixels within the image as the important feature, and use the back propagation neural network to mine the internal link between color and depth from the training samples, which consists of the hazy images and their corresponding ground truth depth map. With the trained neural network, we can easily restore the depth information as well as the scene radiance from the hazy image. Experimental results show that the proposed approach is able to produce a high-quality haze-free image with the single hazy image and achieve the real-time requirement. In this paper, we present an end-to-end network, called Cycle-Dehaze, for single image dehazing problem, which does not require pairs of hazy and corresponding ground truth images for training. That is, we train the network by feeding clean and hazy images in an unpaired manner. Moreover, the proposed approach does not rely on estimation of the atmospheric scattering model parameters. Our method enhances CycleGAN formulation by combining cycle-consistency and perceptual losses in order to improve the quality of textural information recovery and generate visually better haze-free images. Typically, deep learning models for dehazing take low resolution images as input and produce low resolution outputs. However, in the NTIRE 2018 challenge on single image dehazing, high resolution images were provided. Therefore, we apply bicubic downscaling. After obtaining low-resolution outputs from the network, we utilize the Laplacian pyramid to upscale the output images to the original resolution. We conduct experiments on NYU-Depth, I-HAZE, and O-HAZE datasets. Extensive experiments demonstrate that the proposed approach improves CycleGAN method both quantitatively and qualitatively. Abstract This paper provides a comprehensive survey of methods dealing with visibility enhancement of images taken in hazy or foggy scenes. The survey begins with discussing the optical models of atmospheric scattering media and image formation. This is followed by a survey of existing methods, which are categorized into: multiple image methods, polarizing filter-based methods, methods with known depth, and single-image methods. We also provide a benchmark of a number of well-known single-image methods, based on a recent dataset provided by Fattal (2014) and our newly generated scattering media dataset that contains ground truth images for quantitative evaluation. To our knowledge, this is the first benchmark using numerical metrics to evaluate dehazing techniques. This benchmark allows us to objectively compare the results of existing methods and to better identify the strengths and limitations of each method. Outdoor images taken in bad weather conditions, such as haze and fog, look faded and have reduced contrast. Recently there has been great success in single image dehazing, i.e., improving the visibility and restoring the colors from a single image. A crucial step in these methods is the calculation of the air-light color, the color of an area of the image with no objects in line-of-sight. We propose a new method for calculating the air-light. The method relies on the haze-lines prior that was recently introduced. This prior is based on the observation that the pixel values of a hazy image can be modeled as lines in RGB space that intersect at the air-light. We use Hough transform in RGB space to vote for the location of the air-light. We evaluate the proposed method on an existing dataset of real world images, as well as some synthetic and other real images. Our method performs on-par with current state-of-the-art techniques and is more computationally efficient.