We present a method for improving segmentation tasks on images affected by adherent rain drops and streaks. We introduce a novel stereo dataset recorded using a system that allows one lens to be affected by real water droplets while keeping the other lens clear. We train a denoising generator using this dataset and show that it is effective at removing the effect of real water droplets, in the context of image reconstruction and road marking segmentation. To further test our de-noising approach, we describe a method of adding computer-generated adherent water droplets and streaks to any images, and use this technique as a proxy to demonstrate the effectiveness of our model in the context of general semantic segmentation. We benchmark our results using the CamVid road marking segmentation dataset, Cityscapes semantic segmentation datasets and our own real-rain dataset, and show significant improvement on all tasks.
We base our simple synthetic droplet model on the works of @cite_30 @cite_27 and @cite_20 , by storing proto-droplet normal maps which are subsequently warped and combined at run time using an approach similar to meta-balls @cite_33 .
@cite_30 @cite_27 @cite_33 @cite_20
2038133417 1585341626 2244686166
In this paper we present a novel approach to improved image registration in rainy weather situations. To this end, we perform monocular raindrop detection in single images based on a photometric raindrop model. Our method is capable of detecting raindrops precisely, even in front of complex backgrounds. The effectiveness is demonstrated by a significant increase in image registration accuracy which also allows for successful image restoration. Experiments on video sequences taken from within a moving vehicle prove the applicability to real-world scenarios. In this paper, we propose a novel raindrop shape model for the detection of view-disturbing, adherent raindrops on inclined surfaces. Whereas state-of-the-art techniques do not consider inclined surfaces because they assume the droplets as sphere sections with equal contact angles, our model incorporates cubic Bezier curves that provide a low dimensional and physically interpretable representation of a raindrop surface. The parameters are empirically deduced from numerous observations of different raindrop sizes and surface inclination angles. It can be easily integrated into a probabilistic framework for raindrop recognition, using geometrical optics to simulate the visual raindrop appearance. In comparison to a sphere section model, the proposed model yields an improved droplet surface accuracy up to three orders of magnitude. The mathematical description of three dimensional surfaces usually falls in one of two classifications: parametric and algebraic. The form is defined as all points which satisfy some equation: F(x,y,z)=0. This form is ideally suited for image space shaded picture drawing, the pixel coordinates are substituted for x and y and the equation is solved for z. Algorithms for drawing such objects have been developed primarily for first and second order polynomial functions. This paper presents a new algorithm applicable to other functional forms, in particular to the summation of several gaussian density distributions. The algorithm was created to model electron density maps of molecular structures but can be used for other artistically interesting shapes.