In this paper we present a framework for the rendering of dynamic 3D virtual environments which can be integrated in the development of videogames. It includes methods to manage sounds and particle effects, paged static geometries, the support of a physics engine and various input systems. It has been designed with a modular structure to allow future expansions. We exploited some open-source state-of-the-art components such as OGRE, PhysX, ParticleUniverse, etc.; all of them have been properly integrated to obtain peculiar physical and environmental effects. The stand-alone version of the application is fully compatible with Direct3D and OpenGL APIs and adopts OpenAL APIs to manage audio cards. Concluding, we devised a showcase demo which reproduces a dynamic 3D environment, including some particular effects: the alternation of day and night influencing the lighting of the scene, the rendering of terrain, water and vegetation, the reproduction of sounds and atmospheric agents.
In the area of environmental effects simulation, several methods for reproducing weather phenomena, like particle-based rain techniques, are presented in @cite_2 . The authors describe in details those methods to render, in a real-time system, very complex atmospheric physical phenomena such as strong rainfall, falling raindrops dripping off objects' surfaces, various reflections in surface materials and puddles, water ripples and puddles on the streets and so on.
@cite_2
2119535410 2017416107 2177195834 2209874411 2895618990 1965572510 2525037006 2798744505 2011826789 2170961588 2768189935 2963800716 1976056157 2901852115 2407858842 2282974639 2113968972 2405480216 2097900287 2143390037
The visual effects of rain are complex. Rain consists of spatially distributed drops falling at high velocities. Each drop refracts and reflects the environment, producing sharp intensity changes in an image. A group of such falling drops creates a complex time varying signal in images and videos. In addition, due to the finite exposure time of the camera, intensities due to rain are motion blurred and hence depend on the background intensities. Thus, the visual manifestations of rain are a combination of both the dynamics of rain and the photometry of the environment. In this paper, we present the first comprehensive analysis of the visual effects of rain on an imaging system. We develop a correlation model that captures the dynamics of rain and a physics-based motion blur model that explains the photometry of rain. Based on these models, we develop efficient algorithms for detecting and removing rain from videos. The effectiveness of our algorithms is demonstrated using experiments on videos of complex scenes with moving objects and time-varying textures. The techniques described in this paper can be used in a wide range of applications including video surveillance, vision based navigation, video movie editing and video indexing retrieval. In the context of extracting information from video, bad weather conditions like rain can have a detrimental effect. In this paper, a novel framework to detect and remove rain streaks from video is proposed. The first part of the proposed framework for rain removal is a technique to detect rain streaks based on phase congruency features. The variation of features from frame to frame is used to estimate the candidate rain pixels in a frame. In order to reduce the number of false candidates due to global motion, frames are registered using phase correlation. The second part of the proposed framework is a novel reconstruction technique that utilizes information from three different sources, which are intensities of the rain affected pixel, spatial neighbors, and temporal neighbors. An optimal estimate for the actual intensity of the rain affected pixel is made based on the minimization of registration error between frames. An optical flow technique using local phase information is adopted for registration. This part of the proposed framework for removing rain is modeled such that the presence of local motion will not distort the features in the reconstructed video. The proposed framework is evaluated quantitatively and qualitatively on a variety of videos with varying complexities. The effectiveness of the algorithm is quantitatively verified by computing a no-reference image quality measure on individual frames of the reconstructed video. From a variety of experiments that are performed on output videos, it is shown that the proposed technique performs better than state-of-the-art techniques. Raindrops adhered to a windscreen or window glass can significantly degrade the visibility of a scene. Modeling, detecting and removing raindrops will, therefore, benefit many computer vision applications, particularly outdoor surveillance systems and intelligent vehicle systems. In this paper, a method that automatically detects and removes adherent raindrops is introduced. The core idea is to exploit the local spatio-temporal derivatives of raindrops. To accomplish the idea, we first model adherent raindrops using law of physics, and detect raindrops based on these models in combination with motion and intensity temporal derivatives of the input video. Having detected the raindrops, we remove them and restore the images based on an analysis that some areas of raindrops completely occludes the scene, and some other areas occlude only partially. For partially occluding areas, we restore them by retrieving as much as possible information of the scene, namely, by solving a blending function on the detected partially occluding areas using the temporal intensity derivative. For completely occluding areas, we recover them by using a video completion technique. Experimental results using various real videos show the effectiveness of our method. Visual distortions on images caused by bad weather conditions can have a negative impact on the performance of many outdoor vision systems. One often seen bad weather is rain which causes significant yet complex local intensity fluctuations in images. The paper aims at developing an effective algorithm to remove visual effects of rain from a single rainy image, i.e. separate the rain layer and the de-rained image layer from an rainy image. Built upon a non-linear generative model of rainy image, namely screen blend mode, we proposed a dictionary learning based algorithm for single image de-raining. The basic idea is to sparsely approximate the patches of two layers by very high discriminative codes over a learned dictionary with strong mutual exclusivity property. Such discriminative sparse codes lead to accurate separation of two layers from their non-linear composite. The experiments showed that the proposed method outperformed the existing single image de-raining methods on tested rain images. The primary accomplishment of this research has been demonstrating the power of the deconvolution technique developed by Neuman and de Marsily (1976) in dealing with noisy rainfallrunoff records of short duration. Such records are encountered in arid environments where rainfall often occurs in short isolated bursts and the data are measured with a considerable margin of error. Our research work consisted of superimposing known noise on synthetic rainfallrunoff data and examining the ability of the Neuman -de Marsily deconvolution method to estimate the correct impulse response of the system when the data include only a single storm event. Approximately 50 Monte Carlo simulation runs were performed for each of three different noise models considered in our work. The results clearly demonstrated that the deconvolution model leads to reliable estimates and can be used with confidence in the presence of realistic noise levels. In addition to the Monte Carlo simulation tests and their analysis, certain improvements were introduced into the original deconvolution technique. In particular, the original version of the technique required that the hydrologist exercise subjective judgement in choosing the "best" solution for the deconvolution problem from a large number of admissible solutions. Our new method of selecting the "best" result is based on a comparative analysis of residuals and is more reliable than the earlier subjective approach. The improved method has been applied to real as well as synthetic rainfall runoff data. The visual effects of rain are complex. Rain produces sharp intensity changes in images and videos that can severely impair the performance of outdoor vision systems. In this paper, we provide a comprehensive analysis of the visual effects of rain and the various factors that affect it. Based on this analysis, we develop efficient algorithms for handling rain in computer vision as well as for photorealistic rendering of rain in computer graphics. We first develop a photometric model that describes the intensities produced by individual rain streaks and a dynamic model that captures the spatio-temporal properties of rain. Together, these models describe the complete visual appearance of rain. Using these models, we develop a simple and effective post-processing algorithm for detection and removal of rain from videos. We show that our algorithm can distinguish rain from complex motion of scene objects and other time-varying textures. We then extend our analysis by studying how various factors such as camera parameters, rain properties and scene brightness affect the appearance of rain. We show that the unique physical properties of rain--its small size, high velocity and spatial distribution--makes its visibility depend strongly on camera parameters. This dependence is used to reduce the visibility of rain during image acquisition by judiciously selecting camera parameters. Conversely, camera parameters can also be chosen to enhance the visibility of rain. This ability can be used to develop an inexpensive and portable camera-based rain gauge that provides instantaneous rain-rate measurements. Finally, we develop a rain streak appearance model that accounts for the rapid shape distortions (i.e. oscillations) that a raindrop undergoes as it falls. We show that modeling these distortions allows us to faithfully render the complex intensity patterns that are visible in the case of raindrops that are close to the camera. In this paper, we address a rain removal problem from a single image, even in the presence of heavy rain and rain accumulation. Our core ideas lie in our new rain image models and a novel deep learning architecture. We first modify the commonly used model, which is a linear combination of a rain streak layer and a background layer, by adding a binary map that locates rain streak regions. Second, we create a model consisting of a component representing rain accumulation (where individual streaks cannot be seen, and thus visually similar to mist or fog), and another component representing various shapes and directions of overlapping rain streaks, which normally happen in heavy rain. Based on the first model, we develop a multi-task deep learning architecture that learns the binary rain streak map, the appearance of rain streaks, and the clean background, which is our ultimate output. The additional binary map is critically beneficial, since its loss function can provide additional strong information to the network. In many cases though, rain streaks can be dense and large in their size, thus to obtain the clean background, we need spatial contextual information. For this, we utilize the dilated convolution. To handle rain accumulation (again, a phenomenon visually similar to mist or fog) and various shapes and directions of overlapping rain streaks, we propose an iterative information feedback (IIF) network that removes rain streaks and clears up the rain accumulation iteratively and progressively. Overall, this multi-task learning and iterative information feedback benefits each other and constitutes a network that is end-to-end trainable. Our extensive evaluation on real images, particularly on heavy rain, shows the effectiveness of our novel models and architecture, outperforming the state-of-the-art methods significantly. In this paper, we address the problem of video rain removal by constructing deep recurrent convolutional networks. We visit the rain removal case by considering rain occlusion regions, i.e. the light transmittance of rain streaks is low. Different from additive rain streaks, in such rain occlusion regions, the details of background images are completely lost. Therefore, we propose a hybrid rain model to depict both rain streaks and occlusions. With the wealth of temporal redundancy, we build a Joint Recurrent Rain Removal and Reconstruction Network (J4R-Net) that seamlessly integrates rain degradation classification, spatial texture appearances based rain removal and temporal coherence based background details reconstruction. The rain degradation classification provides a binary map that reveals whether a location is degraded by linear additive streaks or occlusions. With this side information, the gate of the recurrent unit learns to make a trade-off between rain streak removal and background details reconstruction. Extensive experiments on a series of synthetic and real videos with rain streaks verify the superiority of the proposed method over previous state-of-the-art methods. Abstract Short-term high-resolution precipitation forecasting has important implications for navigation, flood forecasting, and other hydrological and meteorological concerns. This article introduces a pixel-based algorithm for Short-term Quantitative Precipitation Forecasting (SQPF) using radar-based rainfall data. The proposed algorithm called Pixel- Based Nowcasting (PBN) tracks severe storms with a hierarchical mesh-tracking algorithm to capture storm advection in space and time at high resolution from radar imagers. The extracted advection field is then extended to nowcast the rainfall field in the next 3 hr based on a pixel-based Lagrangian dynamic model. The proposed algorithm is compared with two other nowcasting algorithms (WCN: Watershed-Clustering Nowcasting and PER: PERsistency) for ten thunderstorm events over the conterminous United States. Object-based verification metric and traditional statistics have been used to evaluate the performance of the proposed algorithm. It is shown that the proposed algorithm is superior over comparison algorithms and is effective in tracking and predicting severe storm events for the next few hours. We consider real-time rendering of scenes in participating media, capturing the effects of light scattering in fog, mist and haze. While a number of sophisticated approaches based on Monte Carlo and finite element simulation have been developed, those methods do not work at interactive rates. The most common real-time methods are essentially simple variants of the OpenGL fog model. While easy to use and specify, that model excludes many important qualitative effects like glows around light sources, the impact of volumetric scattering on the appearance of surfaces such as the diffusing of glossy highlights, and the appearance under complex lighting such as environment maps. In this paper, we present an alternative physically based approach that captures these effects while maintaining real time performance and the ease-of-use of the OpenGL fog model. Our method is based on an explicit analytic integration of the single scattering light transport equations for an isotropic point light source in a homogeneous participating medium. We can implement the model in modern programmable graphics hardware using a few small numerical lookup tables stored as texture maps. Our model can also be easily adapted to generate the appearances of materials with arbitrary BRDFs, environment map lighting, and precomputed radiance transfer methods, in the presence of participating media. Hence, our techniques can be widely used in real-time rendering. Raindrops adhered to a glass window or camera lens can severely hamper the visibility of a background scene and degrade an image considerably. In this paper, we address the problem by visually removing raindrops, and thus transforming a raindrop degraded image into a clean one. The problem is intractable, since first the regions occluded by raindrops are not given. Second, the information about the background scene of the occluded regions is completely lost for most part. To resolve the problem, we apply an attentive generative network using adversarial training. Our main idea is to inject visual attention into both the generative and discriminative networks. During the training, our visual attention learns about raindrop regions and their surroundings. Hence, by injecting this information, the generative network will pay more attention to the raindrop regions and the surrounding structures, and the discriminative network will be able to assess the local consistency of the restored regions. This injection of visual attention to both generative and discriminative networks is the main contribution of this paper. Our experiments show the effectiveness of our approach, which outperforms the state of the art methods quantitatively and qualitatively. Raindrops adhered to a glass window or camera lens can severely hamper the visibility of a background scene and degrade an image considerably. In this paper, we address the problem by visually removing raindrops, and thus transforming a raindrop degraded image into a clean one. The problem is intractable, since first the regions occluded by raindrops are not given. Second, the information about the background scene of the occluded regions is completely lost for most part. To resolve the problem, we apply an attentive generative network using adversarial training. Our main idea is to inject visual attention into both the generative and discriminative networks. During the training, our visual attention learns about raindrop regions and their surroundings. Hence, by injecting this information, the generative network will pay more attention to the raindrop regions and the surrounding structures, and the discriminative network will be able to assess the local consistency of the restored regions. This injection of visual attention to both generative and discriminative networks is the main contribution of this paper. Our experiments show the effectiveness of our approach, which outperforms the state of the art methods quantitatively and qualitatively. Dynamic weather such as rain and snow causes complex spatio-temporal intensity fluctuations in videos. Such fluctuations can adversely impact vision systems that rely on small image features for tracking, object detection and recognition. While these effects appear to be chaotic in space and time, we show that dynamic weather has a predictable global effect in frequency space. For this, we first develop a model of the shape and appearance of a single rain or snow streak in image space. Detecting individual streaks is difficult even with an accurate appearance model, so we combine the streak model with the statistical characteristics of rain and snow to create a model of the overall effect of dynamic weather in frequency space. Our model is then fit to a video and is used to detect rain or snow streaks first in frequency space, and the detection result is then transferred to image space. Once detected, the amount of rain or snow can be reduced or increased. We demonstrate that our frequency analysis allows for greater accuracy in the removal of dynamic weather and in the performance of feature extraction than previous pixel-based or patch-based methods. We also show that unlike previous techniques, our approach is effective for videos with both scene and camera motions. We present a LIDAR simulation framework that can automatically generate 3D point cloud based on LIDAR type and placement. The point cloud, annotated with ground truth semantic labels, is to be used as training data to improve environmental perception capabilities for autonomous driving vehicles. Different from previous simulators, we generate the point cloud based on real environment and real traffic flow. More specifically we employ a mobile LIDAR scanner with cameras to capture real world scenes. The input to our simulation framework includes dense 3D point cloud and registered color images. Moving objects (such as cars, pedestrians, bicyclists) are automatically identified and recorded. These objects are then removed from the input point cloud to restore a static background (e.g., environment without movable objects). With that we can insert synthetic models of various obstacles, such as vehicles and pedestrians in the static background to create various traffic scenes. A novel LIDAR renderer takes the composite scene to generate new realistic LIDAR points that are already annotated at point level for synthetic objects. Experimental results show that our system is able to close the performance gap between simulation and real data to be 1 6 in different applications, and for model fine tuning, only 10 20 extra real data could help to outperform the original model trained with full real dataset. Heavy precipitation for several days and weeks always leads to some extreme nature disasters. Long-lead term precipitation forecasting plays an important role on the prevision of such calamities. Most works focus on the generation of training labels with allocation of the proper corresponding spatio-temporal information. In this paper, we will provide a different path by performing regression analysis using the precipitation amounts at particular locations. This method is called Hierarchical Clustering based Bayesian Structural Vector Autoregression (HC-BSVAR). The approach for HC-BSVAR is divided into two steps. First, we apply a hierarchical clustering algorithm to identify the Elite locations and then transfer the 3-dimensional data space into a new traditional 2-dimensional data space. Every column of the new data frame is a hydro-meteorological feature of the original data and each row represents a time point (day) in the original space. Secondly, an economic-based multivariate time series model called Bayesian-based Structural Vector Autoregression (BSVAR) is exploited to perform the final prediction result. The prediction quality will be vary for different cut of tree structure which generated by hierarchical clustering. The coefficient for determination of each location by each level of cut is applied to quantize the quality of prediction. The relationship between the cut level of clustering geographic locations and the regression model performance are also discussed, based on the result of prediction quality. Long-lead prediction of heavy precipitation events has a significant impact since it can provide an early warning of disasters, like a flood. However, the performance of existed prediction models has been constrained by the high dimensional space and non-linear relationship among variables. In this study, we study the prediction problem from the prospective of machine learning. In our machine-learning framework for forecasting heavy precipitation events, we use global hydro-meteorological variables with spatial and temporal influences as features, and the target weather events that last several days have been formulated as weather clusters. Our study has three phases: 1) identify weather clusters in different sizes, 2) handle the imbalance problem within the data, 3) select the most-relevant features through the large feature space. We plan to evaluate our methods with several real world data sets for predicting the heavy precipitation events. We propose a weather recognition method from in-vehicle camera images that uses a subspace method to judge rainy weather by detecting raindrops on the windshield. "Eigendrops" represent the principal components extracted from raindrop images in the learning stage. Then the method detects raindrops by template matching. In experiments using actual video sequences, our method showed good detection ability of raindrops and promising results for rainfall judgment from detection results. A type of extreme disastrous floods are associated with a sequence of prior heavy precipitation events occurring frequently from over several days to several weeks. Transitional methods for precipitation clusters prediction usually rely on the measurement and analyses of meteorological variables. However while a short-term prediction of certain location depends only on variables in near spatial and temporal neighborhood, predictions with long lead time must consider variables in a long time window and large spatial neighborhoods, this means an enormous amount of potentially influencing variables and only a subset of them strongly relate to prediction. Processing a deluge of variables and discovering strongly relevant features pose a significant challenge for big data analytics. Finding influencing variables calls for automated methods of strongly relevant feature selection, which is what online streaming feature selection provides. In particular, online streaming feature selection, which deals with the stream of features sequentially added while the total data observations are fixed, aims to select a subset of strongly relevant features from the original feature set. In this paper, we apply four state-of-the-art online streaming feature selection methods for building long-lead extreme floods forecasting models, which identify optimal size of strongly relevant meteorological variables and confine learning the prediction model on the relevant feature set instead of the original entire feature set. The prediction models are evaluated and compared systematically on the historical precipitation and associated meteorological data collected in the State of Iowa. Images of outdoor scenes captured in bad weather suffer from poor contrast. Under bad weather conditions, the light reaching a camera is severely scattered by the atmosphere. The resulting decay in contrast varies across the scene and is exponential in the depths of scene points. Therefore, traditional space invariant image processing techniques are not sufficient to remove weather effects from images. We present a physics-based model that describes the appearances of scenes in uniform bad weather conditions. Changes in intensities of scene points under different weather conditions provide simple constraints to detect depth discontinuities in the scene and also to compute scene structure. Then, a fast algorithm to restore scene contrast is presented. In contrast to previous techniques, our weather removal algorithm does not require any a priori scene structure, distributions of scene reflectances, or detailed knowledge about the particular weather condition. All the methods described in this paper are effective under a wide range of weather conditions including haze, mist, fog, and conditions arising due to other aerosols. Further, our methods can be applied to gray scale, RGB color, multispectral and even IR images. We also extend our techniques to restore contrast of scenes with moving objects, captured using a video camera. This paper introduces a novel visualization approach that can effectively facilitate the analysis, control, and refinement of dynamic environmental simulations on the World Wide Web. This approach overcomes drawbacks of current Internet Geographic Information System (GIS) technologies by providing an effective and efficient mechanism for two-way and sustained communication and synchronization between the visualization and the modeling processes. The critical aspect of this approach is the establishment of a virtual environment on the Internet using the applet-servlet-socket architecture that supports real-time interactive and collaborative visualization of an environmental modeling process. In this virtual environment, the model residing in the simulation application server is fed with real-time rainfall data from a remote data server through a socket connection. The computational modeling and visualization can take place simultaneously on the application server and the client sides. As modeling computations proceed on the server, modeling results and outputs stream into the client side continuously. The client interface is updated with live 3D displays (not in the sense of predefined 3D animations provided by AVI or dynamic GIF files). Meanwhile, these 3D graphics can act as a support for further user interactions. A hydrological model, TOPMODEL, is implemented using the proposed web-based visualization environment to demonstrate the applicability of the proposed approach in facilitating environmental modeling and simulation.