In this paper, we attempt to employ convolutional recurrent neural networks for weather temperature estimation using only image data. We study ambient temperature estimation based on deep neural networks in two scenarios a) estimating temperature of a single outdoor image, and b) predicting temperature of the last image in an image sequence. In the first scenario, visual features are extracted by a convolutional neural network trained on a large-scale image dataset. We demonstrate that promising performance can be obtained, and analyze how volume of training data influences performance. In the second scenario, we consider the temporal evolution of visual appearance, and construct a recurrent neural network to predict the temperature of the last image in a given image sequence. We obtain better prediction accuracy compared to the state-of-the-art models. Further, we investigate how performance varies when information is extracted from different scene regions, and when images are captured in different daytime hours. Our approach further reinforces the idea of using only visual information for cost efficient weather prediction in the future.
In addition to weather condition classification, more weather properties have also been investigated. Jacobs and his colleagues @cite_11 initiated a project for collecting outdoor scene images captured by static webcams over a long period of time. The collected images form the Archive of Many Outdoor Scene (AMOS) dataset @cite_11 . Based on the AMOS dataset, they proposed that webcams installed across the earth can be viewed as image sensors and can enable us to understand weather patterns and variations over time @cite_0 . More specifically, they adopted principal component analysis and canonical correlation analysis to predict wind velocity and vapor pressure from a sequence of images.
@cite_0 @cite_11
2147016261 2160110230
We present a photometric stereo technique that operates on time-lapse sequences captured by static outdoor webcams over the course of several months. Outdoor webcams produce a large set of uncontrolled images subject to varying lighting and weather conditions. We first automatically select a suitable subset of the captured frames for further processing, reducing the dataset size by several orders of magnitude. A camera calibration step is applied to recover the camera response function, the absolute camera orientation, and to compute the light directions for each image. Finally, we describe a new photometric stereo technique for non-Lambertian scenes and unknown light source intensities to recover normal maps and spatially varying materials of the scene. Ten mood variables were related to eight weather variables in a multidimensional study. Data were collected from 24 male subjects over 11 consecutive days. The mood variables were as follows: concentration, cooperation, anxiety, potency, aggression, depression, sleepiness, scepticism, control, and optimism. The weather variables included: hours of sunshine, precipitation, temperature, wind direction, wind velocity, humidity, change in barometric pressure and absolute barometric pressure. Humidity, temperature and hours of sunshine had the greatest effect on mood. High levels of humidity lowered scores on concentration while increasing reports of sleepiness. Rising temperatures lowered anxiety and scepticism mood scores. Humidity was the most significant predictor in regression and canonical correlation analysis. Implications for school and office performance are discussed which highlight the importance of humidity as a weather variable. Images of outdoor scenes captured in bad weather suffer from poor contrast. Under bad weather conditions, the light reaching a camera is severely scattered by the atmosphere. The resulting decay in contrast varies across the scene and is exponential in the depths of scene points. Therefore, traditional space invariant image processing techniques are not sufficient to remove weather effects from images. We present a physics-based model that describes the appearances of scenes in uniform bad weather conditions. Changes in intensities of scene points under different weather conditions provide simple constraints to detect depth discontinuities in the scene and also to compute scene structure. Then, a fast algorithm to restore scene contrast is presented. In contrast to previous techniques, our weather removal algorithm does not require any a priori scene structure, distributions of scene reflectances, or detailed knowledge about the particular weather condition. All the methods described in this paper are effective under a wide range of weather conditions including haze, mist, fog, and conditions arising due to other aerosols. Further, our methods can be applied to gray scale, RGB color, multispectral and even IR images. We also extend our techniques to restore contrast of scenes with moving objects, captured using a video camera. The discovery of patterns and events in the physical world by analysis of multiple streams of sensor data can provide benefit to society in more than just surveillance applications by focusing on automated means for social scientists, anthropologists and marketing experts to detect macroscopic trends and changes in the general population. This goal complements analogous efforts in documenting trends in the digital world, such as those in social media monitoring. In this paper we show how the contents of a street webcam, processed with state-of-the-art deep networks, can provide information about patterns in clothing and their relation to weather information. In particular, we analyze a large time series of street webcam images, using a deep network trained for garment detection, and demonstrate how the garment distribution over time significantly correlates to weather and temporal patterns. Finally, we additionally provide a new and improved labelled dataset of garments for training and benchmarking purposes, reporting (58.19 , ) overall accuracy on the ACS test set, the best performance yet obtained. Current vision systems are designed to perform in clear weather. Needless to say, in any outdoor application, there is no escape from "bad" weather. Ultimately, computer vision systems must include mechanisms that enable them to function (even if somewhat less reliably) in the presence of haze, fog, rain, hail and snow. We begin by studying the visual manifestations of different weather conditions. For this, we draw on what is already known about atmospheric optics. Next, we identify effects caused by bad weather that can be turned to our advantage. Since the atmosphere modulates the information carried from a scene point to the observer it can be viewed as a mechanism of visual information coding. Based on this observation, we develop models and methods for recovering pertinent scene properties, such as three-dimensional structure, from images taken under poor weather conditions. Abstract We estimate weather information from single images, as an important clue to unveil real-world characteristics available in the cyberspace, and as a complementary feature to facilitate computer vision applications. Based on an image collection with geotags, we crawl the associated weather and elevation properties from the web. With this large-scale and rich image dataset, various correlations between weather properties and metadata are observed, and are used to construct computational models based on random forests to estimate weather information for any given image. We describe interesting statistics linking weather properties with human behaviors, and show that image’s weather information can potentially benefit computer vision tasks such as landmark classification. Overall, this work proposes a large-scale image dataset with rich weather properties, and provides comprehensive studies on using cameras as weather sensors. This work addresses the problem of semantic foggy scene understanding (SFSU). Although extensive research has been performed on image dehazing and on semantic scene understanding with clear-weather images, little attention has been paid to SFSU. Due to the difficulty of collecting and annotating foggy images, we choose to generate synthetic fog on real images that depict clear-weather outdoor scenes, and then leverage these partially synthetic data for SFSU by employing state-of-the-art convolutional neural networks (CNN). In particular, a complete pipeline to add synthetic fog to real, clear-weather images using incomplete depth information is developed. We apply our fog synthesis on the Cityscapes dataset and generate Foggy Cityscapes with 20,550 images. SFSU is tackled in two ways: (1) with typical supervised learning, and (2) with a novel type of semi-supervised learning, which combines (1) with an unsupervised supervision transfer from clear-weather images to their synthetic foggy counterparts. In addition, we carefully study the usefulness of image dehazing for SFSU. For evaluation, we present Foggy Driving, a dataset with 101 real-world images depicting foggy driving scenes, which come with ground truth annotations for semantic segmentation and object detection. Extensive experiments show that (1) supervised learning with our synthetic data significantly improves the performance of state-of-the-art CNN for SFSU on Foggy Driving; (2) our semi-supervised learning strategy further improves performance; and (3) image dehazing marginally advances SFSU with our learning strategy. The datasets, models and code are made publicly available. Recent descriptions of algorithms applied to images archived from webcams tend to underplay the challenges in working with large data sets acquired from uncontrolled webcams in real environments. In building a database of images captured from 1000 webcams, every 30 minutes for the last 3 years, we observe that these cameras have a wide variety of failure modes. This paper details steps we have taken to make this dataset more easily useful to the research community, including (a) tools for finding stable temporal segments, and stabilizing images when the camera is nearly stable, (b) visualization tools to quickly summarize a years worth of image data from one camera and to give a set of exemplars that highlight anomalies within the scene, and (c) integration with LabelMe, allowing labels of static features in one image of a scene to propagate to the thousands of other images of that scene. We also present proof-of-concept algorithms showing how this data conditioning supports several problems in inferring properties of the scene from image data. A key problem in widely distributed camera networks is locating the cameras. This paper considers three scenarios for camera localization: localizing a camera in an unknown environment, adding a new camera in a region with many other cameras, and localizing a camera by finding correlations with satellite imagery. We find that simple summary statistics (the time course of principal component coefficients) are sufficient to geolocate cameras without determining correspondences between cameras or explicitly reasoning about weather in the scene. We present results from a database of images from 538 cameras collected over the course of a year. We find that for cameras that remain stationary and for which we have accurate image times- tamps, we can localize most cameras to within 50 miles of the known location. In addition, we demonstrate the use of a distributed camera network in the construction a map of weather conditions. Material recognition for real-world outdoor surfaces has become increasingly important for computer vision to support its operation in the wild. Computational surface modeling that underlies material recognition has transitioned from reflectance modeling using in-lab controlled radiometric measurements to image-based representations based on internet-mined images of materials captured in the scene. We propose to take a middle-ground approach for material recognition that takes advantage of both rich radiometric cues and flexible image capture. We realize this by developing a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement. We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS) database, geared towards real use for autonomous agents. The database consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying weather and lighting conditions. We develop a novel approach for material recognition called a Differential Angular Imaging Network (DAIN) to fully leverage this large dataset. With this novel network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that DAIN achieves recognition performance that surpasses single view or coarsely quantized multiview images. These results demonstrate the effectiveness of differential angular imaging as a means for flexible, in-place material recognition. Material recognition for real-world outdoor surfaces has become increasingly important for computer vision to support its operation "in the wild." Computational surface modeling that underlies material recognition has transitioned from reflectance modeling using in-lab controlled radiometric measurements to image-based representations based on internet-mined images of materials captured in the scene. We propose to take a middle-ground approach for material recognition that takes advantage of both rich radiometric cues and flexible image capture. We realize this by developing a framework for differential angular imaging, where small angular variations in image capture provide an enhanced appearance representation and significant recognition improvement. We build a large-scale material database, Ground Terrain in Outdoor Scenes (GTOS) database, geared towards real use for autonomous agents. The database consists of over 30,000 images covering 40 classes of outdoor ground terrain under varying weather and lighting conditions. We develop a novel approach for material recognition called a Differential Angular Imaging Network (DAIN) to fully leverage this large dataset. With this novel network architecture, we extract characteristics of materials encoded in the angular and spatial gradients of their appearance. Our results show that DAIN achieves recognition performance that surpasses single view or coarsely quantized multiview images. These results demonstrate the effectiveness of differential angular imaging as a means for flexible, in-place material recognition. Existing methods for video scene analysis are primarily concerned with learning motion patterns or models for anomaly detection. We present a novel form of video scene analysis where scene element categories such as roads, parking areas, sidewalks and entrances, can be segmented and categorized based on the behaviors of moving objects in and around them. We view the problem from the perspective of categorical object recognition, and present an approach for unsupervised learning of functional scene element categories. Our approach identifies functional regions with similar behaviors in the same scene and or across scenes, by clustering histograms based on a trajectory-level, behavioral codebook. Experiments are conducted on two outdoor webcam video scenes with low frame rates and poor quality. Unsupervised classification results are presented for each scene independently, and also jointly where models learned on one scene are applied to the other. The way people look in terms of facial attributes (ethnicity, hair color, facial hair, etc.) and the clothes or accessories they wear (sunglasses, hat, hoodies, etc.) is highly dependent on geo-location and weather condition, respectively. This work explores, for the first time, the use of this contextual information, as people with wearable cameras walk across different neighborhoods of a city, in order to learn a rich feature representation for facial attribute classification, without the costly manual annotation required by previous methods. By tracking the faces of casual walkers on more than 40 hours of egocentric video, we are able to cover tens of thousands of different identities and automatically extract nearly 5 million pairs of images connected by or from different face tracks, along with their weather and location context, under pose and lighting variations. These image pairs are then fed into a deep network that preserves similarity of images connected by the same track, in order to capture identity-related attribute features, and optimizes for location and weather prediction to capture additional facial attribute features. Finally, the network is fine-tuned with manually annotated samples. We perform an extensive experimental analysis on wearable data and two standard benchmark datasets based on web images (LFWA and CelebA). Our method outperforms by a large margin a network trained from scratch. Moreover, even without using manually annotated identity labels for pre-training as in previous methods, our approach achieves results that are better than the state of the art. This paper is about life-long vast-scale localisation in spite of changes in weather, lighting and scene structure. Building upon our previous work in Experience-based Navigation [1], we continually grow and curate a visual map of the world that explicitly supports multiple representations of the same place. We refer to these representations as experiences, where a single experience captures the appearance of an environment under certain conditions. Pedagogically, an experience can be thought of as a visual memory. By accumulating experiences we are able to handle cyclic appearance change (diurnal lighting, seasonal changes, and extreme weather conditions) and also adapt to slow structural change. This strategy, although elegant and effective, poses a new challenge: In a region with many stored representations - which one(s) should we try to localise against given finite computational resources? By learning from our previous use of the experience-map, we can make predictions about which memories we should consider next, conditioned on how the robot is currently localised in the experience-map. During localisation, we prioritise the loading of past experiences in order to minimise the expected computation required. We do this in a probabilistic way and show that this memory policy significantly improves localisation efficiency, enabling long-term autonomy on robots with limited computational resources. We demonstrate and evaluate our system over three challenging datasets, totalling 206km of outdoor travel. We demonstrate the system in a diverse range of lighting and weather conditions, scene clutter, camera occlusions, and permanent structural change in the environment. We propose the use of deep convolutional neural networks to estimate the transient attributes of a scene from a single image. Transient scene attributes describe both the objective conditions, such as the weather, time of day, and the season, and subjective properties of a scene, such as whether or not the scene seems busy. Recently, convolutional neural networks have been used to achieve state-of-the-art results for many vision problems, from object detection to scene classification, but have not previously been used for estimating transient attributes. We compare several methods for adapting an existing network architecture and present state-of-the-art results on two benchmark datasets. Our method is more accurate and significantly faster than previous methods, enabling real-world applications. Weather recognition is important in practice, while this task has not been thoroughly explored so far. The current trend of dealing with this task is treating it as a single classification problem, i.e., determining whether a given image belongs to a certain weather category or not. However, weather recognition differs significantly from traditional image classification, since several weather features may appear simultaneously. In this case, a simple classification result is insufficient to describe the weather condition. To address this issue, we propose to provide auxiliary weather related information for comprehensive weather description. Specifically, semantic segmentation of weather-cues, such as blue sky and white clouds, is exploited as an auxiliary task in this paper. Moreover, a convolutional neural network (CNN) based multi-task framework is developed which aims to concurrently tackle weather category classification task and weather-cues segmentation task. Due to the intrinsic relationships between these two tasks, exploring auxiliary semantic segmentation of weather-cues can also help to learn discriminative features for the classification task, and thus obtain superior accuracy. To verify the effectiveness of the proposed approach, extra segmentation masks of weather-cues are generated manually on an existing weather image dataset. Experimental results have demonstrated the superior performance of our approach. The enhanced dataset, source codes and pre-trained models are available at https: github.com wzgwzg Multitask_Weather. Correlations were calculated between daily self-evaluated mood reports (4 reports day) of 10 student subjects and 10 meteorological-geophysical variables over a 90-day period. The variables included mean and or change measures of temperature, barometric pressure, relative humidity, sunshine hours, wind speeds and global geomagnetic activity. Multiple regression analyses indicated that the weather matrix used could account for not more than 35 of the mood variance on the day of the evaluation. Lag correlations over the previous week clearly indicated a greater number of significant correlations between mood reports and weather of the previous two days. The mean correlation coefficient was 0.27. In general, "lower moods" were associated with fewer sunshine hours, higher relative humidity, and smaller humidity variations than "higher moods". It was concluded that mood reports can show weak responsivity to antecedent weather fluctuations. Weather recognition is widely required in many areas, and it is also a challenging and brand-new subject. This paper proposes an approach to recognize weather based on images captured by in-vehicle vision system. We bring three groups of features, including histogram of gradient amplitude, HSV color histogram, road information, and employ an algorithm based on Real AdaBoost, making use of the category structure to achieve the task of classification. Experiments confirm superior performances on our dataset collected from images captured by vision system. Visual localization systems that are practical for autonomous vehicles in outdoor industrial applications must perform reliably in a wide range of conditions. Changing outdoor conditions cause difficulty by drastically altering the information available in the camera images. To confront the problem, we have developed a visual localization system that uses a surveyed three-dimensional (3D)-edge map of permanent structures in the environment. The map has the invariant properties necessary to achieve long-term robust operation. Previous 3D-edge map localization systems usually maintain a single pose hypothesis, making it difficult to initialize without an accurate prior pose estimate and also making them susceptible to misalignment with unmapped edges detected in the camera image. A multihypothesis particle filter is employed here to perform the initialization procedure with significant uncertainty in the vehicle's initial pose. A novel observation function for the particle filter is developed and evaluated against two existing functions. The new function is shown to further improve the abilities of the particle filter to converge given a very coarse estimate of the vehicle's initial pose. An intelligent exposure control algorithm is also developed that improves the quality of the pertinent information in the image. Results gathered over an entire sunny day and also during rainy weather illustrate that the localization system can operate in a wide range of outdoor conditions. The conclusion is that an invariant map, a robust multihypothesis localization algorithm, and an intelligent exposure control algorithm all combine to enable reliable visual localization through challenging outdoor conditions. © 2009 Wiley Periodicals, Inc. This paper details an empirical study of large image sets taken by static cameras. These images have consistent correlations over the entire image and over time scales of days to months. Simple second-order statistics of such image sets show vastly more structure than exists in generic natural images or video from moving cameras. Using a slight variant to PCA, we can decompose all cameras into comparable components and annotate images with respect to surface orientation, weather, and seasonal change. Experiments are based on a data set from 538 cameras across the United States which have collected more than 17 million images over the the last 6 months.