Sky cloud images captured by ground-based cameras (a.k.a. whole sky imagers) are increasingly used nowadays because of their applications in a number of fields, including climate modeling, weather prediction, renewable energy generation, and satellite communications. Due to the wide variety of cloud types and lighting conditions in such images, accurate and robust segmentation of clouds is challenging. In this paper, we present a supervised segmentation framework for ground-based sky cloud images based on a systematic analysis of different color spaces and components, using partial least squares (PLS) regression. Unlike other state-of-the-art methods, our proposed approach is entirely learning-based and does not require any manually-defined parameters. In addition, we release the Singapore Whole Sky IMaging SEGmentation Database (SWIMSEG), a large database of annotated sky cloud images, to the research community.
As color is the most discriminating feature in sky cloud images, most works in the literature use color for cloud segmentation. @cite_4 showed that the ratio of red and blue channels from RGB color space is a good candidate for segmentation and tuned corresponding thresholds to create binary masks. @cite_18 exploited the difference of red and blue channels for successful detection and subsequent labeling of pixels. @cite_24 also used the difference of red and blue channels in their superpixel-based cloud segmentation framework. @cite_14 used the Saturation (S) channel for calculating cloud coverage. Mantelli- @cite_22 investigated the locus of cloud pixels in the RGB color model. @cite_21 proposed cloud detection using an adaptive threshold technique in the normalized blue red channel. @cite_25 proposed a cloud detection framework using superpixel classification of image features.
@cite_18 @cite_14 @cite_4 @cite_22 @cite_21 @cite_24 @cite_25
2031640526 1603854802 2416791088 1565402342
AbstractCloud detection is the precondition for deriving other information (e.g., cloud cover) in ground-based sky imager applications. This paper puts forward an effective cloud detection approach, the Hybrid Thresholding Algorithm (HYTA) that fully exploits the benefits of the combination of fixed and adaptive thresholding methods. First, HYTA transforms an input color cloud image into a normalized blue red channel ratio image that can keep a distinct contrast, even with noise and outliers. Then, HYTA identifies the ratio image as either unimodal or bimodal according to its standard deviation, and the unimodal and bimodal images are handled by fixed and minimum cross entropy (MCE) thresholding algorithms, respectively. The experimental results demonstrate that HYTA shows an accuracy of 88.53 , which is far higher than those of either fixed or MCE thresholding alone. Moreover, HYTA is also verified to outperform other state-of-the-art cloud detection approaches. Automatic cloud detection from satellite imagery is a necessary preprocessing step in remote sensing. Given that humans can easily “see” clouds in an image because of salient region features, we adopt a visual attention technique in computer vision to automatically identify images with a significant cloud cover. The proposed method generates a rough cloud mask by using a top-down visual saliency model to qualitatively distinguish cloud images from noncloud images. First, an image is downsized for rapid processing. Some basic saliency maps of clouds are then generated by multilevel segmentation, the computation of cloud visual saliency features, and feature classification. Thereafter, we fuse the basic saliency maps by using a most-votes-win strategy to generate the cloud mask. With the cloud mask, a threshold is used to classify the images as cloud or noncloud images. A total of 200 RapidEye images are tested by using the algorithm. Of the cloud images, 92 are correctly identified. The average processing time is 1.8 s per image. In this paper, we explore various aspects of fusing LIDAR and color imagery for pedestrian detection in the context of convolutional neural networks (CNNs), which have recently become state-of-art for many vision problems. We incorporate LIDAR by up-sampling the point cloud to a dense depth map and then extracting three features representing different aspects of the 3D scene. We then use those features as extra image channels. Specifically, we leverage recent work on HHA [9] (horizontal disparity, height above ground, and angle) representations, adapting the code to work on up-sampled LIDAR rather than Microsoft Kinect depth maps. We show, for the first time, that such a representation is applicable to up-sampled LIDAR data, despite its sparsity. Since CNNs learn a deep hierarchy of feature representations, we then explore the question: At what level of representation should we fuse this additional information with the original RGB image channels? We use the KITTI pedestrian detection dataset for our exploration. We first replicate the finding that region-CNNs (R-CNNs) [8] can outperform the original proposal mechanism using only RGB images, but only if fine-tuning is employed. Then, we show that: 1) using HHA features and RGB images performs better than RGB-only, even without any fine-tuning using large RGB web data, 2) fusing RGB and HHA achieves the strongest results if done late, but, under a parameter or computational budget, is best done at the early to middle layers of the hierarchical representation, which tend to represent midlevel features rather than low (e.g. edges) or high (e.g. object class decision) level features, 3) some of the less successful methods have the most parameters, indicating that increased classification accuracy is not simply a function of increased capacity in the neural network. In this paper we study the problem of object detection for RGB-D images using semantically rich image and depth features. We propose a new geocentric embedding for depth images that encodes height above ground and angle with gravity for each pixel in addition to the horizontal disparity. We demonstrate that this geocentric embedding works better than using raw depth images for learning feature representations with convolutional neural networks. Our final object detection system achieves an average precision of 37.3 , which is a 56 relative improvement over existing methods. We then focus on the task of instance segmentation where we label pixels belonging to object instances found by our detector. For this task, we propose a decision forest approach that classifies pixels in the detection window as foreground or background using a family of unary and binary tests that query shape and geocentric pose features. Finally, we use the output from our object detectors in an existing superpixel classification framework for semantic scene segmentation and achieve a 24 relative improvement over current state-of-the-art for the object categories that we study. We believe advances such as those represented in this paper will facilitate the use of perception in fields like robotics.