This paper addresses the problem of floods classification and floods aftermath detection utilizing both social media and satellite imagery. Automatic detection of disasters such as floods is still a very challenging task. The focus lies on identifying passable routes or roads during floods. Two novel solutions are presented, which were developed for two corresponding tasks at the MediaEval 2018 benchmarking challenge. The tasks are (i) identification of images providing evidence for road passability and (ii) differentiation and detection of passable and non-passable roads in images from two complementary sources of information. For the first challenge, we mainly rely on object and scene-level features extracted through multiple deep models pre-trained on the ImageNet and Places datasets. The object and scene-level features are then combined using early, late and double fusion techniques. To identify whether or not it is possible for a vehicle to pass a road in satellite images, we rely on Convolutional Neural Networks and a transfer learning-based classification approach. The evaluation of the proposed methods are carried out on the large-scale datasets provided for the benchmark competition. The results demonstrate significant improvement in the performance over the recent state-of-art approaches.
@cite_21 rely on hand-crafted visual features, such as the colour and edge directivity descriptor (CEDD) @cite_6 , color layout @cite_48 and Gabor wavelets @cite_33 . A more sophisticated solution has been proposed for textual information (i.e., description, title and users' tags) relying on word embeddings trained on the entire YFCC100m dataset @cite_49 . Each textual feature is extracted separately, and then concatenated to form a single feature vector. Moreover, to translate users' tags into English, a machine translation technique has been employed. In @cite_55 , handcrafted visual features are concatenated into a single feature vector followed by dimensionality reduction and classification phases. Term Frequency Inverse Document Frequency (TFIDF) @cite_35 are measured for users' tags to represent textual features. In @cite_4 , handcrafted visual features along with textual information are used for the classification of flood related images.
@cite_35 @cite_4 @cite_33 @cite_48 @cite_55 @cite_21 @cite_6 @cite_49
1978394996 2758405190 2154461202 1972195523 2773853695 2750871820 2163502583 2250384498
The experimental evidence accumulated over the past 20 years indicates that textindexing systems based on the assignment of appropriately weighted single terms produce retrieval results that are superior to those obtainable with other more elaborate text representations. These results depend crucially on the choice of effective term weighting systems. This paper summarizes the insights gained in automatic term weighting, and provides baseline single term indexing models with which other more elaborate content analysis procedures can be compared. We introduce a domain-speci c and late-fusion algorithm to cope with the challenge raised in The MediaEval 2017 Multimedia Satel- lite Task. Several known techniques are integrated based on domain- speci c criteria such as late fusion, tuning, ensemble learning, object detection using deep learning, and temporal-spatial-based event con rmation. Experimental results show that the proposed algo- rithm can overcome the main challenges of the proper discrimina- tion of the water levels in di erent areas as well as the consideration of di erent types of ooding events. Recognizing smiles is of much importance for detecting happy moods. Gabor features are conventionally widely applied to facial expression recognition, but the number of Gabor features is usually too large. We proposed to use Pyramid Histogram of Oriented Gradients (PHOG) as the features extracted for smile recognition in this paper. The comparisons between the PHOG and Gabor features using a publicly available dataset demonstrated that the PHOG with a significantly shorter vector length could achieve as high a recognition rate as the Gabor features did. Furthermore, the feature selection conducted by an AdaBoost algorithm was not needed when using the PHOG features. To further improve the recognition performance, we combined these two feature extraction methods and achieved the best smile recognition rate, indicating a good value of the PHOG features for smile recognitions. This paper proposes a color feature description for image and video retrieving applications such as personal video recorder, which manages a large amount of data. This descriptor specifies the spatial distribution of colors with a few nonlinear quantized DCT coefficients of grid based average colors. The tradeoff between the number of the coefficients enclosed in the descriptor and retrieval cost is studied. The experimental results show that the descriptor enclosing six for luminance and three for each chrominance coefficient achieves the best trade-off between the storage cost and retrieval efficiency. It requires 6 bits for DC and 5 bits for AC coefficients, and therefore total storage cost is just 63 bits per image. This description, named color layout descriptor, has been accepted as a part of the MPEG-7 final committee draft. This working note describes the work of the WISC team on the Multimedia Satellite Task at MediaEval 2017. We describe the runs that our team submitted to both the DIRSM and FDSI subtasks, as well as our evaluations on the development set. Our results demonstrate high accuracy in the detection of flooded areas from user-generated content in social media. In the first subtask consisting of disaster image retrieval from social media, we found that tags defined by users to describe the images are very helpful for achieving high accuracy classification. In the second subtask consisting of detecting flood in satellite images, we found that social media can increase the precision in analyses when combined with satellite images by taking advantage of spatial and temporal overlaps between data sources. This paper deals with a new low level feature that is extracted from the images and can be used for indexing and retrieval. This feature is called "Color and Edge Directivity Descriptor" and incorporates color and texture information in a histogram. CEDD size is limited to 54 bytes per image, rendering this descriptor suitable for use in large image databases. One of the most important attribute of the CEDD is the low computational power needed for its extraction, in comparison with the needs of the most MPEG-7 descriptors. The objective measure called ANMRR is used to evaluate the performance of the proposed feature. An online demo that implements the proposed feature in an image retrieval system is available at: http: orpheus.ee.duth.gr image_retrieval. This publicly available curated dataset of almost 100 million photos and videos is free and legal for all.