A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> The relationship between probability theory and fuzzy logic has long been an object of discussion and some controversy. The position articulated in this article is that probability theory by itself is not sufficient for dealing with uncertainty and imprecision in real-world settings. To enhance its effectiveness, probability theory needs an infusion of concepts and techniques drawn from fuzzy logic-especially the concept of a linguistic variable and the calculus of fuzzy if-then rules. In the final analysis, probability theory and fuzzy logic are complementary rather than competitive. <s> BIB001 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> Abstract A comparative study on a short-term water level prediction using artificial neural networks (ANN) and neuro-fuzzy system is addressed in this paper. The performance of the traditional approaches applied for such a hydrological task can often be constrained by data availability and simplifying assumptions in the processes description. In this paper, the ANN and neuro-fuzzy approaches are used for handling the situations with scarce data, where the predictions are based on the upstream hydrological conditions only. The models have been tested on two different river reaches in Germany. Moreover, the obtained results are compared to those of linear statistical models. Both ANN and neuro-fuzzy systems have performed comparably well and accurate for the purpose, explicitly outperforming the linear statistical models for a longer prediction horizon. The trained neural networks are partly implemented on-line, as a prototype of a web-based water level predictor. <s> BIB002 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> In the present context of water resource scarcity, a complete approach for long-term storage/transfer/distribution system management is proposed. The main management objective of such a kind of system is to manage reserves and releases so as to minimize the deficit between supply and demand by taking into account demand and contribution prediction. Thus, a sliding horizon approach and especially an adaptation procedure of weighting parameters of the minimization criterion based on fuzzy logic is implemented. This criterion adaptation concept seems entirely judicious, in view of the difficulty to define weighting parameters of such optimization problems and their influence on the relevant solution obtained. Fuzzy logic is shown to be very adequate when it comes to apprehend finely the stakes in presence in the long-term management. The long-term management problem is solved with improved heuristics using linear programming and dynamic programming in order to reduce the effects of space discretization which is so restrictive in this context. The suggested approach is applied to a case study which permits to highlight its relative simplicity of implementation. <s> BIB003 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> Abstract In this paper, a dynamic fuzzy network and its design based on genetic algorithm with variable-length chromosomes is proposed. First, the dynamic fuzzy network constituted from a series of dynamic fuzzy if–then rules is proposed. One characteristic of this network is its ability to deal with temporal problems. Then, the proposed genetic algorithm with variable-length chromosomes is adopted into the design process as a means of allowing the application of the network in situations where the actual desired output is unavailable. In the proposed genetic algorithm, the length of each chromosome varies with the number of rules coded in it. Using this algorithm, no pre-assignment of the number of rules in the dynamic fuzzy network is required, since it can always help to find the most suitable number of rules. All free parameters in the network, including the spatial input partition, consequent parameters and feedback connection weights, are tuned concurrently. To further promote the design performance, genetic algorithm with variable-length chromosomes and relative-based mutated reproduction operation is proposed. In this algorithm, the elite individuals are directly reproduced to the next generation only when their averaged similarity value is smaller than a similarity threshold; otherwise, the elites are mutated to the next generation. To show the efficiency of this dynamic fuzzy network designed by genetic algorithm with variable-length chromosomes and relative-based mutated reproduction operation, two temporal problems are simulated. The simulated results and comparisons with recurrent neural and fuzzy networks verify the efficacy and efficiency of the proposed approach. <s> BIB004 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> Until recently, local governments in Spain were using machines with rolling cylinders for verifying taximeters. However, the condition of the tires can lead to errors in the process and the mechanical construction of the test equipment is not compatible with certain vehicles. Thus, a new measurement device needs to be designed. In our opinion, the verification of a taximeter will not be reliable unless measurements taken on an actual taxi run are used. GPS sensors are intuitively well suited for this process, because they provide the position and the speed with independence of those car devices that are under test. But there are legal problems that make difficult the use of GPS-based sensors: GPS coordinate measurements do not match exactly real coordinates and, generally speaking, we are not given absolute tolerances. We can not know whether the maximum error is always lower than, for example, 7 m. However, we might know that 50% of the measurements lie on a circle with a radius of 7 m, centered on the real position. In this paper we describe a practical application where these legal problems have been solved with soft computing based technologies. In particular, we propose to characterize the uncertainty in the GPS with fuzzy techniques, so that we can reuse certain recent algorithms, formerly intended for being used in genetic fuzzy systems, to this new context. Specifically, we propose a new method for computing an upper bound of the length of the trajectory, taking into account the vagueness of the GPS data. This bound will be computed using a modified multiobjective evolutionary algorithm, which can optimize a fuzzy valued function. The accuracy of the measurements will be improved further by combining it with restrictions based on the dynamic behavior of the vehicles. <s> BIB005 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> Machine learning techniques often have to deal with noisy data, which may affect the accuracy of the resulting data models. Therefore, effectively dealing with noise is a key aspect in supervised learning to obtain reliable models from data. Although several authors have studied the effect of noise for some particular learners, comparisons of its effect among different learners are lacking. In this paper, we address this issue by systematically comparing how different degrees of noise affect four supervised learners that belong to different paradigms. Specifically, we consider the Naive Bayes probabilistic classifier, the C4.5 decision tree, the IBk instance-based learner and the SMO support vector machine. We have selected four methods which enable us to contrast different learning paradigms, and which are considered to be four of the top ten algorithms in data mining (Yu et al. 2007). We test them on a collection of data sets that are perturbed with noise in the input attributes and noise in the output class. As an initial hypothesis, we assign the techniques to two groups, NB with C4.5 and IBk with SMO, based on their proposed sensitivity to noise, the first group being the least sensitive. The analysis enables us to extract key observations about the effect of different types and degrees of noise on these learning techniques. In general, we find that Naive Bayes appears as the most robust algorithm, and SMO the least, relative to the other two techniques. However, we find that the underlying empirical behavior of the techniques is more complex, and varies depending on the noise type and the specific data set being processed. In general, noise in the training data set is found to give the most difficulty to the learners. <s> BIB006 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> In this paper, the problem of managing sewage sludge disposal on agricultural soils is addressed. Sewage sludge application on agricultural soils is recommended by governments in order to recycle nutrients and organic matter. However, this managing practice may lead to ecological and human risks. In this work, we propose the use of a multi-attribute preference approach to rate and rank a set of possible soils where some defined sewage sludge could be applied. The paper is focused on the assessment of the preference value functions. The variables involved in this problem are carefully explained and different preference assessment methods are proposed according to the different nature of the variables involved. In particular, the paper explains how fuzzy expert systems could be used for modelling the interactions between sludge and soil properties. <s> BIB007 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> This article briefly describes the advantages and disadvantages of crowdsourcing applications applied to disaster relief coordination. It also discusses several challenges that must be addressed to make crowdsourcing a useful tool that can effectively facilitate the relief progress in coordination, accuracy, and security. <s> BIB008 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> Real world pandemics such as SARS 2002 as well as popular fiction like the movie Contagion graphically depict the health threat of a global pandemic and the key role of epidemic intelligence (EI). While EI relies heavily on established indicator sources a new class of methods based on event alerting from unstructured digital Internet media is rapidly becoming acknowledged within the public health community. At the heart of automated information gathering systems is a technology called text mining. My contribution here is to provide an overview of the role that text mining technology plays in detecting epidemics and to synthesise my existing research on the BioCaster project. <s> BIB009 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Building the original data matrix <s> On 11th March 2011, a 9.0-magnitude megathrust earthquake occurred in the ocean near Japan. This was the first large-scale natural disaster in Japan since the broad adoption of social media tools (such as Facebook and Twitter). In particular, Twitter is suitable for broadcasting information, naturally making it the most frequently used social medias when disasters strike. This paper presents a topical analysis using text mining tools and shows the tools' effectiveness for the analysis of social media data analysis after a disaster. Though an ad hoc system without prepared resources was useful, an improved system with some syntactic pattern dictionaries showed better results. <s> BIB010
As said before, many different sources of information can be involved in the observation of an ES. In the recent years, data coming from smart sensors or images are quite usual. Today, the web is still not a common information source for ES analysis, but it might come in the near future as a relevant provider of both structured and unstructured data. This means that when an ES is analyzed, different data sources might be combined and some hard preprocessing might be required even before getting the very first original data matrix suitable for analysis. Thus, when information sources are images, and a data mining approach is to be adopted, some feature extraction methods  need to be applied to transform the image into a vector of indicators identifying the image itself, which can be properly represented in a classical two-dimensional data matrix. For example, in a land use application, satellite images are often used, and they might be synthesized in a vector of indicators like existence of river or not in the image, surface of the urban area, surface of forest, surface of crop area, etc. Feature extraction processes often are domain-dependent and require enough contextual knowledge to determine which are the relevant features to be extracted from the image, according also to the goals of the problem. Also, data might come from smart sensors, which are on-line, monitoring several parameters of an ES, like a Waste Water Treatment Plant (WWTP) for example. In that case, data corresponds to a certain instant time, and normally each observation has a time stamp, expressed in one of the variables of the data matrix. For instance, the data matrix can include biochemical parameters of the WWTP along time (like concentration of organic matter, suspended solids, etc.), providing, for example, daily means. In most applications, each parameter (or variable) will represent a column of the data matrix and the timestamp will be an additional variable. Sometimes, some pre-processing is required to integrate the information of several sensors in a single indicator (like mean water level of a bioreactor for example, measured by several sensors in different points) and the imprecision or uncertainty associated to the measurements must be properly treated at pre-processing level, in particular the noise associated to the signals BIB006 . Regarding imprecision, georadar, or GPS systems, for example, provide a region where the target can be located with a certain probability, but are not able to provide exact positions of the target in-stances. SCADA provides the minimum and maximum values of a signal in a certain interval. Strictly, despite being able to measure a magnitude, its exact value will be never known; it is only known that the measurement is somewhere in a certain range, bounded by the precision of the measurement instrument or measurement procedure itself BIB005 . In applications with these kinds of inputs, it makes sense to include the uncertainty associated to the measure itself in the representation of the values. Fuzzy sets BIB001 or rough sets  are among the most commonly used representation models for these situations, but require specific data mining methods accepting this kind of fuzzy or rough variables. In the first case, linguistic labels are used to represent the uncertainty associated to the values of the variables for every observation BIB007 . Fuzzy datasets can directly come from a monitoring system, like radar data, which intrinsically provides areas of localization. Also, original crisp data can be artificially fuzzified for specific analysis, taking into account the precision of the measurement instruments and using specific fuzzification algorithms (see BIB002 BIB003  BIB004   ). In this paper, we will provide pre-processing methods oriented to crisp representation of data, which is the most used for simplicity in current real applications. However, the reader has to keep in mind the possibility of dealing with these uncertainties naturally associated with raw data, by using specific uncertainty formalisms which are less known from a practical point of view. Even though the web data is not very popular yet in the current environmental applications, the web provides documents and pages that in some near future might become relevant for the environmental data mining. In that case, text mining methods need to be used to extract relevant characteristics of these documents that might also be represented in a classical two-dimensional data matrix, and can be used as ordinary inputs of classical data mining methods. There is a good chance to discover new findings and conclusions using modern methods of text mining supported by ontologies and semantic patterns. A survey of these methods is given in  . Text mining technology plays an important role in detecting epidemics (epidemic intelligence) based on event alerting from unstructured web pages and social networks BIB009 . A sample of the interest of environmental authorities in the monitoring of social networks for environmental purposes, or emergency management is the challenge on prediction of timing and intensity of seasonal climate events like hurricanes or earthquakes, among others BIB008 BIB010 . Also, sentiment analysis is currently providing a sig-