Survey on Fall Detection and Fall Prevention Using Wearable and External Sensors <s> Model Evaluation <s> Recommender systems are now popular both commercially and in the research community, where many algorithms have been suggested for providing recommendations. These algorithms typically perform differently in various domains and tasks. Therefore, it is important from the research perspective, as well as from a practical view, to be able to decide on an algorithm that matches the domain and the task of interest. The standard way to make such decisions is by comparing a number of algorithms offline using some evaluation metric. Indeed, many evaluation metrics have been suggested for comparing recommendation algorithms. The decision on the proper evaluation metric is often critical, as each metric may favor a different algorithm. In this paper we review the proper construction of offline experiments for deciding on the most appropriate algorithm. We discuss three important tasks of recommender systems, and classify a set of appropriate well known evaluation metrics for each task. We demonstrate how using an improper evaluation metric can lead to the selection of an improper algorithm for the task of interest. We also discuss other important considerations when designing offline experiments. <s> BIB001 </s> Survey on Fall Detection and Fall Prevention Using Wearable and External Sensors <s> Model Evaluation <s> Data Mining: Practical Machine Learning Tools and Techniques offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining. Thorough updates reflect the technical changes and modernizations that have taken place in the field since the last edition, including new material on Data Transformations, Ensemble Learning, Massive Data Sets, Multi-instance Learning, plus a new version of the popular Weka machine learning software developed by the authors. Witten, Frank, and Hall include both tried-and-true techniques of today as well as methods at the leading edge of contemporary research. *Provides a thorough grounding in machine learning concepts as well as practical advice on applying the tools and techniques to your data mining projects *Offers concrete tips and techniques for performance improvement that work by transforming the input or output in machine learning methods *Includes downloadable Weka software toolkit, a collection of machine learning algorithms for data mining tasks-in an updated, interactive interface. Algorithms in toolkit cover: data pre-processing, classification, regression, clustering, association rules, visualization <s> BIB002 </s> Survey on Fall Detection and Fall Prevention Using Wearable and External Sensors <s> Model Evaluation <s> Providing accurate and opportune information on people's activities and behaviors is one of the most important tasks in pervasive computing. Innumerable applications can be visualized, for instance, in medical, security, entertainment, and tactical scenarios. Despite human activity recognition (HAR) being an active field for more than a decade, there are still key aspects that, if addressed, would constitute a significant turn in the way people interact with mobile devices. This paper surveys the state of the art in HAR based on wearable sensors. A general architecture is first presented along with a description of the main components of any HAR system. We also propose a two-level taxonomy in accordance to the learning approach (either supervised or semi-supervised) and the response time (either offline or online). Then, the principal issues and challenges are discussed, as well as the main solutions to each one of them. Twenty eight systems are qualitatively evaluated in terms of recognition performance, energy consumption, obtrusiveness, and flexibility, among others. Finally, we present some open problems and ideas that, due to their high relevance, should be addressed in future research. <s> BIB003
Evaluation is a fundamental module in the learning process because it permits to systematically assess different learning techniques and compare them with one another BIB002 . Performance indicators are frequently used to measure the efficiency of the inferred structure BIB001 . The error rate is a common predictor employed in the assessment of learning machines. The system predicts the class of each instance: if it is correctly classified is marked as a success, otherwise is marked as an error. So, the error rate is a ratio of the errors over the whole set of instances. As stated before, the dataset is usually split and one part is used for training and the rest for testing. The latter is a considerable conditioning factor in the performance of the system. For instance, using the training set for testing purposes can be misleading; the estimated performance is highly optimistic. Another way to measure the performance of the system is through cross-validation. A fixed numbers of folds or partitions of the data are selected, some are used for training and the rest for testing. The process is repeated many times until all of the folds (group of instances) are used exactly once for testing. The results of every iteration are averaged and an overall error rate is calculated. Cross-validation is perhaps the method of choice in most of fall-related systems. Statistical tests are usually used with cross-validation to compare the performance of a classifier for a particular dataset BIB003 . The results of a classifier are commonly stored in an array known as confusion matrix. It allows to visualize the learning algorithm's performance in a specific table. An example of a confusion matrix is depicted in Figure 4 . • True Positives (TP): Number of positive instances that were classified as positive. • True Negatives (TN): Number of negative instances that were classified as negative. • False Positives (FP): Number of negative instances that were classified as positive. • False Negatives (FN): Number of positive instances that were classified as negative. Performance indicators used to evaluate the efficiency of learning algorithms are presented next. The accuracy of the system is the most extensively used performance indicator in classification problems. It is defined as follows: The recall or sensitivity or true positive rate is the ratio of the correctly classified positive instances over the entire set of positive instances. The precision or positive predicted value is the ratio of the number of correctly classified positive instances to the entire set of instances classified as positives. The specificity (SPC) or true negative rate is the proportion of the correctly classified instances as negative over the entire set of negative instances. The fall-out or false positive rate (FPR) is the proportion of the incorrectly classified instances as positive over the entire set of negative instances. A way to combine this indicators is through the F-measure (a measure of a test's accuracy). ROC (Receiver Operating Characteristic) curves are also used as a tool for diagnostic test evaluation. The performance of a binary classifier is plotted as the classifier's discrimination threshold is varied. It is created by plotting the recall vs. the fall-out, at various threshold settings. The threshold is used to determine the class the current instance belongs to, when the output of the classifier is a real value (continuous output). Figure 5 shows an example of a ROC curve. 