From visual data exploration to visual data mining: a survey <s> Selection of a Technique <s> In this paper we take a step toward addressing a pressing general problem in the development of data visualization systems — how to measure their effectiveness. The step we take is to define a model for specifying the generation of test data that can be employed for standardized and quantitative testing of a system’s performance. These test data sets, in conjunction with appropriate testing procedures, can provide a basis for certifying the effectiveness of a visualization system and for conducting comparative studies to steer system development. <s> BIB001 </s> From visual data exploration to visual data mining: a survey <s> Selection of a Technique <s> Visual data mining techniques have proven to be of high value in exploratory data analysis, and they also have a high potential for mining large databases. In this article, we describe and evaluate a new visualization-based approach to mining large databases. The basic idea of our visual data mining techniques is to represent as many data items as possible on the screen at the same time by mapping each data value to a pixel of the screen and arranging the pixels adequately. The major goal of this article is to evaluate our visual data mining techniques and to compare them to other well-known visualization techniques for multidimensional data: the parallel coordinate and stick-figure visualization techniques. For the evaluation of visual data mining techniques, the perception of data properties counts most, while the CPU time and the number of secondary storage accesses are only of secondary importance. In addition to testing the visualization techniques using real data, we developed a testing environment for database visualizations similar to the benchmark approach used for comparing the performance of database systems. The testing environment allows the generation of test data sets with predefined data characteristics which are important for comparing the perceptual abilities of visual data mining techniques. <s> BIB002 </s> From visual data exploration to visual data mining: a survey <s> Selection of a Technique <s> A variety of information visualization tools have been developed recently, but relatively little effort has been made to evaluate the effectiveness and utility of the tools. This article describes results from two empirical studies of two visualization tools for depicting hierarchies, in particular, computer file and directory structures. The two tools examined implement space-filling methodologies, one rectangular, the Treemap method, and one circular, the Sunburst method. Participants performed typical file/directory search and analysis tasks using the two tools. In general, performance trends favored the Sunburst tool with respect to correct task performance, particularly on initial use. Performance with Treemap tended to improve over time and use, suggesting a greater learning cost that was partially recouped over time. Each tool afforded somewhat different search strategies, which also appeared to influence performance. Finally, participants strongly preferred the Sunburst tool, citing better ability to convey structure and hierarchy. <s> BIB003
Classification schemes provide some initial insight on which techniques are oriented to certain data types, but one doesn't know for sure what makes a visualization technique more suitable than others to explore a particular data set. Selection of a system/technique depends largely on the task being supported and it is still a largely intuitive and ad hoc process. One has to rely on previous experience and knowledge and use multiple techniques so as to weight their relative strengths and weaknesses. Keim and Kriegel BIB002 and Keim  compare different visualization techniques by rating their capabilities in terms of data characteristics (maximum number of data attributes, maximum number of data items, capability of handling categorical data), tasks supported (clustering, multivariate hot spots), and visualization characteristics (visual overlap and learning curve). Albeit valid starting points for comparison, these are subjective evaluations based on personal experience. Not much work has been done on practical empirical evaluation of systems or techniques, either. One such work BIB003 describes an evaluation of two hierarchical information visualization techniques, the classical TreeMap and the Sunburst displays, at conveying attribute and structure information of computer directory and file structures and assist users in file browsing tasks. Additional work on controlled empirical evaluation of techniques and tools can generate valuable contributions to the field. In a distinct approach toward tackling the problem of measuring the effectiveness of data visualization systems, Keim et al. BIB001 try to define a model for specifying the generation of test data to be employed for standardized and quantitative testing of a system's performance. Coupled with appropriate testing procedures, such test data sets could provide a basis for certifying the effectiveness of a system and for comparing techniques. Achieving such a goal is a difficult matter, though, because "general" data sets are unlikely to meet the needs of users from specific domains. One attempt at quantitatively evaluating Table Data visualizations, by Hoffman  , is based on the definition of a Display Utilization Grid. This is an X-Y grid, defined on the visualization display area, whose resolution ideally matches the display area resolution. Thus, each grid element covers one pixel and it contains a list of all the data records that caused its activation. From such information, a set of metrics is computed for screen utilization and overlap and another one is computed on the graphical primitives used in the visualizations, namely marks, lines, and polygons. Two types of overlap statistics can be computed from these metrics, space overlap and object overlap. Space overlap results from a "crowding of points" in a certain region of the screen and it can usually be minimized by zooming in or by improving screen resolution. Object overlap results from multiple data items (for example, two identical data records) being mapped into overlapping graphical objects and generally cannot be alleviated by using higher screen resolution. Based on such metrics, Hoffman produced several visualizations of 10 publicly available data sets used for testing mining algorithms and "evaluated" their effectiveness to detect outliers, clusters, or interesting patterns in the data. However, the experimentation was not extensive and formal enough to be conclusive regarding the usefulness of such metrics for a priori selection of the potentially more effective techniques for a particular situation.