The development and design of visualization solutions that are truly usable is essential for ensuring both their adoption and effectiveness. User-centered design principles, which focus on involving users throughout the entire development process, are well suited for visualization and have been shown to be effective in numerous information visualization endeavors. In this paper, we report a two year long collaboration with combustion scientists that, by applying these design principles, generated multiple results including an in situ visualization technique and a post hoc probability distribution function (PDF) exploration tool. Furthermore, we examine the importance of user-centered design principles and describe lessons learned over the design process in an effort to aid others who also seek to work with scientists for developing effective and usable scientific visualization solutions.
We take a new, scenario-based look at evaluation in information visualization. Our seven scenarios, evaluating visual data analysis and reasoning, evaluating user performance, evaluating user experience, evaluating environments and work practices, evaluating communication through visualization, evaluating visualization algorithms, and evaluating collaborative data analysis were derived through an extensive literature review of over 800 visualization publications. These scenarios distinguish different study goals and types of research questions and are illustrated through example studies. Through this broad survey and the distillation of these scenarios, we make two contributions. One, we encapsulate the current practices in the information visualization research community and, two, we provide a different approach to reaching decisions about what might be the most effective evaluation of a given information visualization. Scenarios can be used to choose appropriate research questions and goals and the provided examples can be consulted for guidance on how to design one's own study. A revision of the #1 text in the Human Computer Interaction field, Interaction Design, the third edition is an ideal resource for learning the interdisciplinary skills needed for interaction design, human-computer interaction, information design, web design and ubiquitous computing.The authorsare acknowledged leaders and educators in their field, with a strong global reputation. They bring depth of scope to the subject in this new edition, encompassing the latest technologies and devices including social networking, Web 2.0 and mobile devices. The third edition also adds, develops and updates cases, examples and questions to bring the book in line with the latest in Human Computer Interaction.Interaction Design offers a cross-disciplinary, practical and process-oriented approach to Human Computer Interaction, showing not just what principles ought to apply to Interaction Design, but crucially how they can be applied. The book focuses on how to design interactive products that enhance and extend the way people communicate, interact and work. Motivating examples are included to illustrate both technical, but also social and ethical issues, making the book approachable and adaptable for both Computer Science and non-Computer Science users. Interviews with key HCI luminaries are included and provide an insight into current and future trends.The book has an accompanying website www.id-book.com which has been updated to include resources to match the new edition. Traditionally, studies of data visualization techniques and systems have evaluated visualizations with respect to usability goals such as effectiveness and efficiency. These studies assess performance-related metrics such as time and correctness of participants completing analytic tasks. Alternatively, several studies in InfoVis recently have evaluated visualizations by investigating user experience goals such as memorability, engagement, enjoyment and fun. These studies employ somewhat different evaluation methodologies to assess these other goals. The growing number of these studies, their alternative methodologies, and disagreements concerning their importance have motivated us to more carefully examine them. In this article, we review this growing collection of visualization evaluations that examine user experience goals and we discuss multiple issues regarding the studies including questions about their motivation and utility. Our aim is to provide a resource for future work that plans to evaluate visualizations using these goals. We present an assessment of the state and historic development of evaluation practices as reported in papers published at the IEEE Visualization conference. Our goal is to reflect on a meta-level about evaluation in our community through a systematic understanding of the characteristics and goals of presented evaluations. For this purpose we conducted a systematic review of ten years of evaluations in the published papers using and extending a coding scheme previously established by [2012]. The results of our review include an overview of the most common evaluation goals in the community, how they evolved over time, and how they contrast or align to those of the IEEE Information Visualization conference. In particular, we found that evaluations specific to assessing resulting images and algorithm performance are the most prevalent (with consistently 80-90 of all papers since 1997). However, especially over the last six years there is a steady increase in evaluation methods that include participants, either by evaluating their performances and subjective feedback or by evaluating their work practices and their improved analysis and reasoning capabilities using visual tools. Up to 2010, this trend in the IEEE Visualization conference was much more pronounced than in the IEEE Information Visualization conference which only showed an increasing percentage of evaluation through user performance and experience testing. Since 2011, however, also papers in IEEE Information Visualization show such an increase of evaluations of work practices and analysis as well as reasoning using visual tools. Further, we found that generally the studies reporting requirements analyses and domain-specific work practices are too informally reported which hinders cross-comparison and lowers external validity.