Context: Search-based software testing promises to provide users with the ability to generate high-quality test cases, and hence increase product quality, with a minimal increase in the time and effort required. One result that emerged out of a previous study to investigate the application of search-based software testing (SBST) in an industrial setting was the development of the Interactive Search-Based Software Testing (ISBST) system. ISBST allows users to interact with the underlying SBST system, guiding the search and assessing the results. An industrial evaluation indicated that the ISBST system could find test cases that are not created by testers employing manual techniques. The validity of the evaluation was threatened, however, by the low number of participants. Objective: This paper presents a follow-up study, to provide a more rigorous evaluation of the ISBST system. Method: To assess the ISBST system a two-way crossover controlled experiment was conducted with 58 students taking a Verification and Validation course. The NASA Task Load Index (NASA-TLX) is used to assess the workload experienced by the participants in the experiment. Results: The experimental results validated the hypothesis that the ISBST system generates test cases that are not found by the same participants employing manual testing techniques. A follow-up laboratory experiment also investigates the importance of interaction in obtaining the results. In addition to this main result, the subjective workload was assessed for each participant by means of the NASA-TLX tool. The evaluation showed that, while the ISBST system required more effort from the participants, they achieved the same performance. Conclusions: The paper provides evidence that the ISBST system develops test cases that are not found by manual techniques, and that interaction plays an important role in achieving that result.
At the medium level, a user can more directly guide the search by replacing the fitness function. Takagi proposed Interactive Evolutionary Computation (IEC), which he describes as an Evolutionary Computation (EC) that optimizes systems based on subjective human evaluation'' @cite_21 . This would allow the human user to guide the search according to their preference, intuition, emotion and psychological aspects'' @cite_21 . IEC could then see a wider spectrum of applications, including arts and animation.
@cite_21
2114395490 2151019861 2099020927 2178937556
We propose Visualized IEC as an interactive evolutionary computation (IEC) with visualizing individuals in a multidimensional searching space in a 2D space. This visualization helps us envision the landscape of an n-D searching space; so that it is easier for us to join an EC search, by indicating the possible global optimum estimated in the 2D mapped space. We experimentally evaluate the effect of visualization using a benchmark function. We use self-organizing maps for this projection of individuals onto a 2D space. The experimental result shows that the convergence speed of the GA with human search on the visualized space is at least five times faster than a conventional GA. We survey the research on interactive evolutionary computation (IEC). The IEC is an EC that optimizes systems based on subjective human evaluation. The definition and features of the IEC are first described and then followed by an overview of the IEC research. The overview primarily consists of application research and interface research. In this survey the IEC application fields include graphic arts and animation, 3D computer graphics lighting, music, editorial design, industrial design, facial image generation, speed processing and synthesis, hearing aid fitting, virtual reality, media database retrieval, data mining, image processing, control and robotics, food industry, geophysics, education, entertainment, social system, and so on. The interface research to reduce human fatigue is also included. Finally, we discuss the IEC from the point of the future research direction of computational intelligence. This paper features a survey of about 250 IEC research papers. We describe two approaches to reducing human fatigue in interactive evolutionary computation (IEC). A predictor function is used to estimate the human user's score, thus reducing the amount of effort required by the human user during the evolution process. The fuzzy system and four machine learning classifier algorithms are presented. Their performance in a real-world application, the IEC-based design of a micromachine resonating mass, is evaluated. The fuzzy system was composed of four simple rules, but was able to accurately predict the user's score 77 of the time on average. This is equivalent to a 51 reduction of human effort compared to using IEC without the predictor. The four machine learning approaches tested were k-nearest neighbors, decision tree, AdaBoosted decision tree, and support vector machines. These approaches achieved good accuracy on validation tests, but because of the great diversity in user scoring behavior, were unable to achieve equivalent results on the user test data. We evaluate and analyse a framework for evolutionary visual exploration EVE that guides users in exploring large search spaces. EVE uses an interactive evolutionary algorithm to steer the exploration of multidimensional data sets toward two-dimensional projections that are interesting to the analyst. Our method smoothly combines automatically calculated metrics and user input in order to propose pertinent views to the user. In this article, we revisit this framework and a prototype application that was developed as a demonstrator, and summarise our previous study with domain experts and its main findings. We then report on results from a new user study with a clearly predefined task, which examines how users leverage the system and how the system evolves to match their needs. While we previously showed that using EVE, domain experts were able to formulate interesting hypotheses and reach new insights when exploring freely, our new findings indicate that users, guided by the interactive evolutionary algorithm, are able to converge quickly to an interesting view of their data when a clear task is specified. We provide a detailed analysis of how users interact with an evolutionary algorithm and how the system responds to their exploration strategies and evaluation patterns. Our work aims at building a bridge between the domains of visual analytics and interactive evolution. The benefits are numerous, in particular for evaluating interactive evolutionary computation IEC techniques based on user study methodologies.