Role of context in usability evaluations: A review <s> Based on physical context <s> There are new challenges to us, as researchers, on how to design and evaluate new mobile applications because they give users access to powerful computing devices through small interfaces, which typically have limited input facilities. One way of overcoming these shortcomings is to utilize the possibilities of multimodality. We report in this paper how we designed, developed, and evaluated a multimodal mobile application through a combination of laboratory and field studies. This is the first time, as far as we know, that a multimodal application has been developed in such a way. We did this so that we would understand more about where and when users envisioned using different modes of interaction and what problems they may encounter when using an application in context. <s> BIB001 </s> Role of context in usability evaluations: A review <s> Based on physical context <s> Environmental conditions arguably influence mobile website usability, especially as users on the move may be exposed to fluctuating contextual conditions. Since the aspects of context cannot be readily simulated in a mobile usability study, the development of theories and models explaining how the context of use may affect usability has yet to be sufficiently addressed. In this paper we propose a model hypothesizing on the effects of environmental distractions on the usability of mobile sites. Environmental distractions are categorized into auditory, visual and social and we suggest both direct and moderated effects of these distractions on efficiency, effectiveness, and perceived satisfaction. The paper contributes to the mobile HCI literature by developing a concrete set of empirically testable hypotheses and a method for their subsequent assessment. A preliminary test (N=30) was conducted to investigate our main hypotheses. Results confirmed that environmental distractions have a direct effect on mobile website usability. <s> BIB002 </s> Role of context in usability evaluations: A review <s> Based on physical context <s> In usability evaluations, experiments are often conducted in closed laboratory situations to avoid disturbing influences. Due to non-realistic usage assumptions, this approach has important shortcomings when mobile Human Computer Interactions (m-HCI) have to be evaluated. Field studies allow to perform experiments close to real-world conditions, but potentially introduce influences caused by the environment. In this paper, we aim at distinguishing application shortcomings from environmental disturbances which both potentially cause decreased user performance. Our approach is based on monitoring environmental conditions during the usability experiment, such as light, acceleration, sound, temperature, and humidity, and relating them to user actions. Therefore, a mobile context-framework has been developed based on a small Wireless Sensor Network (WSN). First results are presented that point at increased delays and error rates of user tasks under induced environmental disturbances. Additionally, we demonstrate the potential of environmental monitoring for understanding user performance. <s> BIB003 </s> Role of context in usability evaluations: A review <s> Based on physical context <s> Usability testing has traditionally been conducted in dedicated laboratory. Alternatively, field study can be an option to conduct the usability testing. As HCI researchers who work with children, we have experiences in conducting usability tests in both natural and laboratory environments. We tested two different children applications in two different environments: testing drawing applications at their preschool and testing an educational game in our usability laboratory. Thus, this paper highlights our experiences in conducting both usability tests. Although the literature suggests that laboratory and field studies should be done for two different purposes, our experiences have told us that we can actually integrate the steps of field studies into the guidelines for lab tests. These steps are particularly useful for the novice designers and researchers who have never done usability evaluations with children before. <s> BIB004 </s> Role of context in usability evaluations: A review <s> Based on physical context <s> This study proposes a new methodology for conducting synchronous remote usability studies using a three-dimensional virtual usability testing laboratory built using the Open Wonderland toolkit. This virtual laboratory method is then compared with two other commonly used synchronous usability test methods: the traditional lab approach and WebEx, a web-based conferencing and screen sharing approach. A study was conducted with 48 participants in total, 36 test participants and 12 test facilitators. The test participants completed 5 tasks on a simulated e-commerce website. The three methodologies were compared with respect to the following dependent variables: the time taken to complete the tasks; the usability defects identified; the severity of these usability defects; and the subjective ratings from NASA-TLX, presence and post-test subjective questionnaires. The three methodologies agreed closely in terms of the total number defects identified, number of high severity defects identified and the time taken to complete the tasks. However, there was a significant difference in the workload experienced by the test participants and facilitators, with the traditional lab condition imposing the least and the virtual lab and the WebEx conditions imposing similar levels. It was also found that the test participants experienced greater involvement and a more immersive experience in the virtual world condition than the WebEx condition. These ratings were not significantly different from those in the traditional lab condition. The results of this study suggest that participants were productive and enjoyed the virtual lab condition, indicating the potential of a virtual world based approach as an alternative to the conventional approaches for synchronous usability testing. <s> BIB005
The importance of physical context in usability evaluations have been researched for a long. Out of the many factors that can effect usability evaluations, physical context is considered to directly influence the behaviour of the people involved in the usability evaluations. The physical context may include the location, the temperature, the time, the light etc. Tsiaousis & Giaglis BIB002 examined the effects of environmental distractions on mobile website usability. They proposed a model hypothesizing on the effects of environmental distractions on the usability of mobile sites. They categorized the environmental distractions into auditory, visual and social. A preliminary test on 20 users was conducted to investigate the effect of environmental distractions on mobile website usability. Results confirmed that environmental distractions have direct effect on mobile website usability. Hummel et al. BIB003 developed a mobile context-framework based on a small wireless sensor network, to monitor environmental conditions such as light, acceleration, sound, temperature, and humidity during the usability experiments. User experiments have been conducted in a laboratory with seven test persons where the environmental conditions were changed. Under varying environmental conditions the performance of the users on the average was decreased in terms of higher error rates and delays. Kaikkonen et al.  carried out usability testing of mobile consumer application in two environments: in a laboratory and in a field with a total of 40 test users. Results indicate that conducting a time-consuming field test may not be worthwhile when searching user interface flaws to improve user interaction. They found that field testing is worthwhile when combining usability tests with a field pilot or contextual study where user behaviour is investigated in a natural context. Razak et al. BIB004 conducted usability testing with children in both laboratory and field. Drawing applications were tested in their preschool and an educational game was tested in the usability laboratory. The results indicate that field study is more suitable for understanding children experience with technology than it is with testing for usability problems and laboratory study is more suitable for evaluating user interfaces and interaction with the application than it is with understanding children's experience. Andrrzejczak & Liu  examined the effect of location on the user's stress level during usability evaluation. User stress levels were assessed by Spielberger's State-Trait Anxiety Inventory; using the paper survey's baseline and experimental stress scores. In addition, user performance data was recorded through task times and subjective user assessments. The data suggested no significant differences exist between participant data in both baseline and experimental anxiety scores. This implies that remote testing as a cost-efficient way to conduct user testing, may be a viable alternative to traditional lab testing without altering the test's effectiveness. Madathil BIB005 performed a synchronous remote usability test using a three-dimensional virtual world, and empirically compared it with WebEx, a web-based two-dimensional screen sharing and conferencing tool, and the traditional lab method. The results suggest that virtual lab method is as effective as the traditional lab and WebEx based methods in terms of the time taken by the test participants to complete the tasks and the number of higher severity defects identified. Test participants and facilitators alike experienced lower overall workload in the traditional lab environment than in either of the remote testing environments. Baillie & Schatz BIB001 evaluated a multimodal mobile application through a combination of laboratory and field studies. The users were given a set of four action scenarios to be performed. The results were surprising; only one action scenario was completed in the time frame whereas three out of four action scenarios were completed in lesser time. Error rates were higher in lab than in the field. The reason for such performances by the users could be that the users feel more relaxed in the field.