Role of context in usability evaluations: A review <s> Action Analysis Questionnaires <s> This paper describes the strengths and weaknesses of two usability assessment methods frequently applied to web sites. It uses case histories of WWW usability studies conducted by the authors to illustrate issues of special interest to designers of web sites. The discussion not only compares the two methods, but also discusses how an effective usability process can combine them, applying the methods at different times during site development. PREREQUISITES FOR ASSESSING WEB SITE USABILITY The two methods discussed in this paper for assessing the usability of web sites both require the usability specialist to have three vital pieces of background information: the purpose of the web site; profiles of its intended users; and typical scenarios for users accessing the site. These elements are equally important in evaluating the usability of any product or service, but here is how they apply especially to Web site evaluation. • When discussing the purpose of a web site, it’s helpful to consider three categories. Web sites that supply descriptions of companies (or other organizations) and their products, services, informational offerings, or events can be described as informational sites. Web sites that provide explicit links to extensive databases are called search sites. Web sites that behave like products, where users perform other tasks in addition to reading or retrieving information, are referred to as transactional sites. Multi-purpose sites blur these boundaries. • Unfortunately, the definition of user in our increasingly Web-centric environment is becoming more vague, because “anyone can access the site.” However, we must keep in mind which site visitors are the most likely—or the most welcome—and focus usability efforts on those subgroups. • Finally, the scenario for accessing a web site might be a straightforward URL to a home page, or a more roundabout path through a link in search-engine results to a page deep in the bowels of a site. Evaluators should keep in mind that any web page might be the user’s door to that web site. Although users may perform more complex tasks in transactional sites, the free-form nature of navigation in any type of web site makes ensuring (and measuring) success more complex in the Web environment. Home <s> BIB001 </s> Role of context in usability evaluations: A review <s> Action Analysis Questionnaires <s> The human-computer interaction community aims to increase the awareness and acceptance of established methods among software practitioners. Indeed, awareness of the basic usability methods will drive an Information Society for all. <s> BIB002 </s> Role of context in usability evaluations: A review <s> Action Analysis Questionnaires <s> We describe an empirical, between-subjects study on the use of think-aloud protocols in usability testing of a federal data-dissemination Web site. This double-blind study used three different types of think-aloud protocols: a traditional protocol, a speech-communication protocol, and a coaching protocol. A silent condition served as the control. Eighty participants were recruited and randomly pre-assigned to one of four conditions. Accuracy and efficiency measures were collected, and participants rated their subjective satisfaction with the site. Results show that accuracy is significantly higher in the coaching condition than in the other conditions. The traditional protocol and the speech-communication protocol are not statistically different from each other with regard to accuracy. Participants in the coaching condition are more satisfied with the Web site than participants in the traditional or speech-communication condition. In addition, there are no significant differences with respect to efficiency (time-on-task). This paper concludes with recommendations for usability practitioners. <s> BIB003
Heuristic evaluations are expert evaluations of products or systems, including information systems and documentation. They're conducted by usability specialists, domain experts, or preferably by "double experts" with both usability and domain experience BIB001 . Advantage of evaluation is that it can produce results in a limited time because it does not involve timeconsuming participant recruiting. The disadvantage is that the results of heuristic evaluation cannot be fully trusted as no real users are involved. Cognitive walkthrough is a task-oriented method by which the analyst explores the system's functionalities; that is, it simulates step-by-step user behavior for a given task. It emphasizes cognitive issues, such as learnability, by analyzing the mental processes required of the users BIB002 . Advantages include independence from end users and a fully functioning prototype, helping designers to take on a potential user's perspective, effective identification of problems arising from interaction with the system, and the ability to help to define users' goals and assumptions. Disadvantages of cognitive walkthrough include possible tediousness and the danger of an inherent bias due to improper task selection, emphasis on low-level details, and non involvement of the end user BIB002 . Action analysis involves a walkthrough of the actions a user will perform with regard to physical, cognitive, and perceptual loading. Advantages include precise prediction of how long a task will take, and a deep insight into users' behavior. Disadvantages of action analysis include it is very time-consuming and requires high expertise BIB002 . With the think aloud protocol, areas where a user is struggling and the reasons for the difficulties are verbally articulated. The usability practitioner uses this information along with other metrics to identify problem areas of the Web site or application being assessed and to devise suggestions for improvement. One of the most common think aloud protocols that usability practitioners engage in today is concurrent think aloud under which the participant is encouraged to "think out loud" while working on a task BIB003 . Field observation is the simplest of all methods. It involves visiting one or more users in their workplaces. Notes must be taken as unobtrusively as possible to avoid interfering with their work. Observation focuses on major usability catastrophes BIB002 . Questionnaires are indirect usability measures. 