A Review On Evaluation And Benefits Of Decision Support Systems <s> How Is Evaluation Performed? <s> Evaluation Perspective and Approach - Overview and Case Study Integrating Evaluation Into Development Subject Evaluation Methods Technical Evaluation Methods Empirical Evaluation Methods Managing the Evaluation Process. <s> BIB001 </s> A Review On Evaluation And Benefits Of Decision Support Systems <s> How Is Evaluation Performed? <s> : The advance of Internet and Web technologies has boosted the development of electronic commerce. More and more people have changed their traditional trading behaviors and started to conduct Internet shopping. However, the exponentially increasing product information provided by Internet enterprises causes the problem of information overload, and this inevitably reduces the customer's satisfaction and loyalty. To overcome this problem, in this paper we propose a multi-agent system that is capable of eliciting expert knowledge and of recommending optimal products for individual consumers. The recommendations are based on both product knowledge from domain experts and the customer's preferences from system–consumer interactions. In addition, the system also uses behavior patterns collected from previous consumers to predict what the current consumer may expect. Experiments have been conducted and the results show that our system can give sensible recommendations, and it is able to adapt to the most up-to-date preferences for the customers. <s> BIB002 </s> A Review On Evaluation And Benefits Of Decision Support Systems <s> How Is Evaluation Performed? <s> This research explores how consumers use online decision aids with screening and evaluation support functionalities under varying product attribute-load conditions. Drawing on resource-matching theory, we conducted a 3 × 2 factorial experiment to test the interaction between decision aid features (i.e., low versus high-screening support, and aids with weight assignment and computation decision tools) and attribute load (i.e., large versus small number of product attributes) on decision performance. The findings reveal that: (1) where the decision aids render cognitive resources that match those demanded for the task environment, consumers will process more information and decision performance will be enhanced; (2) where the decision aids render cognitive resources that exceed those demanded for the task environment, consumers will engage in less task-related elaboration of decision-making issues to the detriment of decision performance; and (3) where the decision aids render cognitive resources that fall short of those demanded for the task environment, consumers will use simplistic heuristic decision strategies to the detriment of decision performance or invest additional effort in information processing to attain a better decision performance if they perceive the additional investments in effort to be manageable. <s> BIB003
The prototyping approach often used in decision support system development suggests the use of a multifaceted evaluation approach. Technical, empirical, and subjective facets comprise such an approach. Technical evaluation methods relate to system verification, whereas empirical evaluation methods assess validation. Subjective evaluation methods are applicable to both verification and validation, although they are used primarily in evaluating validation BIB001 . A number of recent studies are discussed below, indicating varied evaluation methods suggested and in use. The Defense Acquisition Guidebook, Chapter 9 "Test and Evaluation," (Defense, Department of, 2012) provides details of the U.S. Department of Defense testing and evaluation strategy for the three primary decision support systems used to acquire materiel and services. An event-driven integrated strategy situated within the program's overall acquisition strategy is used. The author claims the process used results in resource efficiencies and an enhanced data set useful for separate evaluations. The Department bases the integrated strategy on an evaluation framework. This framework indicates the links between key program and user decisions. It also describes developmental and operational areas requiring evaluation related to those decisions. Key performance parameters, key system attributes, critical technical parameters, key test measures (measures of suitability and effectiveness), planned test methods, key test resources, facility and infrastructure needs are specified. Additionally, any limitations or major risks to completing the evaluations are identified. The author notes that it is important in evaluation framework discussions to provide concise descriptions of links between key decisions in the system life cycle, areas needing evaluation to support those decisions, and the test methods needed to get the data for the evaluations. Examining research in the area of financial decision support,  indicate much of the focus falls into three areas: financial analysis (most common), risk management, and fraud detection. Most of the studies in their sample analyzed examined needs aspects of the DSS without explicitly involving the users. The authors point out that many evaluation methods, both quantitative and qualitative, are discussed in the design science literature as being appropriate, but all of the financial DSS studies they analyzed conducted an evaluation based on metrics such as precision, recall or accuracy, or based on simulation.  examines decision support in the realm of transport infrastructure assessments. Barfod suggests that in recent years, some organizational decisions need to incorporate certain key variables that are not socio-economic in nature, and assessment methods need to be used that can adequately capture the variables. Claiming that much prior research used simplistic cost-benefit analysis, the author proposes a composite model for assessment (COSIMA). The COSIMA DSS proposed uses multi-criteria decision analysis (MCDA) in conjunction with a cost-benefit analysis (required by regulatory authorities in some settings) to assess complex decision problems. The author suggests incorporating a decision conference including not only the decision-maker(s) but any other stakeholders who might interact with the DSS. This would constitute an intervention phase within the preliminary problem-structuring phase in which model building occurs. It would also allow for assessment revision to arrive at a shared understanding before the detailed analytic structure is derived. Asserting that examining the information load in terms of alternatives provided is more important than the number of attributes provided in the context of online electronic commerce, BIB003 explicitly manipulate the number of alternatives provided, examining the effect of decision aids on the decision performance. Using resource-matching theory, the authors test impacts on decision performance of specific decision aids (low screening support, high screening support, and weight evaluation support) in low and high product attribute-load conditions. They show that "DSS can lead to greater information processing or effort minimization, depending on whether there is a match or mismatch in cognitive resources available to the consumers and those demanded in the task environment." Karbing et al (2012) discuss the importance of conducting a retrospective evaluation of a DSS using archival data before conducting a prospective evaluation using live data. Quality of model fit can be evaluated with this retrospective evaluation in order to avoid in-use errors that might arise should the initial evaluation not be performed. Phillips-Wren et al (2011) use a combined process-outcome DSS evaluation approach. They suggest that linking the DSS benefits to the DSS architecture provides a better measure of the underlying decision value to the individual user than evaluating either the decision-making process or the outcome of the process in isolation. The approach they consider is that suggested by  , wherein a learning component is added to the intelligence, design, choice, and implementation decision phases first introduced by Simon (1960), and a four-level systems approach links the technical architecture of the DSS to the outcome and process elements at the top level. The authors use Analytic Hierarchy Process (AHP) to evaluate an intelligent DSS published by BIB002 . They demonstrate how AHP allows for unique criteria weights assigned to the DSS components for each individual user, and that the user effectiveness is enhanced by the application of AHP to a combined process-outcome approach. Using a simulation method, Ben-Zvi (2012) examines whether perceived DSS effectiveness is affected by six variables, including: familiarity with the system, perceived usefulness, use of system, perceived contribution to performance and success, user satisfaction, participation in defining the system, and system expectations being met. The results suggest that perceived DSS effectiveness is positively related to perceived usefulness, perceived contribution of the system, whether the system met users' expectations, and user satisfaction. However, no significant relationship is identified between system use, familiarity with the system, and participation in defining the system with perceived DSS effectiveness. The author concludes that researchers should be mindful of the importance of including certain subjective measures when examining the perceived effectiveness of a DSS. Additionally, the author indicates practitioners should be aware of the inverted U-shaped relationship between http://www.cluteinstitute.com/ © 2013 The Clute Institute system complexity and performance identified by this study and others before it. At some point, a system can become too complex for users to be able to use them effectively. In an experimental setting, examine whether model-driven DSS that facilitate the alignment of users' mental models with the system's recommendations are perceived as more valuable by the users. A dual-feedback DSS is designed to provide both feedback to help users correct their mental models and feedback about how much more could be gained by adjusting their mental model. The authors hypothesize that deep learning resulting in a relatively permanent change in someone's mental model would lead to a reduced cognitive dissonance and, therefore, improved DSS evaluations by users. Their findings suggest that, in order to encourage use, DSS should be designed to facilitate users' understanding of the basis for the DSS recommendations and why those recommendations will positively affect performance.