Although artificial intelligence holds promise for addressing societ al challenges, issues of exactly which tasks to automate and the extent to do so remain understudied. We approach the problem of task delegability from a human-centered perspective by developing a framework on human perception of task delegation to artificial intelligence. We consider four high-level factors that can contribute to a delegation decision: motivation, difficulty, risk, and trust. To obtain an empirical understanding of human preferences in different tasks, we build a dataset of 100 tasks from academic papers, popular media portrayal of AI, and everyday life. For each task, we administer a survey to collect judgments of each factor and ask subjects to pick the extent to which they prefer AI involvement. We find little preference for full AI control and a strong preference for machine-in-the-loop designs, in which humans play the leading role. Our framework can effectively predict human preferences in degrees of AI assistance. Among the four factors, trust is the most predictive of human preferences of optimal human-machine delegation. This framework represents a first step towards characterizing human preferences of automation across tasks. We hope this work may encourage and aid in future efforts towards understanding such individual attitudes; our goal is to inform the public and the AI research community rather than dictating any direction in technology development.
Trust and reliance on machines. Finally, we consider the community's interest in trust. As automation grows in complexity, complete understanding becomes impossible; trust serves as a proxy for rational decisions in the face of uncertainty, and appropriate use of technology becomes a concern @cite_28 . As such, calibration of trust continues to be a popular avenue of research @cite_30 @cite_36 . identify three bases of trust in automation: performance, process, and purpose. Performance describes the automation's ability to reliably achieve the operator's goals. Process describes the inner workings of the automation; examples include dependability, integrity, and interpretability (in particular, interpretable ML has received significant interest @cite_9 @cite_6 @cite_27 @cite_1 ). Finally, purpose refers to the intent behind the automation and its alignment with the user's goals.
@cite_30 @cite_28 @cite_36 @cite_9 @cite_1 @cite_6 @cite_27
2110171129 1973213502 2497215798 2290280890
Automation is often problematic because people fail to rely upon it appropriately. Because people respond to technology socially, trust influences reliance on automation. In particular, trust guides reliance when complexity and unanticipated situations make a complete understanding of the automation impractical. This review considers trust from the organizational, sociological, interpersonal, psychological, and neurological perspectives. It considers how the context, automation characteristics, and cognitive processes affect the appropriateness of trust. The context in which the automation is used influences automation performance and provides a goal-oriented perspective to assess automation characteristics along a dimension of attributional abstraction. These characteristics can influence trust through analytic, analogical, and affective processes. The challenges of extrapolating the concept of trust in people to trust in automation are discussed. A conceptual model integrates research regarding trust in automation and describes the dynamics of trust, the role of context, and the influence of display characteristics. Actual or potential applications of this research include improved designs of systems that require people to manage imperfect automation. Two experiments are reported which examined operators' trust in and use of the automation in a simulated supervisory process control task. Tests of the integrated model of human trust in machines proposed by Muir (1994) showed that models of interpersonal trust capture some important aspects of the nature and dynamics of human-machine trust. Results showed that operators' subjective ratings of trust in the automation were based mainly upon their perception of its competence. Trust was significantly reduced by any sign of incompetence in the automation, even one which had no effect on overall system performance. Operators' trust changed very little with experience, with a few notable exceptions. Distrust in one function of an automatic component spread to reduce trust in another function of the same component, but did not generalize to another independent automatic component in the same system, or to other systems. There was a high positive correlation between operators' trust in and use of the automation;... We present a shared-control framework predicated on a measure of trust in the operator, that is calculated automatically based on the quality of the interactions between a human and autonomous system. This measure of trust is built upon a control-theoretic foundation that rewards stable operation of the system to give more trusted users additional control authority. The level of control authority is used to modify the human input, and as a result, we observe a minimization of the required effort of the controller. We validate this work within a planar crane environment with a receding horizon controller to assist with the regulation of the system dynamics. The human defines the reference trajectory for the controller. In an experimental study users navigate a suspended payload through a set of maze configurations. We find that adaptation of the trust metric over time provides the benefit of substantially ( @math ) improving the automated system's ability to modulate the user's input, resulting in stable reference trajectories that require less effort to track. In effect, the human and automation spend less time fighting each other during task execution, suggesting that the automated system and user each have a better understanding of the other's ability. Throughout the history of automation, there have been numerous accidents attributed to the inappropriate use of automated systems. Over-reliance and under-reliance on automation are well documented problems in fields where automation has been employed for a long time, such as factories and aviation. Research has shown that one of the key factors that influence an operator's reliance on automated systems is his or her trust of the system. Several factors, including risk, workload, and task difficulty have been found to influence an operator's trust of an automated system. With a model of trust based upon these factors, it is possible to design automated systems that foster well-calibrated trust and thereby prevent the misuse of automation. Over the past decade, robot systems have become more commonplace and increasingly autonomous. With the increased use of robot systems in multiple application domains, models of trust and operator behavior for human-robot interaction (HRI) must be created now in order to avoid some of the problems encountered by other automation domains in the past. Since traditional automation domains and HRI are significantly different, we reexamine trust and control allocation (operator's usage of autonomous behaviors) as it relates to robots with autonomous capabilities in order to discover the relevant factors in HRI. This dissertation examines existing work in traditional automation that is relevant to HRI and, based on that information, builds an experimental methodology to closely mimic real world remote robot teleoperation tasks. We also present results from multiple experiments examining the relationship between the different factors being investigated with respect to trust and control allocation. Based on these results, a model for human interaction with remote robots for teleoperation (HARRT) is proposed and design guidelines to help improve overall performance are presented based on the model.