Given the fast rise of increasingly autonomous artificial agents and robots, a key acceptability criterion will be the possible moral implications of their actions. In particular, intelligent persuasive systems (systems designed to influence humans via communication) constitute a highly sensitive topic because of their intrinsically social nature. Still, ethical studies in this area are rare and tend to focus on the output of the required action. Instead, this work focuses on the persuasive acts themselves (e.g. "is it morally acceptable that a machine lies or appeals to the emotions of a person to persuade her, even if for a good end?"). Exploiting a behavioral approach, based on human assessment of moral dilemmas -- i.e. without any prior assumption of underlying ethical theories -- this paper reports on a set of experiments. These experiments address the type of persuader (human or machine), the strategies adopted (purely argumentative, appeal to positive emotions, appeal to negative emotions, lie) and the circumstances. Findings display no differences due to the agent, mild acceptability for persuasion and reveal that truth-conditional reasoning (i.e. argument validity) is a significant dimension affecting subjects' judgment. Some implications for the design of intelligent persuasive systems are discussed.
Recent findings suggest that exerting executive control influences responses to moral dilemmas. In our study, subjects judged how morally appropriate it would be for them to kill one person to save others. They made these judgments in 24 dilemmas that systematically varied physical directness of killing, personal risk to the subject, inevitability of the death, and intentionality of the action. All four of these variables demonstrated main effects. Executive control was indexed by scores on working-memory-capacity (WMC) tasks. People with higher WMC found certain types of killing more appropriate than did those with lower WMC and were more consistent in their judgments. We also report interactions between manipulated variables that implicate complex emotion-cognition integration processes not captured by current dual-process views of moral judgment. Recent work shows an important asymmetry in lay intuitions about moral dilemmas. Most people think it is permissible to divert a train so that it will kill one innocent person instead of five, but most people think that it is not permissible to push a stranger in front of a train to save five innocents. We argue that recent emotion-based explanations of this asymmetry have neglected the contribution that rules make to reasoning about moral dilemmas. In two experiments, we find that participants show a parallel asymmetry about versions of the dilemmas that have minimized emotional force. In a third experiment, we find that people distinguish between whether an action violates a moral rule and whether it is, all things considered, wrong. We propose that judgments of whether an action is wrong, all things considered, implicate a complex set of psychological processes, including representations of rules, emotional responses, and assessments of costs and benefits.