From What to How. An Overview of AI Ethics Tools, Methods and Research to Translate Principles into Practices <s> Methodology <s> A lot of privacy principles have been proposed in the literature with the aim to preserve users’ privacy through the protection of the personal data collected by service providers. Despite the fact that there were remarkable efforts to gather all privacy principles and use them on a common privacy-by-design system, to the best of our knowledge, there is no published methodology that combines in a clear and structured way the existing privacy principles for supporting the design of a Privacy Preserving System. The absence of a widely accepted structured representation of the privacy principles makes their adoption or/and satisfaction difficult and in some cases inconsistent. Considering that privacy protection on its own is not an easy task for an organisation, the “scattered” privacy principles impose significant additional complexity. Consequently, very frequently organizations fail to effectively protect the privacy of their users. In this paper a structured privacy audit methodology that consists of discrete steps that organizations can follow for deciding or/and auditing the privacy protection measures is proposed. Every step is based on the significance of a privacy principle and on the sequence of the audit procedure. <s> BIB001 </s> From What to How. An Overview of AI Ethics Tools, Methods and Research to Translate Principles into Practices <s> Methodology <s> The success of statistical machine learning (ML) methods made the field of Artificial Intelligence (AI) so popular again, after the last AI winter. Meanwhile deep learning approaches even exceed human performance in particular tasks. However, such approaches have some disadvantages besides of needing big quality data, much computational power and engineering effort; those approaches are becoming increasingly opaque, and even if we understand the underlying mathematical principles of such models they still lack explicit declarative knowledge. For example, words are mapped to high-dimensional vectors, making them unintelligible to humans. What we need in the future are context-adaptive procedures, i.e. systems that construct contextual explanatory models for classes of real-world phenomena. This is the goal of explainable AI, which is not a new field; rather, the problem of explainability is as old as AI itself. While rule-based approaches of early AI were comprehensible “glass-box” approaches at least in narrow domains, their weakness was in dealing with uncertainties of the real world. Maybe one step further is in linking probabilistic learning methods with large knowledge representations (ontologies) and logical approaches, thus making results re-traceable, explainable and comprehensible on demand. <s> BIB002
The first task was to design a typology of applied AI tools, for the very practically minded ML community BIB002 , to embed reflection on the ethical principles of beneficence, nonmaleficence, autonomy, justice and explicability (Floridi and colleagues, 2018) ) into the ML pipeline from pre-processing, to use by the end user BIB002 . To do this, the ethical principles were combined with the stages of algorithmic development outlined in the overview of the Information Commissioner's Office (ICO) auditing framework for artificial intelligence and its core components, 13 as shown in Figure 2 , to encourage ML developers to go between design decision and ethical principles regularly. The third, and final task, was to review the recommendations, theories, methodologies, and tools outlined in the reviewed sources, and identify where they may fit in the typology. The ultimate ambition is to ensure that there is a choice of mature tools in each box of the typology (this is not the case currently), so that ML developers have aids to ethically-informed design at each stage of the process. For example, as illustrated in figure 2 , a developer looking to ensure their ML algorithm is 'non-maleficent' (unlikely to cause harm related to privacy or security issues) can start with the foundational principles of privacy by design (Cavoukian and colleagues, 2010) to guide ideation appropriately, use techniques such as data minimisation (Antignac and colleagues, 2016) , training for adversarial robustness (Kolter & Madry, 2018) , and decision-making verification (Dennis and colleagues, 2016) in the train-build-test phases, and end by launching the system with an accompanying privacy audit procedure BIB001 . What is revealing about this example, and what makes it illustrative of one of the overarching limitations of many of the tools included in the typology, is that all the techniques referenced are currently in the academic research stage and, although promising, they require more work before being 'production-ready.' This current lack of 'market-testing' means that applying 