Tools for Achieving Usable Ex Post Transparency: A Survey <s> IV. LITERATURE RESEARCH <s> A review of prior, relevant literature is an essential feature of any academic project. An effective review creates a firm foundation for advancing knowledge. It facilitates theory development, closes areas where a plethora of research exists, and uncovers areas where research is needed. <s> BIB001 </s> Tools for Achieving Usable Ex Post Transparency: A Survey <s> IV. LITERATURE RESEARCH <s> The work presented in this paper introduces the concept of On-line Interactive (OI) privacy feature which is defined as any on-line interactive tool, component or user-interface that creates privacy awareness and supports users in understanding their on-line privacy risks. These features have been developed as an interactive social translucence map that discloses the flow of personal information, a privacy enquiry for a direct chat about users' privacy concerns and a discussion forum presenting users' privacy concerns using their language in an interactive FAQ format. The paper presents an evaluation of a prototype of this set of embedded OI privacy features. The field study presented evaluates the prototype's usability and its effect on users' privacy awareness, understanding and attitude. 100 participants took part in the study and were drawn from groups of experienced and less experienced users. Both quantitative and qualitative data collection methods were used. Findings suggest that OI privacy features increase users' privacy awareness and encourage users to find out more about the uses of their personal data. However, users' ICT skills and Internet experience significantly influence whether a feature is favoured or otherwise. In general, it is concluded that privacy features are very much welcomed and necessary to empower users to manage their privacy concerns but some groups need to be further supported by social and institutional privacy management processes. <s> BIB002 </s> Tools for Achieving Usable Ex Post Transparency: A Survey <s> IV. LITERATURE RESEARCH <s> Context: Many researchers adopting systematic reviews (SRs) have also published papers discussing problems with the SR methodology and suggestions for improving it. Since guidelines for SRs in software engineering (SE) were last updated in 2007, we believe it is time to investigate whether the guidelines need to be amended in the light of recent research. Objective: To identify, evaluate and synthesize research published by software engineering researchers concerning their experiences of performing SRs and their proposals for improving the SR process. Method: We undertook a systematic review of papers reporting experiences of undertaking SRs and/or discussing techniques that could be used to improve the SR process. Studies were classified with respect to the stage in the SR process they addressed, whether they related to education or problems faced by novices and whether they proposed the use of textual analysis tools. Results: We identified 68 papers reporting 63 unique studies published in SE conferences and journals between 2005 and mid-2012. The most common criticisms of SRs were that they take a long time, that SE digital libraries are not appropriate for broad literature searches and that assessing the quality of empirical studies of different types is difficult. Conclusion: We recommend removing advice to use structured questions to construct search strings and including advice to use a quasi-gold standard based on a limited manual search to assist the construction of search stings and evaluation of the search process. Textual analysis tools are likely to be useful for inclusion/exclusion decisions and search string construction but require more stringent evaluation. SE researchers would benefit from tools to manage the SR process but existing tools need independent validation. Quality assessment of studies using a variety of empirical methods remains a major problem. <s> BIB003 </s> Tools for Achieving Usable Ex Post Transparency: A Survey <s> IV. LITERATURE RESEARCH <s> We present a prototype of the user interface of a transparency tool that displays an overview of a user's data disclosures to different online service providers and allows them to access data collected about them stored at the services' sides. We explore one particular type of visualization method consisting of tracing lines that connect a user's disclosed personal attributes to the service to which these attributes have been disclosed. We report on the ongoing iterative process of design of such visualization, the challenges encountered and the possibilities for future improvements. <s> BIB004 </s> Tools for Achieving Usable Ex Post Transparency: A Survey <s> IV. LITERATURE RESEARCH <s> Transparency is a basic privacy principle and social trust factor. However, in the age of cloud computing and big data, providing transparency becomes increasingly a challenge. <s> BIB005 </s> Tools for Achieving Usable Ex Post Transparency: A Survey <s> IV. LITERATURE RESEARCH <s> This technical report documents the procedure of a literature review conducted on usable ex post-transparency enhancing tools (TETs). The review of scientific literature serves the purpose of provi ... <s> BIB006
The methodology used for the literature research was based on the procedure described by Webster et al. BIB001 . However, instead of relying on a carefully selected, handpicked set of articles that serves as a starting point for the subsequent search, the initial approach suggested by Kitchenham et al. BIB003 was chosen. The latter approach relies on an initial database search that yields a set of publications on which, in turn, all further selection is based. The initial search was based on database queries performed on the databases provided by Inspec  and DBLP [21] , as well as the publication databases of the two publishers ACM [22] and IEEE [23] . These databases were chosen because of the high quality of the publications available in or referenced by them. This choice biased the research conducted throughout the retrieved publications towards the disciplines of computer science, information systems and engineering. The thematic restriction was considered as much a limitation for the breadth of the review as it was considered a deliberate choice regarding the feasibility of the systematic approach. Each database was queried using the semantic equivalent of the same set of well-defined search terms. The terms were derived from the intersection of the three thematic areas of privacy, transparency and usability. The logical combination of a set of respective search terms defined the scope in which relevant publications were assumed to reside. The query of the literature databases resulted in over 800 unique papers. After the first review of these papers, it was obvious that the number of publications dealing with ex ante TETs by far exceeded the ones that covered ex post TETs. It was at this point that the contextual scope of the literature review was further narrowed down to implementations of ex post TETs and disregarded papers that exclusively dealt with ex ante scenarios. During the subsequent screening phase, all publications were checked for content-related relevance, as well as for a set of selection criteria that warranted the scientific relevance of the papers. The rigorous screening of the retrieved publications ensured that only publications were selected that represented original implementations that existed in the form of usable prototypes. It was considered a plus if a paper elaborated on the user study conducted to verify the usability of the TET, but such studies were not a criterion for inclusion in the result set. The screening process resulted in 12 publications that met the specified prerequisites. The subsequent snowballing phase traced the references of the publications backwards and forwards, and was carried out according to the recommendations of Webster et al. BIB001 . Scopus was used to generate a set of references based on the original result set retrieved during the initial search phase, and yielded more than 300 additional publications. The screening of these articles was conducted by applying the exact same criteria that had been used during the screening of the initial result set. The screening of the references yielded nine additional publications, resulting in 21 publications that met the specified criteria. A comprehensive description of the information retrieval and screening process is being provided as a technical report BIB006 . That report also provides brief summaries of the individual functionality of each of the reviewed papers. After finalising the report, two additional papers, BIB002 and  , were included that meet the selection criteria. Moreover, the implementations of the TETs presented by Fischer-Hübner et al. BIB005 were broken down into two distinctive TETs, BIB004 and  , increasing the total number of papers to 24. A concise overview of the reviewed papers is provided in Table 1 . Each of the final set of 24 publications describes an implementation that thematically qualifies as an ex post TET, and that meets the criteria specified for the screening process. According to the descriptions provided in the publications, the implementations vary in terms of maturity. They range from prototypical implementations in the testing stage to software tools that have been tested repeatedly, and that appear to be ready to be used by the target audience they were designed for. Similarly, the usage contexts of the reviewed TETs differ in such a way that their variety in terms of functionality impedes a systematic comparison in absolute terms. However, it is possible to compare them in terms of abstract characteristics, which are represented in the form of the taxonomy introduced in Section V.