Clinical practice guideline (CPG) activity has escalated internationally in the last 20 years, leading to increasingly sophisticated methods for CPG developers and implementers. Despite this, there remains a lack of practical support for end-users in terms of effectively and efficiently implementing CPG recommendations into local practice. This paper describes South African experiences in implementing international CPG recommendations for best practice stroke rehabilitation into local contexts, using a purpose-build approach.Composite recommendations were synthesised from 16 international CPGs to address end-user questions about best practice rehabilitation for South African stroke survivors. End-user representatives on the project team included methodologists, policy-makers, clinicians, managers, educators, researchers and stroke survivors. The Adopt–Contextualise–Adapt model was applied as a decision-guide to streamline discussions on endorsement and development of implementation strategies. Where recommendations required contextualisation to address local barriers before they could be effectively implemented, prompts were provided to identify barriers and possible solutions. Where recommendations could not be implemented without additional local evidence (adaptation), options were identified to establish new evidence.The structured implementation process was efficient in terms of time, effort, resources and problem solving. The process empowered the project team to make practical decisions about local uptake of international recommendations, develop local implementation strategies, and determine who was responsible, for what and when. Different implementation strategies for the same recommendation were identified for different settings, to address different barriers.The South African evidence translation experience could be useful for evidence implementers in other countries, when translating CPG recommendations developed elsewhere, into local practice. WHO has called for innovative, evidence-based rehabilitation action to deal with the increasing burden of disability internationally  and has a convenient way of packaging evidence and presenting recommendations to healthcare decision makers” [n = 2), United States of America (n = 5), Australia (n = 5) and one each from Canada, Malaysia, New Zealand and South Africa [Reflecting the increasing prevalence of stroke worldwide, and its impact on individuals, families and society, there has been significant international research activity resulting in many clinical practice guidelines (CPGs). Treweek et al. described CPGs as “ makers” . Our recrica n = , Australglobalizing the evidence but localizing the decision” CPGs generally provide recommendations about ‘what to do’, based on intervention studies . EvidencIn South Africa, stroke is a leading cause of disability among adults of all ages due to co-morbidities such as HIV/AIDS and tuberculosis , 10, 11.We know what we have to do, but we don’t know how to do it” was noted by Ridde as a “recurring comment among global health actors” [There is increasing research about implementing evidence in LMICs , 12–17.  actors” –17. It a actors” , 15. Alo actors” –20). How actors” , 21, 22.a) prepare well; b) involve the relevant people; c) develop a proposal for change that is evidence based, feasible, and attractive; d) study the main difficulties in achieving the change, and select a set of strategies and measures at different levels linked to that problem; of course, within your budget and possibilities; e) define indicators for measurement of success and monitor progress continuously or at regular intervals; and, finally, f) enjoy working on making patients’ care more effective, efficient, safe, and friendly” [An increasing volume of literature has been published since 1990 on frameworks, models and strategies to support behaviour change, evidence uptake, transfer of evidence to practice and evidence implementation. This underscores the intellectual investment of many international researchers in making sense of the complexities of putting evidence into effective, sustainable practice. This interest was flagged in 2001, when Grimshaw et al.  summarisriendly” .ost determinant frameworks provide limited “how-to” support for carrying out implementation endeavours since the determinants usually are too generic to provide sufficient detail for guiding an implementation process” .in Box 2 –37.We mapped our understanding of the complexity of South African rehabilitation , 11, 16 Reflecting the high resource and time costs of developing de novo CPGs, there is increasing interest in transferring CPG recommendations developed in one setting to another. Two approaches have been proposed for transferring CPG recommendations between HICs  which investigated the use of CPGs in South African PHC , 43. An The Adopt–Contextualise–Adapt (ACA) approach was developed to run parallel with the Project SAGE CPG Tiers model , and undThis paper describes experiences in addressing the ‘how to do it’ to implement stroke rehabilitation recommendations from CPGs developed elsewhere into South African contexts. These experiences may assist other LMICs wishing to implement HIC CPG recommendations into local practice.Seed funding was provided by Stellenbosch University and the WHO Alliance for Health Policy and Systems Research in February 2017, to produce evidence-based guidance for stroke rehabilitation relevant to South Africa. The draft guidance document was required to be completed by December 2017 (a 10-month project). There was neither time nor resources to ‘recreate the wheel’ of evidence about ‘what to do’, nor, in fact, was there any need for this given the wealth of evidence-based guidance already available internationally for effective stroke rehabilitation . The decThe project team identified 38 questions for which answers (recommendations) were sought from international stroke rehabilitation CPGs. Using the new methods for evidence synthesis , 78 compWe sought guidance from the implementation literature by considering commonly cited frameworks, theories and models (Box 2). We also considered the summaries of gaps in the implementation research literature , 9, 15 aThe scope and purpose of the funding for the new South African-relevant stroke rehabilitation CPG was to address an urgent national need to improve access, equity and quality of rehabilitation for people with stroke. We believed that we could provide the vehicle for ‘how to do it’ by developing locally relevant implementation plans for recommendations complied from 16 international CPGs for stroke rehabilitation .A national team comprising methodologists, stroke researchers, educators, national and provincial policy-makers, facility managers, expert clinicians and stroke survivors was established. Team members brought broad end-user perspectives and knowledge to the table to assist with barrier identification and solutions. Their engagement and input reflected Tier 2 of the CPG Tiers model . The teaThe project’s implementation focus and the ACA model made sense to the project team , 44, 45.The methodologists on the team developed a solution-focused algorithm to operationalise the ACA approach efficiently Fig. . We labeThe first endorsement process Fig. a was to globalise evidence, localize (implementation) decisions” [The second endorsement process Fig. b was to cisions” . Thus, bcisions” .Table 1BThe third endorsement process Fig. c appliedif only we could …. ’, or ‘what about doing …. ’?, or ‘how long would that take us?’.Discussions involved different configurations of end-users (e.g. within the same group (just clinicians or just policy-makers) as well as in mixed groups ). Discussions were based on the ACA decision-making algorithm Fig.  and the For each endorsement, the project team discussed auditable outcome measures that might demonstrate the success of implementation plans.N = 69). The A1 endorsements are provided in Box 3 (Prof Louw can be contacted for other recommendations). Exemplar endorsed recommendations, their composite evidence strength and implementation strategies are provided in Table Of the 78 composite recommendations derived from the included CPGs, 13 had insufficient or equivocal evidence, and the team did not pursue endorsements for these (leaving 65 endorsable recommendations). Of these, four attracted two endorsements each (most commonly adoption A1 or A2) coupled with recognition that additional evidence was needed (C1 or C2)). The most common endorsements overall were A1 (Adopt within the next 6 months) and A2 (Adopt over the longer term) (25.1%) and B1  (24.6%). Figure  coupled enjoy working on making patients’ care more effective, efficient, safe, and friendly” ([This is only the second LMIC research project that we know of that has taken planned steps to progress evidence statements to implementation actions (the first occurring in the Philippines ). In hiniendly”  http://www.strokebestpractices.ca/acute-stroke-management/Malaysian Stroke Guideline (2016) http://www.acadmed.org.my/index.cfm?menuid = 67#Cardiovascular_Diseasehttps://www.nice.org.uk/guidance/cg162National Institute for Health and Clinical Excellence (2013) https://www.rcplondon.ac.uk/guidelines-policy/stroke-guidelinesRoyal College of Physicians (2012) https://www.ahajournals.org/doi/pdf/10.1161/STR.0000000000000098American Heart Association/American Stroke Association (Rehabilitation Guideline) (2015) https://www.guidelinecentral.com/summaries/occupational-therapy-practice-guidelines-for-adults-with-stroke/American Occupational Therapy Association (sourced through Guidelines Clearing House AHRQ) (2013) https://www.rehab.research.va.gov/jour/10/479/pdf/vadodcliniaclguidlines479.pdfDepartment of Defense, Veterans Association Management of Stroke Guidelines (2010) https://www.health.govt.nz/publication/new-zealand-clinical-guidelines-stroke-management-2010New Zealand Guidelines Group (2010) https://www.aci.health.nsw.gov.au/networks/strokeNSW Agency for Clinical Innovation (2016) https://www.sahealth.sa.gov.au/wps/wcm/connect/ae53950047066243b403fc22d29d99f6/Clinical+Guideline_Stroke+Management_Proceudres+and+Protocols_final+Oct14pdf?MOD=AJPERES&CACHEID=ae53950047066243b403fc22d29d99f6&CACHE=NONESouth Australian Dept Health Stroke Network (2017) https://www.guidelinecentral.com/summaries/stroke-diagnosis-and-initial-management-of-acute-stroke-and-transient-ischaemic-attack-tia/Guidelines Clearing House https://www.sign.ac.uk/assets/sign119.pdfDysphasia (2010) https://www.sign.ac.uk/sign-118-management-of-patients-with-stroke-rehabilitation,-prevention-and-management-of-complicati.htmlRehabilitation (2010) Scottish Intercollegiate Guidelines Networkhttps://www.safetyandquality.gov.au/our-work/clinical-care-standards/acute-stroke-clinical-care-standard/Australian Council on Safety and Quality in Health Care (2015) https://strokefoundation.org.au/What-we-do/Treatment-programs/Clinical-guidelinesAustralian Stroke Foundation (2017) https://www.ncbi.nlm.nih.gov/pubmed/21081029South African Stroke Society (2010) Box 2 Examples of frameworks, models and strategies for aspects of implementationOttawa Model of Research Use Diffusion of innovations in service organisations Domains of behaviour change The OFF theory Pipeline of knowledge uptake The Knowledge to Action Model TCU Organisational Readiness for Change Translating Evidence into Practice Model Behaviour Change Wheel Consolidated Framework for Implementation Research Steps for developing a theory-informed intervention Conceptual framework for planning and improving evidence-based practices Context and Implementation of Complex Interventions framework Ontology of Behaviour Change Interventions Box 3 Recommendations with A1 endorsements (adoptable in South African rehabilitation sites within 6 months)4. There are consistent strong recommendations that the rehabilitation processes should commence in the acute setting as soon as the person with stroke is medically safe and/or able to participate.8. There are consistent suggestions that a standardised clinical assessment should be applied by a professional skilled in the management of dysphagia (currently speech and language therapists).10. There are consistent suggestions that education should be made available to all healthcare providers about adverse events following stroke.11. There are consistent suggestions that all patients with a stroke should be mobilised as early as possible, to lessen likelihood of complications such as pneumonia, deep vein thrombosis, pulmonary embolism and pressure sores.12. There are consistent suggestions that patients with mild and moderate stroke should be provided with frequent, short activity sessions.13. There are consistent strong recommendations against the routine use of splints or prolonged positioning of upper or lower limb muscles in a lengthened position (stretch) for stroke survivors who are at risk of developing contracture.Previous functional abilities;Impairment of psychological functioning  and communication;Impairment of body functions, including pain/orientation;Activity limitations and participation restrictions, e.g. positioning, moving, transfer and handling;Swallowing;Pressure area risk;Continence;Nutritional status and hydration;Environmental factors .42. There are consistent suggestions that comprehensive assessment of rehabilitation needs should include:48. There are consistent strong recommendations that physiotherapists, occupational therapists, speech and language therapists, and dieticians bring specific competencies and skills to patient assessment and rehabilitation planning as members of the multidisciplinary team. They operate most effectively when sharing assessment and rehabilitation tasks, and communicating findings verbally and in written form in patient notes.50. There are consistent suggestions that treatment decisions should be clearly documented.51. There are consistent suggestions that progression of rehabilitation programmes should be documented, including reason for progression and patient responses.53. There are consistent suggestions that progress reports on interventions and outcomes should be communicated regularly within the team, and to the patient and family.58. There are consistent suggestions that adaptive and assistive devices should be used for safety and function, if other methods of performing the task/activity are not available or cannot be learned or if the patient’s safety is a concern.60. There are consistent strong recommendations that Discharge Planning should include all members of the multidisciplinary team, and the patient and family.61. There are consistent strong recommendations that Discharge Planning should articulate patient and family circumstances.65. There are consistent suggestions that every member of the multidisciplinary team should take responsibility for planning and monitoring the continuation of care.67. There are consistent strong recommendations that information about patient progress should be recorded formally in patient notes and shared at Discharge Planning meetings.68. There are consistent suggestions that one member of the multidisciplinary team should take overall responsibility of Discharge Planning to ensure continuity.74. There are consistent suggestions that longer term care for stroke survivors should reflect their goals and circumstances.Prepare well: The composite evidence base, derived from the 16 stroke rehabilitation CPGs, was welcomed by the team, which recognised that it did not have to do this work prior to discussing local implementation , 6. The Invite the relevant people: The initial meeting was attended by approximately 40 people. Subsequent meetings were attended by a core group of approximately 15 people , as well as new people who developed an interest in the project, or who deputised for others who could not attend. This was why repeated explanations of the new CPG methods was essential. Most participants attended at their own cost , as the budget for the project did not extend to travel costs, venue hire, reimbursement for time or even refreshments. Attendance at meetings often required extensive travel (8 hours driving or 2-hour flights) and overnight stays. An important feature was the regular attendance at meetings of National and Provincial rehabilitation policy-makers, who provided insights into current and future Government policy, priorities, appetite for change and funding.Develop a proposal for change that is evidence based, feasible and attractive: The ACA decision-making algorithm Fig. , the barStudy the main difficulties in achieving the change, and select a set of strategies and measures at different levels linked to that problem, within your budget and possibilities: The biggest challenges in this work were to give participants the confidence to identify and ‘unpick’ local barriers to effective evidence uptake, and to encourage them to identify practical local solutions. Given the range of South African settings in which stroke rehabilitation occurs, it was remarkable therefore that so many  recommenDefine indicators for measurement of success and monitor progress continuously or at regular intervals: Whilst participants recognised the importance of demonstrating, using defensible data, that quality improvements were being made in delivery of equitable, accessible and effective care, the task of identifying appropriate measures has not yet been achieved. This reflects not only the lack of standard national, provincial or even regional data collection about any health condition or rehabilitation activity , 18, butThe Grimshaw steps  provided


To measure non-cognitive facets of competence, we developed and tested a new method that we refer to as Embedded Experience Sampling (EES). Domain-specific problem-solving competence is a multi-faceted construct that is not limited to cognitive facets such as domain knowledge or problem-solving strategies but also comprises non-cognitive facets in the sense of domain-specific emotional and motivational dispositions such as, for instance, interest and self-concept. However, in empirical studies non-cognitive facets are usually either neglected or measured by generalized self-report questionnaires that are detached from the performance assessment. To enable an integrated measurement, we developed the EES method to collect data on non-cognitive facets during scenario-based low-stakes assessments. Test-takers are requested to stop at certain times and spontaneously answer short items (EES items) regarding their actual experience of the problem situation. These EES items are embedded in an EES event that resembles typical social interactions with non-player characters. To evaluate the feasibility and validity of the method, we implemented EES in a series of three studies in the context of commercial vocational education and training (VET): A feasibility study with 77 trainees, a pilot study with 20 trainees, and the main study with 780 trainees who worked on three complex problem scenarios in a computer-based office simulation. In the present paper, we investigate how test-takers perceived the EES events, and whether social desirability biased their answers, and investigate the internal structure of the data and the relationship between EES data and data from several other sources. Interview data and survey data indicated no biases due to social desirability and no additional burden for the test-takers due to the EES events. A correlation analysis following the multitrait-multimethod approach as well as the calibration of a multidimensional model based on Item Response Theory (IRT) also supported the construct validity. Furthermore, EES data shows substantial correlations with test motivation but almost zero correlations with data from generalized retrospective self-report questionnaires on non-cognitive facets. Altogether, EES offers an alternative approach to measuring non-cognitive facets of competence under certain conditions. For instance, EES is also based on self-reporting and thus might not be suitable for high-stakes testing. This contribution outlines the characteristics and implementation of EES and presents findings concerning its validity gained by conducting three empirical studies throughout the developmental process.Problem-solving competence has gained increasing attention in educational science as well as in vocational education and training (VET) and professional development. In vocational and professional contexts, problem-solving competence is important because of a general trend toward higher-order skills owing to the ongoing automatization and outsourcing of routine tasks that not only affect blue-collar work in production lines but also white-collar work e.g., . ProblemIn his seminal report, When solely focusing on the assessment of cognitive facets of competence, it is implicitly assumed that test-takers invest maximum effort to perform as well as possible. Test performance is interpreted as maximum performance in the sense of Based on a literature review, we developed a competence model that distinguishes knowledge application, action regulation, self-concept, and interest as components of domain-specific problem-solving competence . We furtThe non-cognitive components (self-concept and interest) mirror the expectancy-value theory of achievement motivation  and the Based on a multidimensional understanding of competence, a crucial question is how non-cognitive facets are measured. Two basic options in dealing with the multidimensionality of the construct can be distinguished .Following this very common approach, non-cognitive facets are part of a multifaceted construct of competence but are measured separately, usually by administering retrospective self-report questionnaires. Those self-reports remain detached from the actual performance. In general, self-reports are considered face-valid  but therin situ assessment of non-cognitive facets is based on the repeated measurement of emotional states in the context of different problem scenarios. A multitrait-multimethod approach  which was introduced by in situ and furthermore integrates these self-reports into the storyline of authentic problem scenarios. Test-takers are briefly interrupted during the test and requested to answer short questions (EES items) regarding their momentary experience. These EES items are embedded into the test situation in authentic EES events that resemble ordinary social interaction at the workplace . Closed-ended questions were used in order to spare the test-takers the time they would need to write down their answers. Furthermore, they improve the comparability of the answers and facilitate the implementation of EES in large-scale assessments regarding psychometric scaling. EES items focus on difficult to monitor non-cognitive competence facets such as interest, attitudes, commitment, and self-concept.Our development of Embedded Experience Sampling (EES) builds on traditional ESM. In order to measure the non-cognitive facets in computer-based tests on problem-solving competence, EES aims at collecting self-report data on non-cognitive facets A similar approach was applied in PISA 2006 as an “embedded science interest assessment”. Directly after working on selected test items regarding science competence, the participants were requested to rate their situational interest in the prior item context. The data were calibrated in Item Response Theory (IRT) models to assess trait interest . Howeverin situ faces limitations: for instance, social desirability may affect individuals’ responses and possibly lead to a bias in the psychometric data in terms of construct-irrelevant variance ,•Whether social desirability biased their answers (RQ2),•The internal structure of the data (RQ3) and•The relationship between EES data and (a) CSSM data, (b) test motivation, and (c) generalized retrospective self-reports (RQ4).The studies were part of the research project ‘modellng and measuring domain-specific problem-solving competence of industrial clerks’ (DomPL-IK), which was funded by the German Federal Ministry of Education and Research (Grant No. 01DB081119–01DB1123). The apprenticeship program to become an industrial clerk is the fifth most frequent of nearly 330 state-recognized apprenticeship programs in the well-respected German dual system of vocational education and training (VET). Apprenticeship programs usually require 3 years to complete and are characterized by a combination of workplace learning in the training company and classroom-based learning in state-run vocational schools. Certified industrial clerks usually work in back-office departments of industrial or service companies. A general description of the research project and selected results have been published in In the present article, we focus on the development and validation of the EES approach by analyzing EES data from two pilot studies and the main study. In a first feasibility study, we investigated how participants perceived the EES events, whether social desirability played a role, whether the EES data met the requirements of the MTMM approach, and how EES data were correlated to retrospective measures of interest and self-concept. The aims of the second pilot study was to test the computer-based office simulation that, for the first time, also included a computer-based implementation of EES events. Additional data were collected to investigate the subjective experience of the EES, social desirability in EES responses, and the relation to CSSM data and test motivation. Finally, the computer-based assessment of domain-specific problem-solving competence was implemented in a large-scale study with almost 800 participants in vocational schools in six federal German states. The resulting EES data were calibrated in a psychometric model based on Item Response Theory (IRT). Parts of this final step of the test development are published in N = 77 students in vocational education and training (VET) of two vocational business schools in Germany. All participants were enrolled in a 3-year apprenticeship program to become industrial clerks and were nearing the end of their 2nd year of the apprenticeship. The sample included 28 male and 49 female participants who showed a typical age distribution . Participation was voluntary and all participants provided written informed consent.The feasibility of implementing EES in the assessment of domain-specific problem-solving competence was investigated in a pilot study with Data were collected in computer-equipped classrooms. At the beginning of the data collection sessions the researchers introduced themselves, the project, and the agenda. First, the participants completed several self-report questionnaires including scales on vocational interest and work-related self-efficacy. Next, they worked on three authentic, computer-based business problems including the completion of several EES items , a supplier selection (40 min), and a make-or-buy decision (50 min). Each scenario started with an email from a supervisor which included a problem and a variety of documents of varying relevance, transparency, and credibility. All scenarios required participants to go through multiple processes of information seeking, processing, and interpreting. To complete a scenario, the participants had to reply to the initial email with a well-founded proposed solution. The test environment provided “open book” conditions meaning that participants could look up technical terms, formulae, legal regulations etc. in a large reference work. However, they were not allowed to consult any other sources such as the internet. The participants used Microsoft Excelence see . For furIn this feasibility study, four EES events were implemented into each of the above problem scenarios. In this early stage of the project, EES events were paper-based and came in separate envelopes that were numbered consecutively and placed on each participant’s desk (see Appendix The data of the two EES items concerning the competence facet “confidence in one’s competence” (C1) were condensed into one scale for each scenario. The internal consistencies were not satisfactory . “Situational confidence in one’s solution” (C2) was measured with a single item see . The datWe administered a scale designed to measure work-related self-efficacy . The scaTo investigate how the participants experienced the EES, two group discussions in class (with approximately 20 participants each) and 11 individual interviews were conducted. Participants were asked how they experienced the procedure . They were asked whether they had deliberated about alternative responses and whether answering these questions had caused additional stress during their work on the problem situations.r < 0.30 indicate small effects, 0.30 < r < 0.50 indicate medium effects, and r > 0.50 indicate large effects. The interview data were categorized with regard to social desirability and the additional burden of answering the EES items while working on the problem scenarios. The data was analyzed using IBM SPSS 24.Following a multitrait-multimethod (MTMM) approach, the various facets of competence are multiple traits and the three scenarios are multiple methods. Although the variables were not normally distributed (Shapiro–Wilk tests), parametric Pearson correlations were calculated since this method is considered robust . In corrThe mean values for the EES variables range between 1.71 and 3.19 on a four-step scale  within a scenario (method) argue for situational influences of the scenario, while high monotrait-heteromethod correlations between the same competence facets (traits) measured in different scenarios (method) argue for trait influences. r = 0.28 while the mean correlation of all 12 monotrait-heteromethod combinations is r = 0.33, which is consistent with the MTMM assumption. Heterotrait-monomethod correlations different from zero are plausible because the theoretical constructs are not assumed to be fully independent of each other. The monotrait-heteromethod correlations are higher which supports the assumption of internal validity and thus supports H3a . We hypothesized small correlations (H4c) even though the theoretical constructs are quite similar.Finally, by calculating mean scores across the EES variables, we received two EES-based scales, one for self-concept and one for interest. The correlations between EES-based scales and scales from generalized self-reports of work-related self-efficacy and vocational interest were close to zero and not significant .To test the computer-based implementation of EES events and the subjective experience of the EES, 21 VET students participated voluntarily in this pilot study and provided written informed consent. Eight participants were male and 13 were female; the participants were 20.3 years old on average  and via a short questionnaire on test motivation and one’s experience with the EES events directly after the problem scenario. Furthermore, the participants completed a longer questionnaire that included biographic information as well as several standardized scales, one of which was applied to measure a disposition toward socially desirable responding.In this pilot study of the technological implementation, four EES events were defined. However, due to a technical malfunction the fourth EES event was not presented to the participants. In this validation study, the EES events were also presented within the office simulation for the first time. Embedded experience sampling data were condensed in the same way as in the feasibility study (Study 1), resulting in four EES variables for the competence facets shown in ®). In 5-min intervals, the participants were requested to rate three statements on a scale from 0 to 100. The items were: (1) Right now, this is very interesting. (2) Right now, I am making great efforts. (3) Right now, I am making great progress. Participants were carefully instructed that this data collection was not part of the assessment and that they were expected to answer honestly according to their actual experience, while no such announcement was made for the EES events. To become familiarized with the method, the first point of measurement was before the problem scenario and was not included in the analysis. Six measurement points followed during the problem scenario at minute 4′, 9′, 14′, 19′, 24′, and 29′. Scales were calculated from the six items of each statement. Internal consistencies  were C.A. = 0.70 for “interesting,” C.A. = 0.78 for “effort” and C.A. = 0.67 for “progress.”Continuous state sampling method data was collected during the problem scenario via mobile devices ”  in a Gerauthenticity of the EES events  and the additional burden due to the EES events .In the same short questionnaire directly after the problem scenario two additional questions were aimed at assessing the We administered an adapted version of the Effort Thermometer which For correlation analysis, Kendall’s tau-b correlations were calculated because the data were not normally distributed and the sample size was small. The data was analyzed using IBM SPSS 24.On average, the participants experienced the problem scenario as not being very interesting . They invested medium effort according to the CSSM scale “effort” and showed a correspondingly medium test motivation as measured by the Effort Thermometer. With regard to the EES events, the participants did not report that they tried to “look good” when answering the EES items. On average, they experienced the EES events as being quite authentic and hardly as an additional burden .A total of 786 VET students participated in the study, of which six were excluded from the analyses due to missing data . The participating VET students were in the 2nd or 3rd year of a 3-year commercial apprenticeship program, 50.1% were female and the sample showed a typical right skewed age distribution . Cronbach’s alpha was 0.69. An adapted version of a scale to measure dispositional interests in students . Cronbach’s alpha was 0.76.We administered a questionnaire on work-related self-efficacy  which costudents  was admiTo assess the cognitive facets of competence  as an approach to measuring non-cognitive facets of domain-specific problem-solving competence within a computer-based office simulation. The feasibility and validity of EES were investigated throughout three studies by using different measures and analysis approaches. Most of the results support the validity of EES. The results are discussed with regard to the research questions and hypotheses that were outlined previously see .Research question 1 aimed at the test-takers’ perception of the EES events in terms of ecological validity. It was hypothesized that participants in low-stake tests do not experience EES events as an additional and unrealistic burden, a finding supported by group discussions and individual interviews in study 1 and by survey data in study 2. Despite experiencing the scenario as quite difficult, they considered it to be authentic and, on average, did not evaluate EES as an additional burden.Research question 2 aimed at social desirability as a potential bias in terms of construct validity. In study 1, the participants’ responses in group discussions and individual interviews gave no reasons to assume biases from social desirability. In study 2, the dispositional tendency for impression management was uncorrelated with the EES responses and situational impression management  showed very small correlations with the EES responses. Altogether, social desirability does not appear as a source of bias in EES responses.Research question 3 aimed at assessing the consistency of the EES data with assumptions of the Multitrait-Multimethod approach (MTMM) and the requirements of a multidimensional model based on Item Response Theory (IRT) in terms of internal validity. In study 1, low correlations of heterotrait-monomethod combinations and higher correlations of monotrait-heteromethod combinations support the assumption of internal validity, however, the differences are only small. In study 3, the EES data was calibrated in a multidimensional IRT model and showed satisfactory EAP/PV reliabilities for five of the six facets while one facet had to be excluded due to low reliability. Altogether, our analysis supports the assumption of internal validity.Research question 4 aimed at the correlation of EES data with CSSM data (Continuous State Sampling Method) and test motivation in terms of convergent validity and the correlation between EES data and generalized retrospective self-reports in terms of divergent validity. Substantial correlations between EES data and test motivation support the assumption of convergent validity, while the correlations between EES data and CSSM data were more heterogeneous. Low  correlations between EES data and generalized retrospective self-reports in study 1 and study 3 emphasize the significance of the measurement approach.Altogether, we collected data on the feasibility and validity of EES throughout three field studies on problem-solving competence in the business domain and found very promising results. Embedding self-reports on situational experience into the “storyline” of authentic problem scenarios produces reliable and valid data on non-cognitive facets of problem-solving competence.Both, the methodological approach and the empirical studies have their limitations. First and foremost, we have not tested for external validity, namely by measuring whether emotional states in the test situations are good proxies for emotional states in respective work situations, which constitutes a strong assumption; not only for the non-cognitive facets but also for the cognitive facets of competence. However, it is very difficult to put together an appropriate research design and collect the respective data to investigate these assumptions. Furthermore, data collection in EES is still based on self-reporting. In our studies, we did not find indications of social desirability or of an additional burden due to EES. However, these studies comprised low-stakes testing. In high-stakes testing, responses to EES items are prone to manipulation and EES events might be experienced as more disruptive. The operationalization of the non-cognitive facets of problem-solving competence is arguable. In study 1, the internal consistencies of EES items measuring the same facet were not satisfactory. Many alternative items would have been just as appropriate or maybe more appropriate as indicators of the respective facet. We have not experimented widely with the operationalization of the facets. One significant alteration concerned the facet D3 “interest in the progress of/in learning from the problem.” However, this alteration worsened the model fit and resulted in the exclusion of facet D3 from the IRT model in Study 3, while the correlations within the MTMM analysis in Study 1 had been quite promising. We will vary the item content and the item format in future studies and we encourage other research teams to apply similar approaches in their studies, too.Limitations of Study 1 and Study 2 were the smaller sample sizes that did not allow for more sophisticated analyses. In Study 2, the CSSM items could have been more similar to the other EES items. Only the items regarding situational interest were very similar. In future studies, more appropriate CSSM items should be applied. Moreover, physiological measures such as heart rate (HR), heart rate variability (HRV), skin conductance or cortisol may be used to further validate the EES data. One such study was conducted by Twenty years ago, Drawing on our experience, EES is a feasible and informative approach to measuring non-cognitive facets of competence under the following conditions: (1) The computer-based performance assessment is embedded in an immersive and authentic simulation of a real-life domain. (2) The participants are confronted with comprehensive scenarios that require a sustained performance. (3) The participants are introduced to EES within a tutorial prior to the performance assessment. Drawing on our empirical studies, we found indications of the validity of EES. We would like to encourage other researchers to implement EES or similar approaches into their studies of competence assessment because further research is needed for the subsequent development and validation of the method.This study was carried out in accordance with the recommendations of the Ethical Board of the University of Bamberg, Germany and the Board of Data Protection of the Federal State Authority of Bavaria (Germany) with written informed consent from all subjects.AR, KK, and JS contributed to method, data collection, and preparation of the manuscript. AR led the project.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


N-acetylgalactosamine-6-sulfatase (GALNS) enzyme, which impairs lysosomal degradation of keratan sulphate and chondroitin-6-sulphate. The multiple clinical manifestations of MPS IVA present numerous challenges for management and necessitate the need for individualised treatment. Although treatment guidelines are available, the methodology used to develop this guidance has come under increased scrutiny. This programme was conducted to provide evidence-based, expert-agreed recommendations to optimise management of MPS IVA.Mucopolysaccharidosis (MPS) IVA or Morquio A syndrome is an autosomal recessive lysosomal storage disorder (LSD) caused by deficiency of the Twenty six international healthcare professionals across multiple disciplines, with expertise in managing MPS IVA, and three patient advocates formed the Steering Committee (SC) and contributed to the development of this guidance. Representatives from six Patient Advocacy Groups (PAGs) were interviewed to gain insights on patient perspectives. A modified-Delphi methodology was used to demonstrate consensus among a wider group of healthcare professionals with experience managing patients with MPS IVA and the manuscript was evaluated against the validated Appraisal of Guidelines for Research and Evaluation (AGREE II) instrument by three independent reviewers.A total of 87 guidance statements were developed covering five domains: (1) general management principles; (2) recommended routine monitoring and assessments; (3) disease-modifying interventions (enzyme replacement therapy [ERT] and haematopoietic stem cell transplantation [HSCT]); (4) interventions to support respiratory and sleep disorders; (5) anaesthetics and surgical interventions . Consensus was reached on all statements after two rounds of voting. The overall guideline AGREE II assessment score obtained for the development of the guidance was 5.3/7 .This manuscript provides evidence- and consensus-based recommendations for the management of patients with MPS IVA and is for use by healthcare professionals that manage the holistic care of patients with the intention to improve clinical- and patient-reported outcomes and enhance patient quality of life. It is recognised that the guidance provided represents a point in time and further research is required to address current knowledge and evidence gaps.The online version of this article (10.1186/s13023-019-1074-9) contains supplementary material, which is available to authorized users. N-acetylgalactosamine-6-sulfatase (GALNS) enzyme (EC 3.1.6.4), which impairs lysosomal degradation of keratan sulphate and chondroitin-6-sulphate . To gain an understanding of the experience and views of patients with MPS IVA/VI on the holistic management of MPS across several geographies and healthcare systems, the consultations addressed: the biggest challenges faced by patients with MPS IVA/VI; the impact of medical/surgical interventions on patient outcomes, including: physical function/performance, mental/emotional outcomes and socioeconomic outcomes/cost implications; the overall benefit/risk profile associated with medical/surgical interventions and how this influences patient choice; barriers to the use of medical/surgical interventions across each region  and facilitators and barriers for application of the guidance.A systematic literature review was independently conducted by three Biographical Fellows in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement . The foc. As part of the screening, duplicates were removed. Exclusion criteria included: in-vitro/animal studies, non-English language studies, studies that did not report an outcome, diagnosis studies, non-systematic review articles and studies that did not relate to a specific intervention. Only studies conducted in humans were included and evidence was extrapolated from other MPS types. Single case studies were captured and referred to where evidence was scarce. The bibliographies of identified pre-existing guidelines and review articles were checked for additional relevant studies. The following information was extracted from included studies: reference details, patient population, intervention, comparator (where applicable), outcomes , safety and tolerability and reported study limitations. Data were extracted by one Bibliographic Fellow and checked by a second. Any discrepancies in screening and data extraction were resolved through discussion or the intervention of a third reviewer. Results were reported according to PRISMA  criteria. In accordance with the criteria  II Instrument (declared in the acknowledgements section of the manuscript) .The programme was funded by BioMarin; the manufacturer of the recombinant enzymes used to treat MPS IVA and MPS VI, however, they were not involved in any stages of the process and did not influence the scope or content of the programme. BioMarin were absent from all SC meetings, were blinded to the guidance statements and were not involved in the publication process. The programme was managed by an independent secretariat (Lucid Partners Ltd), and the scope of the programme and content, including the development of guidance statements, was led by the SC with editorial support provided by the secretariat. The SC was identified through a systematic expert mapping process, conducted independently of the funder. Conflicts of interests for all SC members (shown in the declarations section) were recorded at the start of the consensus programme and updated throughout the programme. Following the systematic expert mapping exercise, it was noted that some of the SC had previously worked on a consultancy basis with the programme sponsor, who hold the marketing authorisation for approved pharmaceutical therapy in MPS IVA. Efforts were therefore taken to ensure representation from leading experts across other treatment modalities, including HSCT/BMT on the Steering Committee panel and during the modified-Delphi voting process, where a large number of physicians across multiple specialities and geographies were engaged. At several stages during the process, the SC were also required to provide updated conflict of interest disclosures.Recommendations for the management of MPS VI: systematic evidence- and consensus-based guidance).Here we report the results relevant to the management of patients with MPS IVA. The results for MPS VI are published in a companion article . A total of 71 survey submissions were received from 53 institutions across 18 countries and two respondents did not meet the minimum experience threshold and their submissions were excluded , stakeholder involvement (85%), rigour of development (73%), clarity of presentation (78%), applicability (34%) and editorial independence (64%)   versus penetrating keratoplasty [PK]) should be made on a case-by-case basis. There is some evidence extrapolated from the general population to suggest that rejection is more likely to occur following PK than DALK [han DALK , 118, thConsiderations for intervention (e.g. if surgery) Monitored anaesthesia with appropriate sedation, including use of nasal CPAP/NIPPV, may be used when performing eye surgery in patients with MPS IVA. Long-term topical treatment is required following corneal transplantation, as is regular, long-term  ophthalmic assessment to determine the health of the corneal graft and to check for recurrence of corneal deposits and astigmatism control. Signs of rejection require prompt ophthalmic assessment to prevent graft failure. Follow-up is required to monitor for optic neuropathy arising due to raised intracranial pressure. Symptoms can include reduction in visual acuity, abnormal pupil reactions, new onset of visual field defects or optic nerve swelling .Rationale and evidence base Cardiac involvement is most commonly reported in patients with MPS I, II and VI [I and VI . HoweverI and VI . Valve dI and VI .Patient selection for intervention Performance and interpretation of echocardiography should be completed by physician’s familiar with expected pathological findings in patients with MPS IVA [ MPS IVA . Valve r MPS IVA , 121 in  MPS IVA  and shouConsideration for surgery Cardiac surgery in patients with MPS IVA should be performed in a centre of excellence with a team experienced in treating both MPS and high-risk valve replacement. When possible, an anaesthetic specialist should assist the cardiac anaesthesia team during pre-operative assessment to formulate the MPS-related anaesthesia care. Airway care, including the need for tracheostomy, should be assessed on a case-by-case basis. The anaesthesia principles of care (outlined in the anaesthetics recommendations) should be followed during the care of patients with MPS IVA for cardiac procedures.Rationale and evidence base ENT manifestations are common in patients with MPS IVA. These often involve hearing disorders, serous otitis media and deformities of the ossicles, which can have a significant impact on patient functional status and QoL [ and QoL . Mixed h and QoL . Evidenc and QoL , 125. Th and QoL . AdvanceConsiderations for surgery Vaccinations against respiratory pathogens causing influenza and pneumococcus infections are recommended to prevent pneumonia. Insertion of ventilation tubes should be performed according to guidelines for the general paediatric population [pulation . Before pulation . PatientRecommendations for the management of MPS VI: systematic evidence- and consensus-based guidance).MPS IVA and VI are rare, heterogeneous and progressive disorders that are associated with severe morbidity and reduced mortality. Although previously published guidance is available for the management of MPS IVA/VI , 130, thDeveloping guidance in rare diseases, such as MPS IVA/VI, is often challenging. The low patient numbers often mean that there is a paucity of large-scale randomised trials and systematic reviews available in the literature. As such there is a need for robust consensus-based guidance to bridge the gap between the relatively small number of clinicians with experience in treating patients with MPS IVA/VI, and the broader community of healthcare professionals involved in the management of these patients. However, with consideration of the challenges including lack of evidence and limited resources, guidelines remain critical for driving consistency and quality of care in rare diseases .www.agreetrust.org). Three independent reviewers assessed the manuscripts and suggested amendments were addressed where possible; a subsequent second round of review was conducted by all reviewers. The manuscripts were given an overall guideline assessment score of 5.3/7 .Management of MPS IVA/VI requires a coordinated multidisciplinary approach; therefore, it was important that this guidance covered a broad range of topics. However, as is common with rare diseases, high-quality evidence was not available for all the medical and surgical interventions covered in this guidance. As a result, it was necessary to draw on the expert clinical experience of the SC and broader modified-Delphi respondents to develop consensus-led guidance. Key strengths of this programme included the robust methodology used to identify and coordinate experts from a wide range of specialties and geographic spread, the systematic review of available literature and consensus on a broad range of guidance statements. This was achieved through a systematic approach to select the programme Co-Chairs and identify appropriately-qualified respondents for the modified-Delphi voting. Additionally, the inclusion of three PAG representatives as part of the SC group and interview analysis from six global PAGs, ensured that the patient perspective was reflected in the guidance. The comprehensive systematic literature review ensured that the resulting guidance was developed on the current evidence base and the use of a blinded modified-Delphi voting process to gain consensus ensured that the guidance reflected the views of a wide range of specialists with specific experience in managing the complex needs of patients with MPS IVA/VI. The methodological rigour and transparency employed within the development of the guidance was further demonstrated through the review of the manuscripts against the validated AGREE II instrument . Due to the limited funding available within the field of rare diseases, additional sponsorship for this programme was unavailable. However, the Steering Committee group encourages future multi-sponsored consensus programmes for updates to this guidance and in other MPS types/rare diseases.Access to relatively expensive therapies such as ERT, as well as access to other medical and surgical care can be challenging for exceptionally rare diseases. Patients’ access to treatment in rare diseases varies geographically based upon national reimbursement schemes. It is important that inclusion/exclusion criteria for treatment is not arbitrary and does not discriminate against slowly progressing patients, where a health gain cannot often be confirmed over a short period of time , 132. CoThe PAG consultations provided patient and caregiver insight into the barriers and facilitators to the application of this guidance across numerous healthcare systems and geographies, and the results were reviewed and further considered from a clinical perspective by the SC group. It was considered that patients and their families are likely to drive the dissemination and application of this guidance among healthcare professionals and that this can be supported by making the guidance widely available, for example, via the websites of MPS societies. To address knowledge barriers for healthcare professionals, the SC recommends that a number of materials should be developed including: case studies to support implementation of the guidance statements ; a guidance document to provide advice on specific questions that should be asked and data that should be collected at each clinic visit; and guidance and training on how to develop an emergency plan for patients as it is not always possible to get an MDT together quickly. The SC also recommended the development of a frequently asked questions (FAQ) document for use by PAGs and patient organisations. Following publication of the guidance, members from the SC group will partner with organisations such as PAGs and pharmaceutical companies to further understand the facilitators and barriers to the application of the guidance, including cost considerations. Subsequent support programmes will then be developed to deliver the recommended actions within the guidance.The SC highlighted the benefits that existing industry-sponsored MPS registries play in supporting data collection and discussed how these may provide important data on outcomes of medical and surgical interventions. One example is the elosulfase alfa NICE-managed access agreement in England, which aimed to collect data on the long-term efficacy and safety of elosulfase alfa as well as prevalence and effectiveness of other treatment paradigms . The SC Improvements in symptom management and the introduction of ERT for MPS IVA/VI may create new challenges for the future. As life expectancy increases, the challenge will be to manage a growing population of adult patients who may require multiple surgeries throughout their lifetime and therefore the associated anaesthetic risks will gain greater importance. Similarly, the introduction of newborn screening will facilitate early treatment and has the potential to change the natural history of MPS IVA/VI. MPS I has already been added by the Secretary of the Department of Health and Human Services to the list of routine recommended conditions to be screened in the USA, and the inclusion of other MPS types should follow as testing technology becomes more established and the benefits of early diagnosis and treatment become clearer. Biochemical enzyme assays continue to be important for differential diagnosis of MPS types and several assays for enzymes relevant to MPS IVA/VI have been described –138.N-acetylgalactosamine-6-sulfate (GALNS) has been developed for the differential diagnosis of MPS IVA; however, the data suggest this will not be robust enough for newborn screening [A novel urine assay has been developed that allows diagnosis of 10 subtypes of MPS in a single test by measuring oligosaccharides with non-reducing termini in urine samples to provide a diagnostic oligosaccharide signature unique for each MPS subtype. These disease-specific oligosaccharides can be used to monitor response to therapeutic interventions . A fluorcreening . Ongoingcreening . Furthercreening .Evidence suggests that although most patients develop anti-drug antibodies to elosulfase alfa, immunogenicity does not appear to impact efficacy or correlate with the frequency of IARs , 143. HoFuture therapies, including gene therapy, small molecule therapy, and other anti-inflammatory agents such as pentosan polysulphate (PPS)  are beinAcross all therapies, an individualised dosing approach would address one of the main challenges for the pharmacological treatment of patients with MPS. Treatment would be greatly enhanced with the definition of minimal clinically important differences for MPS and the development of improved tools and/or biomarkers for monitoring response to therapy. The development of MPS-specific patient-reported outcome measures is also needed, along with formal standardisation and validation of health-related QoL tools for the different MPS subtypes .Recommendations for the management of MPS VI: systematic evidence- and consensusbased guidance). The guidance is intended for use by healthcare professionals that manage the holistic care of patients with MPS with the intention to improve clinical- and patient-reported outcomes and enhance patient quality of life. It recognised that the guidance provided represents a point in time and further research is required to address current knowledge and evidence gaps. The SC recommends that this guidance is reviewed and updated within 5 years, or sooner if there are significant changes to medical practice.This manuscript provides robust evidence- and consensus-driven guidance that can be used by all healthcare professionals involved in the management of patients with MPS IVA; results for MPS VI are detailed in the companion article Additional file 2:: Tables detailing the evidence levels given to each reference supporting the MPS IVA guidance statements and the Evidence Grades applied to each guidance statement. Evidence levels were assessed using the Oxford Centre for Evidence-based Medicine and were based on the quality of evidence of each reference. For each guidance statement, an overall Evidence Grade was applied, based on the evidence levels of the supporting references. (DOCX 41 kb)Oxford CEBM grading for MPS IVAAdditional file 3:: Tables detailing the evidence levels given to each reference supporting the MPS VI guidance statements and the Evidence Grades applied to each guidance statement. Evidence levels were assessed using the Oxford Centre for Evidence-based Medicine and were based on the quality of evidence of each reference. For each guidance statement, an overall Evidence Grade was applied, based on the evidence levels of the supporting references. (DOCX 41 kb)Oxford CEBM grading for MPS VIAdditional file 4:: Full results of the first round of the modified-Delphi voting, which was used to demonstrate consensus of the guidance statements. (DOCX 152 KB) (DOCX 151 kb)Modified-Delphi voting Round 1Additional file 5:: Full results of the second round of the modified-Delphi voting, which was used to demonstrate consensus of the guidance statements. (DOCX 41 kb)Modified-Delphi voting Round 2


Background. The Zika virus (ZIKV) outbreak in the Americas has caused international concern due to neurological sequelae linked to the infection, such as microcephaly and Guillain-Barré syndrome (GBS). The World Health Organization stated that there is “sufficient evidence to conclude that Zika virus is a cause of congenital abnormalities and is a trigger of GBS”. This conclusion was based on a systematic review of the evidence published until 30.05.2016. Since then, the body of evidence has grown substantially, leading to this update of that systematic review with new evidence published from 30.05.2016 – 18.01.2017, update 1.Methods. We review evidence on the causal link between ZIKV infection and adverse congenital outcomes and the causal link between ZIKV infection and GBS or immune-mediated thrombocytopaenia purpura. We also describe the transition of the review into a living systematic review, a review that is continually updated.Results. Between 30.05.2016 and 18.01.2017, we identified 2413 publications, of which 101 publications were included. The evidence added in this update confirms the conclusion of a causal association between ZIKV and adverse congenital outcomes. New findings expand the evidence base in the dimensions of biological plausibility, strength of association, animal experiments and specificity. For GBS, the body of evidence has grown during the search period for update 1, but only for dimensions that were already populated in the previous version. There is still a limited understanding of the biological pathways that potentially cause the occurrence of autoimmune disease following ZIKV infection.Conclusions. This systematic review confirms previous conclusions that ZIKV is a cause of congenital abnormalities, including microcephaly, and is a trigger of GBS. The transition to living systematic review techniques and methodology provides a proof of concept for the use of these methods to synthesise evidence about an emerging pathogen such as ZIKV. WHO statement IHR 2005). During 2016, the number of countries affected by the ZIKV outbreak had grown from 33 countries (WHO situation report 05.02.2016) to 75 countries (WHO situation report 05.01.2017). By March 9, 2017, 31 countries had reported microcephaly or other congenital central nervous system (CNS) abnormalities potentially associated with ZIKV infection and 23 had reported an increase in the incidence of the immune-mediated condition Guillain-Barré syndrome (GBS) or laboratory confirmed ZIKV in persons with GBS (WHO situation report 10.03.2017). The causal association between ZIKV and adverse neurological outcomes has now been examined in many systematic and non-systematic reviews of research2. Case reports of other conditions in people with ZIKV infection, including immune-mediated idiopathic thrombocytopaenia purpura (ITP), have also been published6.Outbreaks of Zika virus (ZIKV) infection in the Americas have caused international concern owing to the severity of neurological sequelae linked to the infection , on a review of systematically identified studies up to May 30, 2016 and nonsystematically identified studies up to July 29, 20167. The review addressed specific questions about 10 dimensions of causal associations, based on the work of Bradford Hill8 and organised as a causality framework  based its assessment, that there is “sufficient evidence to conclude that Zika virus is a cause of congenital abnormalities and is a trigger of GBS”  congenital brain abnormalities, including microcephaly, in the foetuses and offspring of pregnant women and (b) GBS/ITP in any population. Second, we describe the transition of the review into a living systematic review.This article aims to fulfil two separate objectives. First, we update our systematic reviewPROSPERO protocol). The eligibility criteria, information sources and search strategy, study selection and data extraction are the same as reported in the protocol and in the previous publication7. In brief, the search covers PubMed, Embase and LILACS electronic databases; the Pan American Health Organization (PAHO), WHO, the Centers for Disease Control and Prevention (CDC) and the European Centre for Disease Prevention and Control (ECDC) websites; and several preprint databases . Search terms included ‘Zika virus’ and ‘ZIKV’ and corresponding MESH terms. Two reviewers screen and select articles for inclusion and extract data independently. We included publications that held information on at least one of the ten dimensions of the causality framework, regardless of the study design7. We gathered publications systematically from May 30, 2016 to January 18, 2017 for this update. We refer to the previous version of the review as the baseline review7 and to this current update as update 1. Reporting of the results follows the Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA) statement  in a cohort in Brazil31 and 6.6 (95% CI: 0.8-56.4) in a cohort in French Guiana40. In the Brazilian cohort31, the proportion of any adverse congenital outcomes among ZIKV infected women was high (41.9% [49/117]), compared with the uninfected group (5.2% [3/57]). In a prospective case- control study in Brazil, women with laboratory-confirmed ZIKV had 55.5 (95% CI: 8.6-infinity) times the odds of having a baby with microcephaly compared with women without evidence of ZIKV infection21. A retrospective case- control study in Hawaii found an odds ratio of 11.0 (95% CI: 0.8-147.9)41. In the latter, however, exposure was assessed retrospectively using serology.Exclusion of alternatives. We included 23 new studies in this update77. Many studies included in this review that reported on adverse outcomes of congenital ZIKV excluded TORCH infections77; exposure to toxic chemicals28 or genetic conditions42. Maternal or foetal malnutrition, hypoxic-ischaemic lesions and underlying genetic conditions were not excluded. No single alternative explanation could be given to explain the relation between ZIKV and adverse congenital outcomes.Cessation. We did not find any new publications for this causality dimension. Evidence is still lacking on the effect of intentional removal due to lack of vaccination or elimination of mosquitoes on a large scale.Dose-response. There is still no direct evidence about the association between Zika viral load and probability of adverse congenital outcome in observational studies, or of an association between symptomatic status and outcome. In a study in the United States, Honeinet al. found similar proportions of adverse congenital outcomes in symptomatic and asymptomatic ZIKV-infected mothers32.Animal experiments. This update of the review includes an additional 11 studies86. These studies confirm a consistent relation between a range of contemporary ZIKV and adverse congenital outcomes, including from Brazil85, Puerto Rico79 and Mexico81. The body of evidence coming from animal studies has grown; both in mice and macaques, congenital anomalies such as intra-uterine growth restriction and signs of microcephaly were observed after ZIKV infection85.Analogy. As for the baseline review, evidence for this dimension was not reviewed systematically because our search strategy did not include terms for other infections or conditions. Studies included in this version of the review confirm the analogy between congenital ZIKV and TORCH infections87. Vertical transmission of West Nile virus and dengue virus were summarised in the baseline review. In update 1, we included a case series from El Salvador that reported Chikungunya virus in 169 newborns of women with symptomatic infection; a minority had CNS infection, but microcephaly was not reported88. For most analogous pathogens, infections earlier in the pregnancy have a higher risk of adverse outcomes87.Specificity. We included one study89, suggesting an expansion of evidence of a distinct congenital Zika syndrome (CZS)89. In a review of 34 published reports, the authors suggest five congenital abnormalities that, in conjunction, comprise a pattern that is unique to ZIKV: severe microcephaly with overlapping cranial structures, subcortical location of brain calcifications, macular scarring and retinal mottling, congenital contractures and early pyramidal and extrapyramidal symptoms89.Consistency. The studies included in this version of the review confirm the pattern of consistency observed in the baseline review. ZIKV infection in association with adverse congenital outcomes were reported in a range of study designs from different regions (WHO situation report 05.01.2017), although the proportion of affected infants varies over geographic region and time. ZIKV exposure resulted in adverse congenital outcome in people living in ZIKV endemic areas91 and in female travellers who returned to non-endemic countries93. Direct evidence from epidemiological studies comparing different lineages is lacking due to circulation of a single strain.Conclusion. The evidence added in update 1 of the review confirms the conclusion of a causal association between ZIKV and adverse congenital outcomes. New findings expand the evidence base in the dimensions of biological plausibility, strength of association, animal experiments and specificity.In vitro andin vivo studies elucidate pathways on how these outcomes likely occur. Conclusive evidence on the strength of association is lacking. Studies provide crude overall measures of association, not taking into account potential co-factors.95 and 11 ZIKV-related cases of ITP6 were described in 18 studies.In the search period for update 1 of the review, an additional 154 cases of ZIKV-related GBSTemporality. We found an additional 17 publications that confirmed that ZIKV infection preceded the GBS or ITP at an individual level108 or at a population level111. ZIKV infections seems to be followed by GBS on average between 5 and 10 days. In one case series from Colombia103, the authors distinguished between rapid onset of GBS symptoms after ZIKV symptoms (para-infectious) and post-infectious onset, with an asymptomatic period after ZIKV symptoms before the start of GBS symptoms.Biological plausibility. We did not find any publications about the biological plausibility of ZIKV as a cause of GBS or ITP.Strength of association. We did not find any comparative observational studies during the search period for update 1. Several surveillance studies confirmed an increase in notified GBS cases during ZIKV outbreaks at the population level111. Rate ratios were significantly higher for Brazil, Colombia, the Dominican Republic, El Salvador, Honduras, Suriname and Venezuela when comparing pre-ZIKV GBS incidence and the incidence during the outbreak111; this ratio ranged from 2.0 (95% CI: 1.6-2.6) to 9.8 (95% CI: 7.6-12.5).Exclusion of alternatives. We included 11 publications112 that expanded the list of alternative causes for autoimmune disease that were excluded, such as infections, vaccines, other system illnesses and medication, drugs or other chemicals. Many GBS cases in these publications had serological evidence of previous exposure to DENV, as seen in the baseline review. It remains unclear how large the potential role of co-factors such as antibody dependent enhancement are.Cessation. We did not identify any publications with evidence about the effect of intentional removal/elimination/prevention of ZIKV on either GBS or ITP. An additional publication confirmed evidence that the natural removal of ZIKV resulted in a decrease in GBS cases in Brazil, Colombia, Dominican Republic, El Salvador, Honduras, Suriname and Venezuela111.Dose-response. We did not identify any publications about this dimension for either GBS or ITP.Animal experiments. No additional evidence from animal experiments was identified that support the association between ZIKV infection and GBS/ITP development.Analogy. As for the baseline review, evidence for this dimension was not reviewed systematically because our search strategy did not include terms for other infections or conditions. We did not identify any new publications addressing this dimension for either GBS or ITP.Specificity. We did not identify any new publications addressing this dimension for either GBS or ITP.Consistency. Studies included in update 1 confirmed the consistency of the evidence for 3 of 4 questions about the association between ZIKV and GBS. By geographical region, ZIKV transmission has been associated with the occurrence of GBS in 2 of 4 regions; increased GBS incidence has been reported in the WHO regions of the Americas and the Western Pacific region, but not in the African or Southeast Asian region, despite recent ZIKV circulation113. By study design, the association between ZIKV infection and GBS has been found at individual and population level and with different study designs. By population, ZIKV infection has been linked to GBS in ZIKV endemic regions114 and travellers from non-affected countries who were exposed in these endemic regions112. There was insufficient evidence to examine the consistency of evidence about ZIKV and ITP.Conclusion. The body of evidence has grown during the search period for update 1 but only for dimensions that were already populated in the original publication for GBS. There is still a limited understanding of the biological pathways that potentially cause the occurrence of autoimmune disease following ZIKV infection. Additionally, prospective comparative epidemiological studies are still lacking. It remains unclear how co-factors such as age and previous exposure to flaviviruses influences the risk of developing GBS. The evidence supports a temporal association between ZIKV and ITP but there is an absence of evidence for other dimensions of causality.Automated search and deduplication processes identified 2410 publications about any aspect of ZIKV infection. The next update of this review will address causality dimensions in the realm of epidemiological studies; strength of association, dose-response relationship, specificity and consistency.Statement of principal findings. This systematic review confirms evidence of a causal association between ZIKV and adverse congenital outcomes and between ZIKV and GBS, although evidence about biological plausibility is still lacking. We assessed evidence about an association between ZIKV and ITP but found that this only addressed the dimension of temporality. The review is transitioning from classic systematic review methods to those of a living systematic review.Strengths and limitations of the study. The strengths of this study are the systematic approach to the identification, selection and extraction of data following a causality framework that provides a structure for the consideration of heterogeneous sources of evidence and a large set of review questions. Automation of the review output allows rapid updating of tables of results. We have also developed methods to automate search and deduplication of search results to make the transition to a living systematic review that will allow continual updating of results. The main limitation of the classic systematic review of such a complex topic is the high workload and time required to maintain it. Another limitation, resulting from the large number of review questions, is the time taken to resolve inter-reviewer differences in interpretation of eligibility criteria. This could have resulted in subjectivity over decisions about inclusion in the review. Although a second reviewer checked all extractions, changes in the review team could introduce inconsistency. As in the baseline review, we used case definitions as authors described them in individual publications. This potential source of information bias is likely to decrease over time as standardised case definitions and protocols are adopted115. As in the previous version, we did not systematically apply quality assessment tools to individual studies. Because much of the technical infrastructure was built as the evidence emerged, output was delayed. As much of the LSR methodology was novel, it took time to find a balance between speed and efficiency.Strengths and weaknesses in relation to other publications. Our systematic review differs from most standard reviews because of the number of questions within the dimensions of the causality framework and the number of outcomes. Other recent examples of living systematic reviews only distinguish between two study types (RCT and non-RCT)116 and are guided by only a small set of review questions118. Our review conclusion, confirming evidence for a causal association between ZIKV and GBS differs from that of a review119 of the findings of four case reports122 and one case-control study123. The authors found insufficient evidence to confirm the presence of an acute motor axonal neuropathy variant of GBS. They did not, however, suggest an alternative explanation for the increase in incidence of GBS in the countries that experienced ZIKV outbreaks. The two versions of our review included 64 publications about ZIKV and GBS across ten dimensions of causality.Meaning of the study: possible mechanisms and implications for basic researchers, clinicians or policymakers. The conclusions on the causal relation between ZIKV and adverse congenital outcomes and ZIKV and GBS did not change with this update. We found insufficient evidence about the association between ZIKV and ITP to state with certainty that there is a causal association. The total volume of evidence about the association between ZIKV and GBS is less than for the association with adverse congenital outcomes. There is, in particular a lack of published research to elucidate biological mechanisms for direct neuronal or autoimmune damage in GBS124. The descriptive data about the numbers and types of different studies over time illustrates how evidence about a new, or re-emerging, infection emerges over time. The evidence from many regions that were affected by the ZIKV outbreak remains limited to anecdotal evidence of adverse outcomes, in the form of case reports or case series. The slowing of ZIKV transmission in 2017 means that fewer people are being affected by ZIKV and its complications and fewer people are being enrolled into prospective studies. Further progress in epidemiological research will rely more heavily on research consortia who are contributing to joint analyses of data from existing studies.Unanswered questions and future research. As the volume and complexity of the evidence in different causality dimensions accumulates, the need for expert input and interpretation of the findings of this systematic review increases. The focus of research on ZIKV and causal associations with different types of adverse outcomes is also changing. For congenital abnormalities resulting from ZIKV vertical transmission, epidemiological research should examine CZS in comparative studies, quantify the strength of association with ZIKV, clarify associations with gestational age, symptomatology and viral load and further investigate potential co-factors such as previous dengue infection and flavivirus vaccination. WHO standardised study protocols provide suggestions for exclusion of alternative explanations and exploration of co-factors (Harmonization of ZIKV Research Protocols). For GBS, epidemiological studies are needed to quantify the association with ZIKV more precisely, but also to determine whether there are distinct phenotypes resulting from autoimmune mechanisms or direct neuronal involvement. For ITP, additional evidence across all causality dimensions is needed.Planned updates of a living systematic review. Living systematic review methodology and techniques will continue to develop. Since a chain is only as strong as its weakest link, any processing step has the potential to slow down a living systematic review. Clearly defined protocols that define update frequencies and throughput speed of different actors in the publishing process are vital. The next update of the systematic reviews will use living systematic review methods to assess the evidence for 2017 and early 2018  on a such highly relevant topic, as Zika association with congenital brain abnormalities and Guillain-Barré syndrome. However, I should suggest first, to include other Zika congenital associated abnormalities, as we have the congenital Zika syndrome (CZS) but also extra-Central Nervous System (CNS) abnormalities. The idea to have a living SR is excellent, however very compromising. Right now, this first version, requires urgently to be updated. As the date of update was till January 2017, as a first update.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. This is a systematic review looking at the strength of evidence supporting the link between Zika virus (ZIKV) infection and neurological complications. The intention is to create a living review that is constantly updated as new publications arise, allowing researchers access to a data resource that keeps pace with the rapidly advancing evidence in the field. This is an extremely important initiative if it can provide truly up to date (i.e. as fast as a pubmed or similar search) and appropriately sifted information for those wishing to conduct spot checks on the current status of neuro-ZIKV. From an aspirational perspective I support it strongly, but does it work in practice? Setting aside many  complex methodological considerations that this article raises, how useful is such a resource and who is it there to serve? We would all welcome tailor-made and up-to-date information at our fingertips to help navigate the mountain and minefield of information on ZIKV. I suspect that whilst this review will be widely consulted by those interested in neuro-ZIKV, from policy-makers to researchers, we will still refer back to primary publication for detailed up-to-date information in our specialist areas. At least it will provide a generic overview that facilitates more in depth access to and analysis of the primary data.I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard. This systematic review assessed the causality of Zika virus in the development of congenital brain abnormalities and Guillain-Barre syndrome. To accomplish this, the 10 dimensions of casual associations were reviewed, and evidence obtained for each dimension were narratively reviewed. With the addition of 101 new studies, the authors were able to conclusively establish the causal association between Zika and adverse congenital outcomes. However, not all dimensions could be assessed concerning Guillain-Barre syndrome, even with the additional of new studies within this update. As a result, there is limited understanding within this field of research for Zika virus. Thank you for allowing me to review this article. Overall it was well written, and did a good job addressing all aspects of it's complex objective. Please find below my comments:I would have liked to see inclusion/exclusion criteria for the study (something that could easily be in a figure or table to save room).Please include your search strategy as a supplementary file (just Embase would be sufficient). It sounds like a very simple search the way you've described it in the methods. Also would have liked more information about who designed the search  and if it was peer reviewed using PRESS. Finally, in the "analogy" section of the results you indicate this wasn't reviewed in the baseline review as the search strategy didn't include terms for it. This indicates to me that you must have updated your search between the baseline review and the update, but you don't say this in your methods.There is no assessment for methodological quality, which you do address in your limitations. I think this is an important step when you're including different study designs . Probably not appropriate to give the same weight to a case report as a cross-sectional study, or an animal experiment and an epidemiological study.   For the methods I would have liked to see more information. This review should be able to stand on it's own without the reader having to find the baseline review to find important methodological information:I find Table 1 a bit difficult to read, maybe because instinctively I'd expect the column for "U1, N" to be the total number of studies in the update, rather than just the new studies added with this update. This only becomes obvious when you look at rows like "case series" and you see the U1 has less studies than BR. If you keep this structure, it would be nice to add another column that has total N . General: Living Systematic Review:YES, it meets all criteria for doing a living systematic review  Do the authors provide a clear rationale for the living systematic review?  Partly - While they detail the process of U1-U3, the time for search, between updates, etc. changes with each update (as they anticipate the technology getting better). Living systematic reviews require explicit decisions on how often searching, data extraction, analysis, etc. will be performed.Do the authors clearly state how this will be maintained as a living systematic review?   I feel this review is premature in saying that it is transitioned to a living systematic review. The intention of the living systematic review is to provide the reader with up to date evidence so the lag between the search and publication is minimize. The authors even define a living systematic review as "a systematic review that is continually updated, incorporating relevant new evidence as it becomes available". The search for this study ended in January 2017, and technically is out of date even for a regular systematic review (which we try to publish within a year). The authors say that they plan to update the review twice per year with formal peer review updates, but if that was the case then another 2 reviews should have been done since this date. Additionally, they authors indicate that only module 1 and part of module 3 of their living systematic review automation were in place by December 2017, meaning this wasn't used for this update. Figure 2 already looks to be out of date - in it they propose that U2 would be published by Feb 2018. If they were able to accomplish this then U2 might have been more appropriate to call a living systematic review .   I think it would have been more appropriate to publish this as an update of the systematic review, publish a protocol for how they were going to transition to a living systematic review , and then treat U2 as their 'baseline" living systematic review, at which point the "living" aspect of the review is turned on and updates are given every 6 months (as they proposed).  Do the authors provide a clear rationale for the Living Systematic Review? YesDo the authors clearly state how this will be maintained as a Living Systematic Review? PartlyI have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.


N = 139) showing significantly improved progression-free survival (PFS) versus temsirolimus (N = 141)  ) than in the temsirolimus group (39 [28%]) discontinued treatment due to adverse events (AEs); eight patients in each arm discontinued due to death. Other reasons for discontinuation included refusing further treatment. Median duration of exposure was longer for ibrutinib than temsirolimus , as in the primary analysis.P < 0.0001); consistent with the results of the primary analysis , c factor , with mu


To examine the quality of the evidence relied upon by the World Health Organisation (WHO) in promoting hand hygiene with campaigns such as “Save Lives: Clean Your Hands”.The quality of evidence in the studies quoted by the WHO evidence document is highly variable and the methods used limited. In some of the quoted studies, hand hygiene was the primary outcome, rather than the clinically significant outcome of hospital acquired infection (HAI). When HAI was the primary outcome, it was often poorly defined and reported with scant detail. There was wide variation in the hand hygiene compliance achieved in the intervention studies. The majority of studies where the intervention was a campaign to promote hand hygiene used historical control data with variable attempts to account for the fact that HAI rates may have been declining prior to the hand hygiene intervention. The results from trials with a contemporaneous control were conflicting. There is little evidence that past quality and safety initiatives have improved health outcomes  and therOver recent years a worldwide effort has been directed to promote hand hygiene in hospitals, with the aim of preventing hospital acquired infection (HAI). Agencies with responsibility for improving the quality and safety of healthcare make strong claims for the benefits of hand hygiene and have prioritised public investment in pursuit of this objective. The World Health Organisation (WHO) has been particularly proactive in promoting hand hygiene with campaigns such as “Save Lives: Clean Your Hands”. Our aim in this article is to examine the strength of evidence relied upon by the WHO for the effectiveness of hand hygiene campaigns.The oft-repeated assertion, hand hygiene is the single-most effective intervention to reduce hospital acquired infections, is based on several layers of evidence. There is good evidence that microorganisms responsible for HAI can colonise health workers’ hands and that hand hygiene practices can reduce this colonisation. However, it does not necessarily follow that campaigns to promote hand hygiene are an effective intervention. It may be that adequate compliance with hand hygiene cannot be reliably incorporated into routine hospital practice. Also, even if campaigns to promote hand hygiene do lead to a sustained reduction in colonisation of health workers’ hands, parallel transmission of microorganisms by other routes may mean that there is no reduction in rates of HAI.We are unable to find any published formal systematic review of the effectiveness of hand hygiene for preventing HAI. There is a Cochrane review of compliance with hand hygiene, which found insufficient evidence to conclude that interventions can effectively improve compliance , but theIn contrast to the conclusions quoted above, the WHO guidelines on hand hygiene in healthcare advocate the implementation of hand hygiene programs and rate the evidence for hand hygiene before and after touching the patient as “strongly recommended for implementation and supported by some experimental, clinical, or epidemiological studies and a strong theoretical rationale” .Evidence of hand hygiene to reduce transmission and infections by multidrug resistant organisms in health-care settings [Subsequent to promulgation of their guidelines, the WHO published a document entitled, settings . The autThe WHO evidence document claims to have identified 39 suitable papers. However, it cites just 26 references, only 19 of which are papers with original data relevant to the question. A table in the document summarises eleven studies deemed relevant and higher quality, although no information was provided regarding the criteria used to assess quality.In our judgement, the quality of evidence in the studies quoted by the WHO evidence document is highly variable. The primary outcome in many of the studies was hand hygiene compliance, with HAI a secondary outcome that was often poorly defined and reported with scant detail. The majority of intervention studies where the intervention was a campaign to promote hand hygiene used historical control data with variable attempts to account for the fact that HAI rates may have been declining prior to the hand hygiene intervention. There was also wide variation in the hand hygiene compliance achieved in the intervention studies.The majority of the 19 studies quoted by the WHO evidence document that reported original data did not include a contemporaneous control. Of the studies using historical controls, ten studies compared rates of HAI before and after introduction of a hand hygiene initiative in individual wards or hospitals. Changes in rates of HAI were mixed with some studies reporting a decrease , some reporting an increase , and others reporting little or no change . Two studies attempted to analyse the effectiveness of nation-wide hand hygiene initiatives. A time-series analysis from the United Kingdom reported evidence of a positive impact with reduced rates of HAI associated with various stages of a national hand hygiene campaign (reference 17 in the document). The other nation-wide study documented decreasing rates of HAI in Australia after a national hand hygiene initiative and the authors interpreted this as evidence for effectiveness (reference 10 in the document). However, their method for comparing rates of HAI before and after the hand hygiene initiative is open to question and this study could be interpreted as failing to identify any beneficial impact because HAI rates were already falling prior to the intervention see “”.One group of studies quoted in the WHO evidence document did not directly assess the effectiveness of a hand hygiene intervention. Rather, their approach was to analyse correlations between use of hand hygiene products and rates of HAI. These studies reported decreasing rates of HAI over time, which were positively correlated with increasing usage of hand hygiene products . Correlations between rates of HAI and consumption of hygiene products is very low-level evidence for causation.Only three of the studies quoted in the WHO evidence document included a contemporaneous control and these failed to demonstrate a consistent benefit. One cluster randomised controlled study reported no benefit following a hand hygiene initiative (reference 8 in the document). Another cluster randomised trial conducted across several long-term care facilities  did not report the actual rates of infection but claimed a reduced incidence of infection in the intervention facilities with borderline statistical significance (reference 12 in the document). In the third controlled study, three hospitals introduced a hand hygiene initiative and were prospectively compared to a control hospital (references 14 in the document). There was a decrease in HAI in one intervention hospital but not in the other two, and no information was provided regarding the control hospital.In summary, the studies quoted in the WHO evidence document provide weak evidence. We do not agree that the references cited in the WHO evidence document justifies the authors’ conclusion that the great majority of papers offer convincing evidence. Given the weak methodology of most of the quoted studies, and their inconsistent results, a contrary hypothesis—that campaigns promoting hand hygiene are ineffective in preventing HAI—is plausible.Since the promulgation of the WHO campaign, costly efforts have been made to promote hand hygiene. We find it concerning that such a campaign was initiated on what appears to be weak evidence, and it is to be hoped the evidence base can be improved by future studies. It may be difficult and prohibitively expensive to conduct large enough, well designed, prospective trials to address these questions. Hand hygiene is already widely practiced so that a part of any effect may already have been “used up”, potentially reducing the expected effect size, and increasing the necessary sample size, in any future studies. Other difficulties faced by any attempt at a future definitive study include the need to manage intraclass correlation within participating centres. Furthermore, it may be difficult to include a comparison with interventions which seek to reduce the practice of hand hygiene because of its widely perceived benefit. A more feasible approach could be to conduct a robust systematic review of existing material, and consider a meta-analysis if the results of this review make this possible.Hand hygiene promotion may eventually be found to be an ineffective public health intervention. Public healthcare agencies should lead the synthesis of evidence supporting improvements in clinically important areas. Support for initiatives by public healthcare agencies in the absence of adequate evidence may result in suboptimal allocation of limited health resources and undermine confidence in those agencies. Misdirection of public investment to ineffective interventions may in part account for the observed failure of quality improvement initiatives to deliver improved health outcomes.Following the implementation of a nation-wide hand hygiene initiative in Australia, Grayson et al  publishe