The Australian National COVID-19 Clinical Evidence Taskforce is producing living, evidence-based, national guidelines for treatment of people with COVID-19 which are updated each week. To continually improve the process and outputs of the Taskforce, and inform future living guideline development, we undertook a concurrent process evaluation examining Taskforce activities and experience of team members and stakeholders during the first 5 months of the project.The mixed-methods process evaluation consisted of activity and progress audits, an online survey of all Taskforce participants; and semi-structured interviews with key contributors. Data were collected through five, prospective 4-weekly timepoints (beginning first week of May 2020) and three, fortnightly retrospective timepoints . We collected and analysed quantitative and qualitative data.An updated version of the guidelines was successfully published every week during the process evaluation. The Taskforce formed in March 2020, with a nominal start date of March 23. The first version of the guideline was published two weeks later and included 10 recommendations. By August 24, in the final round of the process evaluation, the team of 11 staff, working with seven guideline panels and over 200 health decision-makers, had developed 66 recommendations addressing 58 topics. The Taskforce website had received over 200,000 page views. Satisfaction with the work of the Taskforce remained very high (>90% extremely or somewhat satisfied) throughout. Several key strengths, challenges and methods questions for the work of the Taskforce were identified.In just over 5 months of activity, the National COVID-19 Clinical Evidence Taskforce published 20 weekly updates to the evidence-based national treatment guidelines for COVID-19. This process evaluation identified several factors that enabled this achievement (e.g. an extant skill base in evidence review and convening), along with challenges that needed to be overcome  and methods questions  which may be useful considerations for other living guidelines projects. An impact evaluation is also being conducted separately to examine awareness, acceptance and use of the guidelines. Living guidelines are a new approach to developing and maintaining rigorous evidence-based guidelines in which any new evidence is rapidly incorporated, ensuring that recommendations are continually up to date with the latest research . Living A small but increasing number of living guideline projects are underway, including in stroke, diabetes, maternal health and the recent living WHO guideline on drugs for COVID-19 –5, howevThe Australian National COVID-19 Clinical Evidence Taskforce is a consortium of 32 Australian health professional organisations representing the full range of health professionals providing care to Australians with COVID-19, co-funded by Australian national and state governments and philanthropic organisations. The Taskforce is developing living guidelines for care of people with suspected or confirmed COVID-19 which are updated each week to reflect new evidence. The structure and methods used by the Taskforce to develop the guidelines have been described previously . In brieGiven the novel approach, unique opportunity to study rapid and living guideline development in new disease and the need to continuously improve processes in this rapidly evolving guideline, we undertook a process evaluation exploring the activity and experience of participants in the Taskforce for the first 5 months of the living guidelines project. The aim of the process evaluation was two-fold: to enable us to improve process and outputs of the Taskforce as guideline development was underway, and to identify factors that might be useful to inform design and development of future living guidelines, by capturing the activities and experiences of Taskforce contributors each month during the living guidelines project.This process evaluation utilised a mixed methods approach consisting of activity and progress audits, online surveys, and semi-structured interviews. A protocol was developed for the process evaluation by TT. The protocol was refined in discussion with the Taskforce Executive team and approved by the Taskforce Steering Committee. Ethics approval was provided by Monash University Human Research Ethics Committee (Project ID: 24536).https://bridges.monash.edu/articles/dataset/National_COVID19_Clinical_Evidence_Taskforce_Process_Evaluation_Data/16926868).Data were collected through five, prospective 4-weekly timepoints (beginning first week of May) and three, fortnightly retrospective timepoints . At each 4-weekly prospective timepoint we conducted an online survey of all participants in the Taskforce, as well as semi-structured interviews with key contributors and an audit of activity and progress. We collected and analysed both quantitative and qualitative data, taking a pragmatic approach focusing on rapidly informing Taskforce decision-making. For retrospective timepoints, we collected and analysed quantitative activity data only. The data underlying the results presented in this study are available on the Monash University Data Repository–BRIDGES . The survey was open for a week in each cycle. Participation was anonymous and voluntary, and completion of the online survey after reading the explanatory statement was considered implied consent.Both quantitative and qualitative data were collected using an online survey tool . Participants were asked to describe:Overall level of satisfaction with the work of the National COVID-19 Clinical Evidence Taskforce What is working well? (Free text)What could be improved? (Free text)With the benefit of hindsight, what would you do differently next time? (Free text)Quantitative data was analysed using simple descriptive statistics, qualitative (free text) data were analysed in combination with the interview data.Interview participants included the Taskforce Executive Team, and purposively selected individuals with other roles in the Taskforce including clinical content experts and members of the evidence synthesis team. Taskforce members have a diverse range of roles. The Executive Team is comprised of clinicians, methodology experts, researchers, and the Taskforce communications and engagement manager, evidence and methodology manager, business manager and administration staff. Evidence team members largely have health science backgrounds and experience in evidence synthesis methodologies. Participants were invited to participate via direct email which included the explanatory statement. Participation was voluntary and agreeing to conduct the interview was considered consent. Five to ten semi-structured interviews were conducted with key members of the Executive Team and different Taskforce members during each of the five, four-weekly data collection cycles. Interviews were conducted via Zoom and were audio-recorded. Detailed notes were also taken. Interview questions were based on a predetermined interview schedule, with questions varied to suit the interviewee’s roles and experience. The interview schedule was developed by one author (TT) in consultation with the other authors.Data were collected on:Satisfaction with progress, what was working well, which areas needed improvementChallenges, barriers, facilitators and enablers encounteredWhat they might do differently with the benefit of hindsightExpectations for the next monthWhat one thing would make the greatest difference to the work of the TaskforceInterviews were conducted by one author (TT) who has extensive experience in qualitative interviewing, and data were de-identified. TT was known to the interviewees, some of whom are long-standing colleagues, and some of whom have been introduced during the work of the Taskforce. None of the participants were managed by or in a reporting relationship with TT. Each month, the interview data were combined with qualitative data from the survey, and were analysed using open coding to identify key concepts which were organised into emerging themes. TT undertook the primary data analysis, SG collaborated on the conceptual development and refining of themes. For this analysis, we primarily reviewed the reports to identify themes across time, but also revisited the primary data as necessary to confirm interpretation and review coding.Every 4 weeks, results from the activity audit, survey and interviews were analysed and a brief report prepared for the Taskforce Steering Committee. Data were also used contemporaneously by the Taskforce Executive to refine Taskforce processes and outputs.The Taskforce formed in late March 2020, with a nominal start date of March 23. The first version of the guideline was published two weeks later and included 10, initially primarily consensus, recommendations. Using the rigorous, evidence-based methods described in detail elsewhere , the guiLevels of activity were very high in the first month of Taskforce. The focus was on securing funding, rapidly establishing the team, convening three guideline panels covering mild, moderate-severe and critical COVID-19, engaging 20 partner organisations and establishing fundamental governance structures.In May and June, the Taskforce built on this initial base to establish clear operational workflows and processes, expanded to include six guideline panels ; further developed engagement with jurisdictional decision makers  and strengthened the communications function.By August 24, in the final round of the process evaluation, five months after establishment, the team consisted of 11 full-time equivalent staff, working with seven guideline panels and over 200 contributing individuals .The evidence identified had informed development of 66 recommendations (version 18). The Taskforce website had received over 200,000 total pageviews and 116,383 unique visits.The first survey was conducted from on May 4–11, seven weeks after the start of the Taskforce. The final survey was conducted from August 24–31, 23 weeks after the start of the Taskforce. The number of people invited to participate in the survey grew from 119 in May, to 207 in August; the response rate declined after the first two rounds, and as the number of individuals contributing to the Taskforce grew. Satisfaction with the work of the Taskforce remained very high (>85% extremely or somewhat satisfied) throughout see .A total of 38 interviews were conducted during the process evaluation, five in the first cycle, seven in the second, eight in the third, ten in the fourth and eight in the final cycle. Eight participants were interviewed multiple times (range: 2–5) and the other six participants were interviewed once. Interviewees included 7 members of the Executive Team, 6 members of the evidence synthesis team and a clinical content expert. Data from the interviews were combined with qualitative data collected through the online survey.An updated version of the guidelines was successfully published every week during the process evaluation. This consistent delivery of the core output of the Taskforce led to pride and delight in the achievement of the Taskforce. Taskforce contributors were “astonished”, “amazed” and “impressed” by what was achieved and reported that they “feel privileged” to have been involved. “It’s amazing what we’ve achieved.”Contributors repeatedly emphasised that a core strength of the Taskforce was that the team undertaking the evidence surveillance and living reviews were highly skilled, collegial and committed—“Trust and team work are extraordinary”. The core of the team was drawn from existing Cochrane Australia staff, rapidly bolstered by additional recruitment  during the start-up phase. Both having an initial team in place from which to build, and rapid recruitment of experienced individuals to expand this team, were major contributors to success.In parallel with the expertise in evidence and guideline development methods, respondents regularly highlighted the extremely high levels of collaboration between Taskforce member organisations, and continuing high levels of engagement and contribution from individual clinicians. “The enthusiasm of the individual members is outstanding.”While the early days of the Taskforce were retrospectively described as “frantic”, clear weekly workflows and processes were quickly established and continually refined. By May, respondents were reporting that the acute establishment phase had passed, and the Taskforce was developing a rhythm and structure; by late July team members reported that operational and evidence processes were running smoothly and efficiently.Managing workload for the team was an ongoing challenge. Workloads during the establishment period were extremely high. While this settled a little as workflows were established and team members developed a “much clearer sense of what [they are] here to do”, for some topic areas such as drug treatments this was offset by an increasing rate of study publication. In the last round of the process evaluation there was recognition that “[we] still have people working ‘til midnight”.The context of the COVID-19 pandemic created additional pressure and stress on the team and other Taskforce contributors, as many faced personal challenges, including increased clinical workloads; and most staff, based in Victoria, Australia, were in lockdown for many months, leading to mental and emotional exhaustion. “[W]e are all in a state of fatigue”. The context also meant it was not feasible for the team to meet face-to-face, and remote working was an additional challenge for communication and efficiency, particularly for new staff without existing relationships with Taskforce colleagues.The structure of the team presented a challenge as the project progressed. In order to rapidly expand the team, many staff were seconded to the Taskforce for a finite period (most often 3–6 months), and although Taskforce funding was continued beyond this point, some had to return to their existing primary roles, creating skill gaps. Similarly, early expedient decisions to contract external providers for functions such as communication introduced challenges at later stages of the process.Supporting guideline development panels was an ongoing challenge for the Taskforce. These panels of clinicians each met weekly to consider the evidence provided by the evidence team, and make or update the guideline recommendations. Providing timely, structured, concise but detailed pre- and post-meeting documentation was a significant undertaking. Discussions about the best size for these panels , and the scope of the portfolios of work addressed by each panel, continued throughout the period of the process evaluation. Some of the panels functioned effectively throughout, whereas others had periods of inefficiency and lack of clarity about contribution to the Taskforce. Highly skilled clinical chairs and methods co-chairs, were a vital contributor to the success of these panels.Building on the initial success of the Taskforce, it was proposed the Taskforce expand scope to include infection prevention and control in healthcare settings. Those in favour of expansion saw an opportunity to address issues of vital concern to the guideline audience, however there were also concerns about whether expanding scope would put at risk the delivery of the primary output of the Taskforce, Australian living recommendations for the clinical care of people with COVID-19. These conversations continued throughout the process evaluation, and in the final round views were still mixed: “Exciting opportunities… logical extension”, “a bit scary in terms of resourcing”.One of the components of the work of the Taskforce is to further build the technology platform to support living guideline production and publication. In the early phases of the work this was felt to be less of an issue, with respondents in June emphasising that “The limit is not tech, it’s humans”. However, the importance of the planned technological improvements was increasingly felt, and by the late August five months into the work, team members were increasingly aware of and frustrated by the tasks that they were doing manually, or things they were unable to do efficiently, because they were beyond the limits of available technology. “[The] tech stuff [is] still really challenging.”One contributor to the delays in delivery of the technological development was siting of the Taskforce within the Monash University governance structure. There was recognition that Monash’s “admin set-up isn’t as agile as we are”. Location of the Taskforce within a university had important benefits including the provision of core financial and human resources infrastructure and governance oversight enabling rapid establishment, but also introduced significant challenges in the form of additional layers of bureaucracy and delays, especially in contracting. These challenges were exacerbated by the university-wide impacts of the COVID-19 pandemic, both on financial management, and increased administrative workload.The all-pervasive uncertainty in which the work of the Taskforce was conducted, both in terms of the trajectory and impact of COVID-19 in Australia and globally, and also questions about the future funding and direction of the work of the Taskforce, became less comfortable for contributors over time. In the initial stages, contributors reported that they felt “privileged” to be involved and were altruistically motivated to contribute on an ad hoc basis. By July the Taskforce had reached “a tipping point” that “comes with maturity” where “people’s expectations are different” and there was increasing desire for a clear roadmap that described the future path of the Taskforce and expectations of ongoing contribution. There was also an increased desire for demonstration that the work of the Taskforce was having a meaningful impact. “I don’t have a good sense of whether we are really ‘landing’ with the target audience” “Are people actually using them?”.Two key issues about the living evidence synthesis methods employed in the guidelines were raised repeatedly throughout the process evaluation; the pace of guideline development, and the thresholds for inclusion of evidence.From the beginning, the Taskforce committed to deliver weekly updates of the guidelines, a pace which was described as “superb”, “relentless” and “onerous”. To meet this aim, a weekly schedule was developed which included meetings of each of the guideline panels, as well as the Guideline Leadership Group to review recommendations, and the Steering Committee to approve recommendations. By June, some respondents were questioning whether weekly meetings were necessary and feasible for all guideline panels, or whether some panels might meet fortnightly or monthly without compromising the value of the guidelines, particularly in areas where less research was being published, or fewer new clinical questions raised. “[The] expectation that every week there will be something new is unrealistic.” Contributors had mixed feelings, with a sense that less frequent meetings were in some way devaluing their contribution, but also frustration if meetings didn’t seem to lead to concrete progress in the form of new recommendations. At the end of the process evaluation this question was still under discussion.Given the context of a novel disease, with little prior research, initially all evidence was valuable to inform recommendations, and every new study was immediately appraised and incorporated. As the availability of research increased, a question arose about the value of the work required to rapidly incorporate results from small or low-quality studies into evidence summaries when it was clear that these would not impact on current recommendations. Some respondents suggested that an option was needed for a “holding bay”, or that thresholds should be set for incorporation of new evidence, perhaps similar to those used in living systematic reviews. Similarly, methods needed to be developed to deal with increasing numbers of studies available in preprint version.Many of the key strengths and challenges identified through this process evaluation have application/relevance to other living guidelines see . RegularStrengths*Weekly updatesHighly skilled clinical chairs, methods co-chairs and evidence teamCollaboration between member organisationsHigh level of engagement of large numbers of clinicians*Clear, weekly workflows and processesUniversity governance–rapid establishment, core human and financial resource infrastructure, governance oversightChallenges*Workload management*Significant external, additional stressors on staff Remote working*Rapid staff procurementSupporting the guideline development panels–ideal size, scope of portfolio of work to be addressedInefficiencies with technology*University governance–contracting delays*Uncertainty–trajectory/impact of the pandemic and the guidelinesDetermining appropriate update schedule*Determining an appropriate, and potentially evolving threshold for inclusion of evidence*Determining what to do with preprint studies* Unique strengths/challenges during a pandemic.Supporting guideline panels is likely to be a consistent challenge across living guidelines however the cumulative effect of supporting numerous members across several panels operating on a weekly update cycle is unique to the experience of the Taskforce. Inefficiencies with technology and determining an appropriate update schedule are challenges likely to be experienced across other living guidelines. Further consideration is required to provide clear recommendations for these areas.At the conclusion of the process evaluation, as of September 10, the living, evidence-based Australian guidelines for the clinical care of people with COVID-19 had been published and updated 20 times in just over 5 months, and included more than 70 recommendations, a significant achievement for one of the world’s first living guidelines, and certainly the most rapidly updated living guideline to date. Conducting a process evaluation alongside the work of the Taskforce was an important opportunity not only to improve Taskforce processes and outputs as the project progressed, but also to reflect on what future living guidelines and evidence synthesis projects might learn from this experience.The work of the Taskforce was made possible by the existence of the team at Cochrane Australia with skills in evidence synthesis and guideline development methods, and the Australian Living Evidence Consortium which has been pioneering and piloting methods and platforms for living evidence synthesis . Having The second strand of expertise that enabled the Taskforce was convening power. Guideline development relies on the integration of both evidence, consumer and clinical expertise, and the ability to engage and support over 200 contributors and manage their diverse, expert input each week was an important contributor to the credibility of the guidelines. Furthermore, the ability to convene more than 30 organisations representing health practitioners providing care to people with COVID-19 across Australia and to achieve 100% consensus across all these organisations was critical to the position of the guidelines as Australia’s ‘one-stop shop’ for COVID-19 clinical care.The questions about pace of guideline updating and thresholds for inclusion are important considerations for future living guidelines projects. As in all projects, there is a trade-off between quality, time and resources at the heart of these decisions, and they are also likely to be dependent on the context and the clinical topic under consideration. Decisions about how often new evidence should be considered should be driven by the rate of emergence of new evidence, clinical uncertainty and importance of the area under review. The frequency with which panels are convened does not have to be directly reflected in the publication schedule for the guideline. A guideline with multiple guideline panels like those convened by the Taskforce can, for example, publish weekly updates with only some panels meeting each week. Indeed, this is the model that the Taskforce has now adopted, reflecting the transition from a focus on primarily developing new recommendations for many questions to the less-resource intensive work of maintaining recommendations in living mode.Existing guidance for living systematic reviews  suggestsThe recent rise in publication of living guidelines –5 and liThis study has some limitations, mostly arising from its pragmatic purpose to inform the conduct of the Taskforce as it was underway. We conducted only a small number of interviews in each round of the process evaluation, however these included all the senior leadership team in each cycle with the exception of two occasions when staff were on leave; along with a selection of other contributors. All Taskforce participants were invited to respond to the survey, and although participation declined over time, numbers were consistently above 25 each cycle, even during school holidays and Melbourne’s second wave of COVID-19. The combination of the regular interviews with the quantitative and qualitative data from the broader survey provides useful insights into the progress and mindset of the Taskforce at each timepoint.A preliminary analysis of the results of each cycle of the process evaluation were reported to the Taskforce Executive and Steering Committee every four weeks, approximately one week after the data was collected and were used to inform and guide a process of continuous improvement of Taskforce processes and outputs. Providing the results rapidly and contemporaneously meant they could be used iteratively to identify and address emerging issues, and ascertain whether previously raised issues were being effectively addressed, while the project was underway. Some examples of use of the process evaluation data included work to refine roles within the Evidence Team and Executive to improve clarity and workflows; processes to restructure guideline panels to optimise efficiency; creation of deputy chair and methods co-chair roles in each panel to ensure appropriate levels of support; and development of governance mechanisms and standard operating procedures in response to issues raised. The rapid cycle of receipt of feedback through the process evaluation, development of responses by the Taskforce Executive and Steering Committee, and request for further feedback from the next timepoint of the process evaluation, also built confidence that the process evaluation was meaningful, that feedback was being heard, and that contributors’ views were valued. One unexpected benefit of the process evaluation interviews was the opportunity provided to interviewees for a time set aside to pause and reflect, and they felt that the opportunity to step away from the business of their day to day work was helpful both personally and professionally. Living guideline developers should consider establishing a similar model of process evaluation to inform the development and implementation of their guidelines.Several factors should be considered when interpreting our findings. The most important of these is the context of a global pandemic in which the Taskforce has operated, which has had multiple impacts. The high profile and vital importance of COVID-19 as a global and national health threat was likely a major motivator for the high-levels of engagement in the work of the Taskforce, both for the member organisations and individual clinicians contributing to guideline panels, and for the Taskforce team. It is likely that this, and the resulting urgent demand for recommendations to guide clinical practice, provided an enabling environment for the high levels of collaboration and cooperation seen in the Taskforce. The COVID-19 context also placed additional pressure and stress on the team and Taskforce contributors, as many faced significant additional personal and professional challenges, including an extended lockdown period for staff based in Victoria. Remote working was the norm for the Taskforce, with staff working from home—every team, panel and group meeting was conducted in an online environment. These contextual factors perhaps limit generalisability of some of our findings, but the global pandemic also provides a useful stress-test of the living evidence model, demonstrating that it is possible to develop rigorous, evidence-based living guidelines even during times of crisis. This process evaluation presents useful learnings on the rapid development and establishment of living guidelines during a pandemic. We have demonstrated that rapid development of living GRADE-based guidelines is both feasible and acceptable and provided some insights to the strengths and challenges of living guideline methods in general.The Australian Guidelines for the Clinical Care of People with COVID-19 are an important example of intensive application of living guideline methods. Despite the challenges of implementing a novel approach to evidence-based guideline development remotely during a pandemic, the Taskforce has demonstrated the feasibility of very frequent updating for topics in which stakeholders value very high levels of guideline currency.The National COVID-19 Clinical Evidence Taskforce process evaluation provided useful data to improve the guidelines and the processes by which they were produced; and identified important considerations for future living guideline projects. An impact assessment is also being conducted to examine the extent of awareness, acceptance and use of the guidelines.


Evidence‐informed practice versus evidence‐based practice educational interventions for improving knowledge, attitudes, understanding, and behavior toward the application of evidence into practice: A comprehensive systematic review of undergraduate health and social care students.11.1Over the past three decades, there has been increasing attention on improving healthcare quality, reliability, and ultimately, patient outcomes, through the provision of healthcare that that is influenced by the best available evidence, and devoid of rituals and tradition  created by Tilson et al., Knowledge: Knowledge refers to learners’ retention of facts and concepts about evidence‐informed practice and evidence‐based practice. Hence, assessments of evidence‐informed practice and evidence‐based practice knowledge might assess a learner's ability to define evidence‐based practice and evidence‐informed practice concepts, list their basic principles or describe levels of evidence.Attitudes: attitudes refer to the values ascribed by the learner to the importance and usefulness of evidence‐informed practice and evidence‐based practice to inform clinical decision‐making.Understanding: understanding refers to learners’ comprehension of facts and concepts about evidence‐based practice and evidence‐informed practice.Behavior: Behavior refers to what learners actually do in practice. It is inclusive of all the processes that a learner would use in the implementation of evidence‐informed practice and evidence‐based practice, such as assessing patient circumstances, values, preferences, and goals along with identifying the learners’ own competence relative to the patient's needs in order to determine the focus of an answerable question.The mode of delivery of the educational program could be in the form of workshops, seminars, conferences, journal clubs and lectures (both face‐to‐face and online). The content, manner of delivery and length of the educational program may differ in each of the studies to be included as there is no standard evidence‐informed practice/evidence‐based practice educational program.In this systematic review, evidence‐informed practice and evidence‐based practice educational interventions that are targeted towards health and social care postgraduate students or registered health and social care practitioners will be excluded. Comparison interventions will include educational interventions that do not advance the teaching of the principles and processes of evidence‐informed practice and/or evidence‐based practice in healthcare or no intervention.1.3Most efforts to apply evidence into healthcare practice have either been unsuccessful or partially successful  between evidence‐informed practice and evidence‐based practice educational interventions?2.Does participating in evidence‐informed practice versus evidence‐based practice educational interventions facilitate the use of evidence in practice ?3.Do both evidence‐informed practice and evidence‐based practice educational interventions targeted at undergraduate health and social care students affect patient outcomes ?4.What factors affect the impact of evidence‐informed practice and evidence‐based practice educational interventions ?Specifically, this systematic review will answer the following questions:33.13.1.1Based on the objectives and the outcome criteria for this systematic review, it is anticipated that relevant studies will have employed not only quantitative methodologies but also qualitative methods. Thus, this systematic review proposes to include both qualitative and quantitative research articles (a mixed‐methods systematic review).Specifically, this systematic review will follow the recommended steps by Sandelowski, Leeman, Knafl, & Crandel, We will then integrate the results of the two separate syntheses by means of an aggregative mixed‐methods synthesis. The two results  will be integrated by translating findings from the quantitative synthesis into qualitative statements, by use of Bayesian conversion. This is because, it is preferable to translate quantitative synthesis into qualitative statements, than it is from qualitative to quantitative, as problems may arise when converting verbal accounts  into numbers or quantities JBI . Figure 1.Randomized controlled trial: Studies in which participants are randomly assigned to intervention and comparison conditions.2.Quasi‐randomised controlled trials: Studies where participants are assigned to intervention and comparison conditions through a quasi‐random approach, such as birth date, students identification number, or date of the week.3.Quasi‐experimental controlled trial with individual level matching: Studies where participants in the intervention and comparison conditions are assigned to conditions in a non‐random manner . For such studies where participants in the intervention and comparison groups are not matched, there must be enough statistical information to enable us to evaluate pre‐test effect sizes for at least one outcome measure.4.Quasi‐experimental controlled trial with pre‐test adjusted outcomes: Studies where the intervention and comparison conditions are assigned in a non‐random manner, but the pretest differences between groups have been adjusted by the study authors. Examples include pre‐test‐adjusted post‐test means and regression coefficients from models that adjust for pretest.5.Quasi‐experimental controlled trial with pretest data: Studies where the intervention and comparison conditions are assigned in a non‐random manner, but pre‐test data are available for each outcome. For such studies, pre‐test data must be reported in a form that allows evaluation of the initial equivalence of the intervention and control groups on those variables through the calculation of effect size. For those outcomes with inapplicable pre‐test data, data for a close proxy of a pre‐test must be available.To be eligible for inclusion in this review, study designs must meet at least one of the following criteria:Other studies to be eligible for inclusion include, before and after studies, prospective and retrospective cohort studies. In order to determine the effects of the educational intervention, the comparison groups that were used in included studies will be analysed. Additionally, primary studies that use descriptive epidemiological study designs will be eligible for inclusion in this systematic review. Examples of eligible epidemiological study designs will include case series, individual case reports, and descriptive cross‐sectional studies, focus groups, and interviews. Furthermore, other qualitative approaches such as ethnography, phenomenology, and grounded theory will be eligible for inclusion. Biases and limitations associated with any included study design will be discussed in relation to the impact it may have on the effectiveness of the intervention.3.1.2To be eligible for inclusion in this systematic review, primary studies must include, as participants, undergraduate pre‐registered health and social care students in higher education (University) from any geographical area. Studies whose participants, are registered health and social care practitioners and postgraduate students will be excluded from this review.3.1.3In this systematic review, we will include studies that evaluate any formal evidence‐based practice and/or evidence‐informed practice educational interventions aimed at improving undergraduate pre‐registered health and social care students’ knowledge, attitudes, understanding and behavior in the application of evidence into healthcare practice. These two interventions will then be compared to determine whether the two concepts act together or individually to facilitate the application of evidence into practice. If it emerges that the two concepts act individually, we will determine which of them better facilitates the effective application of evidence into practice.As mentioned above, in this systematic review, an evidence‐informed practice educational intervention refers to any formal educational program that facilitates the application of but not limited to the evidence‐informed practice model described by McSherry, The mode of delivery of the educational programs may be in the form of workshops, seminars, conferences, journal clubs and lectures (both face‐to‐face and online). The content, manner of delivery and length of the educational program may differ in each of the studies to be included as there is no standard evidence‐informed practice/evidence‐based practice educational program.Evidence‐informed practice and evidence‐based practice educational interventions that are targeted at health and social care postgraduate students or registered health and social care practitioners will be excluded. Comparison conditions will include educational interventions that do not advance the teaching of the principles and processes of evidence‐informed practice and/or evidence‐based practice in healthcare.3.1.41.Participants’ knowledge about evidence‐informed practice and/or evidence‐based practice.2.Participants’ understanding of evidence‐informed practice and/or evidence‐based practice.3.Participants’ attitudes towards evidence‐informed practice and/or evidence‐based practice.4.Participants’ behavior towards evidence‐informed practice and evidence‐based practice.In this systematic review, studies will be eligible for inclusion if they address at least one of the following constructs as an outcome:As previously stated, the above constructs  will be defined based on the Classification Rubric for Evidence‐based practice Assessment Tools in Education (CREATE), developed by Tilson et al., Knowledge: Knowledge will refer to learners’ retention of facts and concepts about evidence‐informed practice and evidence‐based practice. Hence, assessment of evidence‐informed practice and evidence‐based practice knowledge might assess a learner's ability to define evidence‐based practice and evidence‐informed practice concepts, list their basic principles or describe the levels of evidence.Attitudes: attitudes will refer to the values ascribed by the learner to the importance and usefulness of evidence‐informed practice and evidence‐based practice to inform clinical decision‐making.Understanding: understanding will refer to learners’ comprehension of the facts and concepts relating to evidence‐based practice and evidence‐informed practice.Behavior: Behavior will refer to what learners actually do in practice. It is inclusive of all the processes that a learner uses in the application of evidence‐informed practice and evidence‐based practice, such as assessing the scientific evidence, patient circumstances, values, preferences, and goals along with identifying the learners’ own competence relative to the patient's needs in order to determine the focus of an answerable question.1.The use of a standardized questionnaire to evaluate knowledge, attitude, understanding, and behavior towards the application of evidence into practice. Examples of such questionnaires include, but are not limited to the Evidence‐Based Practice Belief (EBPB) Scale and Evidence‐Based Practice Implementation (EBPI) scales developed by Melnyk, Fineout‐Overholt, & May, 2.Examples of unstandardized instruments include, but are not limited to self‐reports from participants and researcher administered measures.Measurement of the above outcomes may be conducted using standardized or unstandardized instruments. This is because, to the researchers’ knowledge, there is no uniform tool for evaluating evidence‐based practice and evidence‐informed practice educational interventions. Specific measures will include, but not be limited to:Studies that measure the impact of evidence‐informed practice and/or evidence‐based practice educational programs on patient outcomes will be included.Examples of patient outcome indicators to be assessed include; user experience, length of hospital stay, nosocomial infections, patient and health practitioner satisfaction, mortality, and morbidity rates.In the current systematic review, no limit will be placed on the duration of the follow‐up. The rationale is to give room for studies with either short or long term follow‐up duration to be eligible for inclusion.This systematic review will include primary studies from any geographical area. However, due to language translation issues, only studies written in English will be included. Studies whose title and abstracts are in English and meet the inclusion criteria, but the full article is reported in another language will be included subject to the availability of translation services.To qualify for inclusion in the current systematic review, studies must have been published during the period from 1996 (the date when evidence‐based practice first emerged in the literature) , wildcards (such as?), phrase operators , and truncations (including *). This will be done in order to ensure search precision and sensitivity. In addition, our search strategy will have three sets of terms: the population, intervention(s) and outcomes. We will use limiting commands to narrow the results by dates, language, and type of study design. Below are examples of anticipated search terms to be used:3.2.2We will import the full set of search results directly into an Endnote Library. Where this is not possible, we will manually enter search results into the Endnote Library. An Endnote library will make it easier to identify duplicates and to manage references.3.2.3The current systematic review will utilize six strategies, in order to identify published and unpublished studies that meet the inclusion criteria described above. These strategies have been outlined below.1.Academic search completeAcademic search premierAMEDAustralian education indexBritish education indexCampbell systematic reviewsCanada bibliographic database (CBCA Education)CINAHLCochrane LibraryDatabase of Abstracts of Reviews on EffectivenessDissertation Abstracts InternationalEducation AbstractsEducation completeEducation full text: WilsonERICEvidence‐based program databaseJBI database of systematic reviewsMedlinePsycInfoPubmedSciELO (Scientific Electronic Library Online)ScopusElectronic Database Search2.GoogleGoogle scholarA web search using search engines3.OpenGrey (System for Information on Grey Literature in Europe)System for information on Grey LiteratureThe Society for Research on Educational EffectivenessVirginia Henderson Global Nursing e‐RepositoryGrey literature searchAppendix 3 illustrates the search strategy for the MEDLINE database searched on the EBSCOhost platform. We will modify the search terms and strategies for the different databases.4.Hand searching. The table of content of at least three journals that published most of the studies identified as eligible for inclusion will be hand searched for additional relevant studies. Examples include Worldviews on Evidence‐Based Nursing Journal, British Medical Journal, and the British Journal of Social Work.5.Track bibliographies of previously retrieved studies and literature reviews. The reference list of previously conducted systematic reviews, meta‐analysis and primary studies will be screened for other relevant studies.6.Contact leading authors. The Corresponding authors of identified eligible abstracts, whose full texts are unavailable will be contacted to request for full‐text reports.3.33.3.1The search output will be screened by two independent authors (either EAK and JBS or SH and RM) for relevant studies. The title and abstract of search output will first be screened, followed by the full text of articles with a seemingly relevant abstract. These articles will be assessed for eligibility using the pre‐specified inclusion and exclusion criteria. Studies that meet the inclusion criteria will be assessed independently by two authors (either SH and RM or EAK and JBS) for methodological validity using standardized critical appraisal instruments from the Joanna Briggs Institute Meta‐Analysis of Statistics Assessment and Review Instrument (JBI‐MAStARI). This includes, but not limited to, the JBI‐MAStARI checklist for case‐control studies, checklist for case reports, checklist for cohort studies, checklist for quasi‐experimental, checklist for randomised controlled trials, and checklist for analytical cross‐sectional studies. Any disagreements that may arise between authors will be resolved through discussion, if no agreement can be reached, a third author will be consulted.3.3.2Data will be extracted from included papers using standardized data extraction tools from JBI‐MAStARI. Information to be extracted from quantitative studies will include study design, interventions, population, outcomes of significance to the review questions, and specific objectives. See Appendix 1 for the quantitative data extraction form. While this form is currently generic, we will include information specific to this systematic review, such as methods used to assess the impact of evidence‐informed practice/evidence‐based practice educational interventions on patient outcomes.3.3.3For each included study, two authors (either RM and SH or EAK and JBS) will independently assess the risk of bias. This will be done using the Cochrane Collaboration's Risk of Bias tool , no attrition, or no evidence of differential attrition will be considered as having a low risk of bias. Use of Intention to Treat (ITT) analysis and methods of account for missing data  will be recorded.We will assess studies for reporting bias to determine whether there are inconsistencies between measured outcomes and reported outcomes. Studies will be considered as having a low risk of bias if the results section of publications clearly show that all pre‐specified outcomes are reported.3.3.4For continuous data, where outcomes on the same scale are presented, we will use mean difference, with 95% confidence interval. However, where outcome measures differ between studies, we will use the standardized mean difference as the effect size metric based on Hedges’ g, which is calculated using the formula below:SMD = Difference in mean outcome between groups ⁄ Standard deviation of outcome among participants.For dichotomous data, we will calculate the risk ratio  for the occurrence of an event. For the purpose of meta‐analysis, we will convert risk ratios to the standardized mean difference, by the use of David Wilson's practical effect size calculator.Meta‐regression will be used to assess the impact of moderator variables on the intervention effect size. Moderator analysis will be conducted if a reasonable number of eligible research articles are identified and if the required data is presented in the report.For studies with one control group versus two or more intervention groups, and all the interventions are regarded as relevant to the study, the following options will be used: 1) if the intervention groups are not similar, the sample size of the control group will be divided into two (or more based on the number of intervention groups) and will then be compared with the intervention groups 2) if the intervention groups are similar, the two groups will be treated as a single group. Therefore, two effect size estimates will be provided in this study. This is to ensure that participants in the control group are not “double counted”  of participants as units of analysis. In the event that cluster‐randomised trials  are identified as eligible, we will use standard conversion criteria as recommended in the Cochrane Handbook , data analysis will be conducted based on the available data.However, if data is missing not at random , we will impute the missing data with replacement values, and treat these values as if they were observed .3.3.7Heterogeneity will be assessed through the comparison of factors such as participant demographics, type of intervention, type of control comparators and outcome measures. Heterogeneity will be assessed and reported visually and by examining the I² statistic, which describes the approximate proportion of variation that is due to heterogeneity rather than sampling error. This will be supplemented by the Chi² test, where a P value < 0.05 indicates heterogeneity of intervention effects. In addition, we will estimate and present Tau², along with its CIs, as an estimate of the magnitude of variation between studies. This will provide an estimate of the amount of between‐study variation.3.3.8We will assess studies for reporting bias to determine whether there are inconsistencies between measured outcomes and reported outcomes. Studies will be considered as having a low risk of bias if the results section of publications clearly show that all pre‐specified outcomes are reported.3.3.9Narrative and statistical methods will be used to synthesise included studies. Data synthesis will be focused on calculating the effect sizes of the included studies. We will conduct meta‐analysis if our search yields sufficient (i.e. 2 or more) eligible studies that can be grouped together satisfactorily. A logical approach will be used when combining studies in meta‐analysis. Decisions on combining studies in meta‐analysis will be based on two reasons: 1) a sufficient number of eligible studies with similar characteristics 2) similar characteristics shared by those eligible studies may include the type of intervention and the targeted outcome of the intervention. Where a meta‐analysis is being conducted, we will employ the Comprehensive Meta‐analysis Software developed by Borentein, Hedges, Higgins, & Rothstein, 3.3.10Heterogeneity will be assessed through the comparison of factors such as participant demographics, type of intervention, type of control comparators and outcome measures. Heterogeneity will be assessed and reported visually and by examining the I² statistic, which describes the approximate proportion of variation that is due to heterogeneity rather than sampling error. This will be supplemented by the Chi² test, where a P value < 0.05 indicates heterogeneity of intervention effects. In addition, we will estimate and present Tau², along with its CIs, as an estimate of the magnitude of variation between studies. This will provide an estimate of the amount of between‐study variation. Sensitivity and subgroup analyses will also be used to investigate possible sources of heterogeneity.3.3.111.Unpublished studies2.Studies with outlier effect sizes3.Studies with a high risk of bias4.Studies with missing information  for methodological validity using the JBI Qualitative Assessment and Review Instrument (JBI‐QARI). Any disagreements that may arise between authors will be resolved through discussion, if no agreement could be reached, a third author will be consulted.Data will be extracted from included papers using standardized data extraction tool, namely, the JBI‐QARI. Information to be extracted from qualitative studies will include population, study methods, details about the phenomena of interest, outcomes of significance to the review question and specific objectives. See Appendix 2 for the qualitative data extraction form. While this form is currently generic, we will include information specific to this systematic review, such as methods used to assess the impact of evidence‐informed practice/evidence‐based practice educational interventions on patient outcomes.Where possible, qualitative research findings will be pooled using JBI‐QARI. This will involve the synthesis or aggregation of findings to generate a set of statements that represent that aggregation. Findings will be assembled based on their quality, and also, by grouping findings with similar meanings together. We will then perform a meta‐synthesis of these groups or categories so as to produce a single set of comprehensive synthesized findings. In the event that textual pooling is not possible, we will present findings in narrative form.Finally, results from both the quantitative review and the qualitative review will be integrated using the JBI Mixed Methods Aggregation Instrument (MMARI). The integration will be achieved by translating findings from the quantitative review into qualitative results through the use of Bayesian conversion to generate synthesized results.44.1Results of the searchIncluded studiesExcluded studies4.2Allocation (selection bias)Blinding (performance bias and detection bias)Incomplete outcome data (attrition bias)Selective reporting (reporting bias)Other potential sources of biasEffects of interventions5Summary of main resultsOverall completeness and applicability of evidenceQuality of the evidencePotential biases in the review processAgreements and disagreements with other studies or reviewsImplications for practiceImplications for research1.Content and Systematic Review methodology: Ms. Elizabeth Adjoa Kumah is a registered general nurse who has worked mainly in the critical care setting as a nurse supervisor and patient advocate. She has been actively engaged in teaching healthcare students in the clinical setting and serving as a mentor. She is currently pursuing a Ph.D. Health program, with evidence‐informed practice educational interventions as the area of research focus. She brings knowledge about the content both in terms of teaching healthcare students about the application of research evidence into practice and theoretically for improving knowledge of evidence‐informed practice and how it enhances evidence‐based practice skills, attitude and behavior in the educational setting. Elizabeth is passionate about improving the standard of patient care and patient outcome, which she believes could be achieved by the effective and consistent implementation of evidence‐informed practice. She will also help with the methodological aspects of the systematic review.2.Content and Systematic review methods: Professor Robert McSherry will bring both methodological as well as content expertise relating to evidence‐informed practice and the development of teaching programs to the team. His area of expertise is around the evidence‐informed practice, patient safety, quality and clinical governance using practice development. Practice development is about promoting person‐centred care and approaches, which Rob has integrated effectively within both educational and research programs. He is the co‐author of a book on systematic reviews and has over thirty years’ experience as a registered general nurse. Robs educational and professional expertise has been recognized and rewarded internationally and nationally. He was awarded the highly prestigious National Teaching Fellow award in the UK in 2011.3.Content and systematic review methods: Dr. Josette Bettany‐Saltikov will bring significant expertise in Systematic review methods and content to this systematic review, both in terms of knowledge about evidence‐based practice and knowledge about developing educational programs. She has taught systematic review methods to university students at all levels for over 15 years. She has also published a book on how to conduct a systematic review and has been involved in three Cochrane reviews, one of which she led. She has authored a number of systematic reviews on diverse topics published in other journals and has significant experience of developing educational programmes from her teaching experience as a university Senior lecturer for 23 years.4.Content and systematic review methods: Professor Sharon Hamilton will bring expertise in systematic reviewing. She is the director of the Teesside Centre for Evidence‐Informed Practice: A Joanna Briggs Institute Centre of Excellence, and has conducted a number of qualitative and quantitative reviews. Sharon is a registered nurse and has research expertise in the evaluation of clinical interventions.5.Information retrieval: Mrs. Julie Hogg brings Information retrieval expertise to the team. Julie is an Academic Librarian at Teesside University and will carry out a thorough and systematic search of the literature.6.Statistical analysis: Mrs. Vicki Whittaker is a very experienced statistician with over 18 years of experience in teaching and advising students and academics on their research projects and clinical trials. She has been involved in data analysis and meta‐analysis of numerous research projects and systematic reviews.The review team declares no potential conflicts of interest.Characteristics of studiesCharacteristics of included studiesCharacteristics of excluded studiesCharacteristics of studies awaiting classificationCharacteristics of ongoing studiesExcluded studiesStudies awaiting classificationOngoing studiesTeesside University, UKThis review forms part of a Ph.D. programme, which is supported and funded by Teesside University, Middlesbrough.No sources of support providedFeedback


The rapid spread of preliminary scientific evidence is raising concerns on its role in producing misinformation during the COVID-19 pandemic. This research investigated how the communication of uncertainty about preliminary evidence affects the spread of its inferred misinformation in a Weibo case study. In total, 3439 Weibo posts and 10,380 reposts regarding the misinformation of pets transmitting COVID-19 were analyzed. The results showed that attitude ambiguity toward the preliminary evidence and the stage when the evidence was first released with uncertainty were associated with higher numbers of likes and retweets of misinformation posts. Our study highlights the internal sources of misinformation and revisits the contextual perspective in misinformation studies. The coronavirus disease pandemic COVID-19) has triggered a “misinfo-demic” that warrants continuous research efforts 9 has tri,3. NotabIn this study, we aim to offer a new theoretical framework to explain the misinfo-demic by focusing on internal sources of misinformation and factors that facilitate the misinformation spread on social media. First, we theorize the rapid dissemination of preliminary scientific evidence on social media as a context where scientific misinformation can arise due to individuals’ naïve understanding of science. For example, the release of evidence that a pet dog tested “weak positive” has fueled the widespread misinformation that pets can transmit COVID-19 .Second, we aim to examine how communicating uncertainty about preliminary evidence affects the spread of COVID-19 misinformation and its refutations. Given the fast-developing nature of science during the pandemic, the communication of preliminary evidence is often swiftly evolving, from communicating with no evidence, evidence that is first released with uncertainty on its interpretations, to evidence that has reached some consensus from the science community. In addition, scientists often approach preliminary evidence with caution, which may signal uncertainty of the evidence. Therefore, it is essential to investigate whether and how uncertainty communicated surrounding preliminary evidence will be a factor that facilitates or inhibits the spread of misinformation.Research has found different sources of misinformation. Lewandowsky et al. extensively discussed four sources of misinformation, including rumors and fiction, governments and politicians, vested interests, and the media . In the Studies also reveal various content, sender, and network factors that can fuel misinformation spread. For example, content topic and novelty facilitate the spread of false news online . SendersWhile contributions of existing scholarship are essential, they suffer from two significant gaps. First, research on internal sources of misinformation is scarce. Though Coronel et al.  reveal mSecond and relatedly, as most studies assume sources of misinformation as external, they focus on attributes of misinformation and individual characteristics that facilitate the misinformation spread. As such, there is a lack of answers to questions about what are the contextual prerequisites for individuals to produce and spread misinformation. Though Rosnow pointed out as early as in 1988 that it is vital “to decipher the experiential contexts that ‘invite’ or ‘allow’ rumors to flourish  (pet), [猫] (cat), [狗] (dog), [家养动物] ; (2) transmission-related keywords: [传染] , [传播] (spread), [宿主] (host); and (3) COVID-19-related keywords: [冠状病毒] (coronavirus),  (novel coronavirus), [肺炎] (pneumonia), [新型肺炎] (novel pneumonia), [新冠肺炎] (novel coronavirus pneumonia), [武汉肺炎] (Wuhan pneumonia), nCov, and COVID-19. These keywords were chosen based on the public conversations related to the misinformation. A total of 3439 original Weibo posts that contain keywords from each of the three keyword sets and 10,380 reposts from those original posts were collected. Note that the data were only a collection of the full data pool based on the Weibo API rule. We employed a machine learning approach to measure users’ attitudes toward the evidence and the misinformation in the Weibo posts and reposts. This process included three steps  was constructed for human annotation. Please see step 3 for the coding logic for reposts that did not have meaningful texts. Coding dataset for the original posts. Two statements were coded in the annotation process for the original post dataset: Misinformation statement: “Pets can transmit COVID-19 to people.” This includes statements that pets, or domestic animals, can transmit the infection to humans, or simply that pets can transmit COVID-19 without mentioning a proper object. Evidence statement: “Pets can be infected with the virus.” This includes statements that pets, or domestic animals, have been infected with the coronavirus or pets tested positive for the virus infection or other similar statements. Three coders, who were required to read a coding instruction and go through a training process before labeling, independently indicated the author’s attitudes toward the two statements for each post into the following four categories: (1) endorsing: posts repeating or confirming the misinformation; (2) rejecting: posts denying the misinformation or citing those who debunked the misinformation; (3) ambiguous: posts relevant to the statement but not showing perceivable attitudes toward it; (4) not mentioned: all other posts not mentioning or relevant to the statement. Krippendorff’s alpha tests revealed an acceptable level of intercoder reliability for the two statements: 0.75 for the misinformation statement and 0.71 for the evidence statement. The coding disagreement was resolved by discussions. Note that an author could have different attitudes towards the two statements in a single post. If original posts were not irrelevant to both statements, they were excluded from further analyses. Coding dataset for the reposts. Three coders annotated the author’s attitude toward the misinformation statement in a repost into the four categories mentioned above and meanwhile were presented with the text of its original post for reference. The Krippendorff’s alpha was 0.74, indicating an acceptable level of intercoder reliability. Disagreements were resolved by discussions.Step 2: Construction and performance of supervised machine learning models. Label annotation was a classification problem. We used the pre-trained Chinese Bidirectional Encoder Representations from Transformers (BERT) models to build our text classifiers. BERT has been widely used in natural language processing as it obtained state-of-the-art performances in a wide range of tasks . The modTo construct classifiers, we first pre-processed the data by removing text elements such as mentions  and hyperlinks and segmenting raw texts into words with the Chinese lexical analyzer Jieba. Then, we randomly partitioned the human-coded original posts or reposts dataset into a training set (70%), a development set (20%), and a test set (10%). The training set was used to build the fine-tuned BERT models. The development set was used to evaluate the accuracy of variations of the models. Once the models achieved adequate performance on the development data, the test set was used to evaluate the final performance of the classifiers based on classification accuracy. We conducted the above processes for both original posts and reposts datasets. For the classifier regarding the original post dataset, the raw accuracy was 0.82 for the misinformation statement and 0.91 for the evidence statement. For the classifier regarding the repost dataset, the raw accuracy was 0.74. Therefore, the obtained machine-learning-based annotation accuracy was adequate for full dataset annotations with acceptable noise. Step 3: Final annotations. For original posts, we used the built classifiers to label the entire dataset. For reposts, we included only reposts of the relevant original posts into the final annotation and separated them into two subsets. The first subset contains reposts with meaningful texts such as original texts and texts from the original posts. This subset was annotated with the built classifiers. The second subset contains reposts that did not have meaningful texts, such as “repost” or “null” text. This subset was coded as the same attitude as that of the original posts toward the misinformation statement.The evidence stage was measured based on the timeline of evidence communication. As discussed, Hong Kong reported the first instance that a dog of a COVID-19 patient tested “weakly positive” for the virus on 28 February 2020; on 4 March, scientists from the WHO concluded it is a case of human-to-animal transmission of the virus. Thus, the evidence stage was identified as the three following stages: no evidence , uncertain evidence , and evidence consensus .Our analyses included three dependent variables: numbers of likes and reposts, and numbers of reposts in different attitudes toward the misinformation. For the total number of likes and reposts, all relevant original posts were analyzed. For the number of reposts in different attitudes, only original posts that have reposts collected in the dataset were analyzed. Given that the dependent variables were frequency types of data and violated the normality assumption of standard parametric tests, we conducted a series of nonparametric tests for the analysis. For H1, we employed a chi square test to examine the association between attitudes toward the evidence and those toward the misinformation. For H2 and 3 and RQ, we used the Aligned Rank Transform (ART) technique for nonparametric factorial analysis in the tests. The ART technique is a ranked-based test procedure that first assigns a rank to each raw data point in an increasing order and then conducts regular parametric factorial tests on those ranks . It is rA total of 2607 (75.8%) original posts were coded as relevant to the misinformation discourse and included in the analysis. Among them, 570 posts received a total of 10,278 reposts. These reposts were also included in the analysis. We observed a low endorsement rate (3.6%) and a high rejection rate (74.1%) of the misinformation for original posts. In addition, we found a high endorsement rate (78.0%) of the evidence. We observed a rate of 6.51% for reposts in endorsing the misinformation and 58.9% in rejecting it. Furthermore, approximately one-fourth (25.1%) of the reposts were coded as irrelevant to the misinformation. χ2(9) = 257.6, p < 0.001. Notably, users of 65.9% (91 out of 138) of posts rejecting the evidence rejected the misinformation, and those of 72.0% (36 out of 50) of posts being ambiguous about the evidence were also uncertain about the misinformation. For users who endorsed the evidence in posts, 76.3% (1552 out of 2034) also rejected the misinformation, and only 2.3% (46 out of 2034) supported the misinformation. Within users who endorsed the misinformation in posts, 49.5% (46 out of 93) also endorsed the evidence. This suggests that users’ attitudes toward the misinformation and the evidence generally corresponded. Nevertheless, users of posts who rejected the misinformation were also likely to endorse the evidence in our dataset. We organized the data into three evidence stages: no evidence (1 January to 27 February), uncertain evidence (28 February to 3 March), evidence consensus (4 March on-ward) see . At the F = 32.3, p < 0.001, partial 2η = 0.072) and reposts  = 13.3, p < 0.001, partial 2η = 0.031). To examine H2, the first test compared numbers of likes and reposts received by original posts in different attitudes toward the evidence and the misinformation. Results from the ART analysis of attitudes toward the evidence in the original posts (four: endorsing vs. rejecting vs. ambiguous vs. not mentioned) * attitudes toward the misinformation in the original posts (three: endorsing vs. rejecting vs. ambiguous) showed that the interaction was significant for both likes  = 14.5, p < 0.001. The second test compared numbers of reposts in different attitudes toward the misinformation received by original posts in different attitudes toward the evidence. Results from ART analysis on the interaction effects of attitudes toward the evidence in the original posts (between four: endorsing vs. rejecting vs. ambiguous vs. not mentioned) * attitudes toward the misinformation in the reposts (within three: endorsing vs. rejecting vs. ambiguous) showed that the interaction was significant, F = 34.1, p < 0.001, partial 2η = 0.052) and reposts  = 29.9, p < 0.001, partial 2η= 0.046). To examine H3, the first test compared numbers of likes and reposts received by original posts in different attitudes toward the misinformation among different evidence stages. Results from the ART analysis of attitudes toward the misinformation in the original posts (three: endorsing vs. rejecting vs. ambiguous) * evidence stages  showed that the interaction was significant for both numbers of likes  = 43.4, p < 0.001. The second test compared numbers of reposts with different attitudes toward the misinformation received by original posts published at different stages. Results from ART analysis on the interaction effects of evidence stages  * attitudes toward the misinformation in the reposts (within three: endorsing vs. rejecting vs. ambiguous) showed that the interaction was significant, For the original post analysis, we planned to run analyses of the three-way between-subject interaction effects of attitudes toward the misinformation in the original posts (three: endorsing vs. rejecting vs. ambiguous) * attitudes toward the evidence in the original posts (four: endorsing vs. rejecting vs. ambiguous vs. not mentioned) * evidence stages  on numbers of likes and reposts. However, in our dataset, there was no instance of posts endorsing the misinformation while signaling evidence ambiguity and published at the uncertain-evidence stage. Similarly, there was no instance of posts rejecting the misinformation while expressing attitude ambiguity and published at the no-evidence stage. Thus, we were unable to test the three-way interactions based on analyses of original posts. F = 10.5, p < 0.001. For the repost analysis, we ran the analysis for a three-way mixed effect of attitudes toward the evidence in the original posts (between four: endorsing vs. rejecting vs. ambiguous vs. not mentioned) * evidence stages  * attitudes toward the misinformation in the reposts (within three: endorsing vs. rejecting vs. ambiguous) on numbers of reposts. The results showed that the three-way interaction effect was significant, We set out to examine a new theoretical framework that the uncertainty communicated with preliminary evidence can promote internal motivated reasoning based on a naïve understanding of science that produces misinformation inferred from the evidence. To fulfil the aim, we tested if the communication of preliminary evidence could be a prompt for individuals’ internal processes to generate misinformation and if communicating the uncertainty of the evidence would facilitate misinformation spread based on social media data. We examined evidence uncertainty communication in two forms: users’ ambiguous attitudes toward the evidence and the stage when the evidence was communicated with uncertainty. This study contributes to the literature of science communication and misinformation in several important ways. First of all, we gained empirical support for our theoretical framework that the uncertainty circulating around preliminary evidence can promote the generation and transmission of misinformation inferred from the evidence. As the results showed that users’ attitudes toward the evidence and the misinformation corresponded, this suggested that users indeed perceived an inherent link between the evidence and the misinformation based on their naïve understanding of science. Importantly, users’ ambiguous attitudes toward the evidence and the uncertain-evidence stage resulted in more likes and retweets of the misinformation and/or fewer likes and retweets of the refutations. This further indicates that the uncertainty signaled in the posts strengthens individuals’ beliefs in such an inherent link. This is likely because the uncertainty prompts individuals to seize and freeze an available short-cut to reduce the uncertainty. However, the current data were unable to demonstrate such a mechanism, and future experimental studies are required. Nevertheless, this study indicates that health policymakers should at least regulate the hasty communication of emerging evidence with inherent uncertainty during a novel health crisis for the purpose of misinformation control.Second, our study highlights and extends Davis and Loftus’ framework  on interThird, this study revisits Rosnow’s proposal  on examiExtending previous research, our investigation revealed that Weibo served as a platform that promoted misinformation and inhibited the propagation of refutations when the emerging evidence was communicated with uncertainty. Particularly, we found that authors endorsing the misinformation while expressing ambiguous attitudes toward the evidence in their posts received more likes and retweets than those rejecting or endorsing the evidence. In addition, users tended not to share the refutation posts when the evidence was first released and had not gained any consensus. These findings suggest that Weibo-like social media platforms may not be suitable for communicating uncertainty toward early evidence during novel crises. They warrant policy attention as studies have suggested a high prevalence in highlighting scientific uncertainty associated with COVID-19 preprints on digital media outlets . In addiInterestingly, and in contrast, we found that Weibo might be good for communicating scientific consensus. The interaction analysis between attitude ambiguity and the evidence stage revealed that users tended to debunk the misinformation in their reposts when the original posts supported the evidence consensus. In comparison, if the original posts signaled uncertainty about the evidence consensus, users tended to spread the misinformation instead. These findings are consistent with the well-documented knowledge that the communication of consensus uncertainty will lead to the endorsement of one’s heuristic beliefs in the inferred (mis)information . NeverthIn general, we found that expressing attitude ambiguity toward the evidence at different evidence stages was associated with different patterns in reposts of misinformation and refutation messages. Our analyses revealed that attitude ambiguity toward the evidence suppressed the dissemination of refutations only at the uncertain-evidence stage, but not at the evidence-consensus stage. In contrast, such ambiguity promoted misinformation to a greater extent at the later stage than at the former stage. A possible explanation of these findings may be that attitude ambiguity at different evidence stages may signal different levels and types of evidence uncertainty. For example, attitude ambiguity toward a piece of uncertain evidence may signal a strong deficient uncertainty about a known gap, and thus the evidence should be less convincible for rebuttals . In compIn this study, we included data before any evidence emerged in the analysis. At first glance, the data seem not so relevant to our research question. However, we think that they are of both theoretical and practical significance. Theoretically, these data provided a baseline for comparison to demonstrate the effect of communicating emerging evidence related to misinformation. We showed that original posts about the rejection of the misinformation received a decreased number of likes after the evidence emerged. Such a decrease was greater at the uncertain-evidence stage than at the evidence-consensus stage. In addition, those original posts received a decrease in rejection reposts on the misinformation only at the uncertain-evidence stage, but not at the evidence-consensus stage. These findings suggest that the effect was associated with evidence uncertainty rather than evidence consensus. Practically, analyses on data from the no-evidence stage showed that posts that induced uncertainty about a piece of “fake” evidence would suppress the dissemination of misinformation rebuttals and promote ambiguous beliefs on the misinformation. This finding provides empirical support on how misinformation can be spread with groundless evidence . Our anaThis study has several limitations. First, we were not able to measure users’ cognitive processes using social media data. Second, we examined our hypotheses with only a circumstance of misinformation, which may be a constraint in result generalization. Third, limited by the method of ART analysis, we were not able to include any covariate to control for potential impacts of other relevant factors, such as the emotional tone of the posts and account attributes. Finally, Weibo is a Twitter-like social media platform where users’ relationships are asymmetrical and information is open and prosperous. It is quite different from Facebook and WeChat, where social interactions occur mainly among closed relationships and information is often private and exclusive. Therefore, comparisons between Weibo and other platforms are needed. In summary, our study provides the first empirical support that misinformation can be induced and spread on social media because of the communication of uncertainty about the emerging scientific evidence during a novel health crisis. Supporting H1, our results showed that attitudes toward the misinformation and the evidence were associated, suggesting that the communication of preliminary evidence could induce misinformation. Our findings also supported H2 that users preferred misinformation to refutations when the original posts expressed ambiguity about the evidence. The findings partly supported H3 by demonstrating that users were averse to refutations at the uncertain-evidence stage. Finally, our results answered the RQ by revealing that users’ attitudes toward the evidence and the stage when the evidence was communicated interactively affected misinformation spread. Our study extends the frameworks of internal sources of misinformation by demonstrating that a naïve understanding of scientific evidence can be a source of misinformation. Additionally, this study revisits the contextual perspective of misinformation and supports that contextual factors are important drivers of misinformation generation and spread. Finally, this study suggests to build a misinformation-unfriendly environment that prevents the generation and spread of misinformation. Particularly, our study suggests that health policymakers should regulate the hasty communication of emerging evidence during a novel health crisis. In addition, authorities should better use Weibo-like social media to communicate scientific consensus rather than uncertainty about the emerging evidence.


Typhoons have caused serious economic losses and casualties in coastal areas all over the world. The big size of the tropical cyclone sample by stochastic simulation can effectively evaluate the typhoon hazard risk, and the typhoon full-track model is the most popular model for typhoon stochastic simulation. Based on the advantages of machine learning in dealing with nonlinear problems, this study uses a backpropagation neural network (BPNN) to replace the regression model in the empirical track model, reestablishes the neural network model for track and intensity prediction in typhoon stochastic simulation, and constructs full‐track typhoon events of 1000 years for Northwest Pacific basin. The validation results indicate that the BPNN can improve the accuracy of typhoon track and intensity prediction. Typhoon is a strong disastrous weather system that occurs in the tropical ocean. It is commonly known as typhoon in the Northwest Pacific. The disasters such as strong wind, rainstorm, and huge waves caused by typhoon have led to serious social and economic losses and casualties in coastal areas. The southeast coastal region of China is one of the areas most seriously affected by typhoon in the world. From 1983 to 2008, the average economic losses caused by typhoons in China were about Chinese Yuan (CNY) 25 billion every year, and the loss showed an obvious growth trend .Typhoon risk analysis has strong statistical characteristics, and the results are often limited by the size and quality of samples. Therefore, the method of stochastic simulation to expand tropical cyclone (TC) samples for typhoon risk assessment has been developed internationally  and has The circular subregion model is a traditional method of typhoon risk analysis. It was first implemented by Russell  for the The empirical track model is developed by Vickery et al.  which isANN is an algorithmic mathematical model for distributed parallel information processing by imitating the behavior characteristics of animal neural networks . ANN hasThe traditional empirical track model uses a statistical regression method to predict the typhoon track and intensity, which could not escape some forecast errors. ANN method has better adaptive learning and nonlinear mapping ability. It is more suitable to deal with nonlinear problems with complicated physical mechanism, causality, or reasoning rules. Therefore, this study intends to use BPNN to reestablish the prediction model of typhoon track and intensity in typhoon stochastic simulation, to improve the accuracy of typhoon track and intensity prediction, and then to improve the accuracy of typhoon risk analysis.c, storm heading θ, and intensity I of typhoon at two adjacent moments. The model is described as follows:a1, a2, etc. are constants; Ψ represents the latitude (°) of typhoon center; λ represents the longitude (°) of typhoon center; c is the Hurricane translation speed (m/s); θ is the storm heading (°); I is the Hurricane relative intensity; i-1, i, and i+1 represent different time steps; the time interval between i + 1 and i is 6 h; Δlnc = lnci+1 − lnci and Δθ = θi+1 − θi; TiS is the monthly averaged sea surface temperature ; and εc, εθ, and εI are random error terms with zero mean.The storm empirical track model was developed by Vickery et al. , which dpda is the ambient pressure (hPa); pdc is the minimum sustainable surface value of central dry partial pressure (hPa); pc is the typhoon central pressure (hPa); and es is the saturation vapor pressure.The concept of typhoon relative intensity was proposed by Darling  based ona1, a2, etc. of the regression model are fitted. In addition, they distinguished the easterly and westerly headed storms and obtained two different sets of model coefficients. For some grid cells with too few historical Hurricane data, the fitted regression model is not reliable, so the reliable regression model of a nearby grid cell is used instead.Vickery et al.  divided The original empirical track model has many coefficients that need to be estimated for each grid cell. Li and Hong  simplifiArtificial neural networks (ANNs) can model any input and corresponding output without considering the interaction mechanism between them . It procThis paper adopts BP neural network, which is widely used in the artificial neural network, including input layer, hidden layer, and output layer . BPNN deyt is the target output and ct is the network output. A set of the input vector and corresponding target output vector constitutes a training pattern of the network, m represents the total number of training patterns, and q is the number of neurons in output layer. The error at the output layer is backpropagated and is allocated to all nodes of each layer as the basis for adjusting the weight of each node. The interconnection weights and biases are iteratively adjusted in the feedforward process and error back propagation. The iteration process continues, until a specified convergence is reached or until a predetermined number of learning times [We selected the most used “maximum squared error” (MSE) as the performance function of the BP network training, which can evaluate the simulating performance of the network.ng times .The historical typhoon dataset used for this study comes from the Yearbook of Tropical Cyclone provided by the China Meteorological Administration (CMA) . The data were first released by the China Central Meteorological Observatory (CCMB) (from 1949 to 1982) and then released by the China National Meteorological Administration (CNMB) (from 1983 to 1992). Since then, it has been maintained and updated by CMA. The database records the relevant information of all tropical cyclones passing through the Northwest Pacific and the South China Sea since 1949. It contains the location and intensity information of typhoons at a time interval of 6 h, including the name and number of typhoons, typhoon center position (longitude and latitude), central pressure, and 2-min mean maximum sustained wind speed near the center .Due to the use of different anemometers, the data from 1949 to 1970 are relatively large compared with the later data. Therefore, before using the typhoon data provided by the CMA, this paper corrected the data before 1970 based on the correction method of Li et al. . And theThe SST data used in this study in the typhoon intensity model are from the moderate resolution imaging spectroradiometer (MODIS) ocean products . MODIS ici, θi, ψ, and λ, as the input data, and the translation speed at the next adjacent time, that is, ci+1, as the output data. For the prediction model of storm heading, the input data are the same as that of the typhoon translation speed model, and the output is the storm heading of the next adjacent time, that is,θi+1. When selecting the relative intensity I and sea surface temperature Ts as the input data for the typhoon intensity model, we found the predicted typhoon intensity was easy to jump. This is mainly because the calculation formula of relative intensity is too complex, and the neural network is prone to overfitting in the grid cells with less historical typhoon data. Therefore, we directly selected the typhoon central pressure at the former 6-hour and 12-hour time and the SST at the former 6-hour time, that is, pi, pi-1, and Tsi, as the input data, and the typhoon central pressure at the next adjacent time, that is, pi+1, as the output data.First, we divided the Northwest Pacific basin into 5° × 5° grid, and the grid number is shown in ci, θi, ψ, λ, and ci+1. That for the prediction model of storm heading is ci, θi, ψ, λ, and θi+1. Pi, pi-1, Tsi, and pi+1 form the training pattern for the typhoon intensity model. Based on the statistical results of historical typhoon data from CMA data, the number of training pattern for easterly and westerly headed storms in each grid is shown in Figures A training pattern for the prediction model of typhoon translation speed is There are many parameters to be determined in a BPNN model, for example, the number of nodes in each layer of neural network, the activation function, and the training function.The node number in input (output) layer depends on the dimension of the input (output) vector. Based on the research results of Vickery et al.  and Li eThere is still no better method to determine the number of nodes in the hidden layer in advance. Too few nodes will make the network performance poor, and too many nodes will prolong the training time and prone to overfitting. Therefore, the node number is usually gradually increased or reduced in the training process, until the required accuracy is achieved. After debugging by trial and error method, the node number of the hidden layer is determined to be 10 (only one hidden layer). When the node number of the hidden layer is 10, for most grid cells, the correlation coefficient between neural network prediction results and actual results is the largest, and the root mean square error is the smallest.R) and root mean square error (RMSE) between the predicted and target results were adopted in this study. The calculation formulas are as follows:yk denotes the predicted results of the network and R| is to 1, the stronger the correlation is. The RMSE represents the error between the values of prediction and actual observation. The smaller the error is, the more accurate the predicted value is.In order to evaluate the prediction results under different neural network models, the correlation coefficient  of typhoon translation speed predicted by the BP model (ANNa2) and regression model  of storm heading predicted by the BP model (ANNb2) and regression model  for eacRMSE  for eacThe process of constructing a virtual typhoon is to first divide the Northwest Pacific basin into a 5° × 5° grid and then build a neural network prediction model  for the typhoon translation speed, storm heading, and intensity  based on the historical typhoon data of each grid. Third, based on the distribution of the starting points of historical typhoons, the starting points of virtual typhoons are randomly selected to initialize the typhoon track. Fourth, the typhoon track and intensity prediction model of the grid where the typhoon is located are used to predict the typhoon position and intensity at the next time. Finally, the complete typhoon track can be obtained by repeating the fourth step.Based on the above method of constructing a virtual typhoon, we constructed a virtual typhoon dataset of 1000 years for Northwest Pacific basin based on the BP model, including 32693 virtual typhoon events. Taking the super typhoon Lekima, which landed in China in 2019, as an example, we used the constructed BP neural network model to predict 12 typhoon tracks that have the same initial state as Lekima, and the predicted and observed typhoon paths are shown in To further validate the reliability of the virtual typhoon dataset constructed based on the BP model, we first selected 46 research stations at a space interval of 100 km along the east coast of China, as shown in Artificial neural network has better self-adapting, self-learning, and nonlinearity mapping capability, which is more suitable for dealing with complex nonlinear problems. This paper used BP neural network to replace the regression model in the original typhoon empirical track model and reestablished prediction models of typhoon track and intensity in typhoon stochastic simulation. For the prediction models of typhoon translation speed, storm heading, and typhoon central pressure, different input and output factors were used to establish different BP models, and the optimal model was selected through model evaluation. Based on the optimal models of typhoon translation speed, storm heading, and typhoon central pressure, this paper constructs virtual typhoon events of 1000 years for Northwest Pacific basin. The validation results indicate that the BPNN can improve the prediction accuracy of typhoon track and intensity.