We aimed to provide clinicians with introductory guidance for interpreting and assessing confidence in on Network meta-analysis (NMA) results.We reviewed current literature on NMA and summarized key points.Network meta-analysis (NMA) is a statistical method for comparing the efficacy of three or more interventions simultaneously in a single analysis by synthesizing both direct and indirect evidence across a network of randomized clinical trials. It has become increasingly popular in healthcare, since direct evidence  are not always available. NMA methods are categorized as either Bayesian or frequentist, and while the two mostly provide similar results, the two approaches are theoretically different and require different interpretations of the results.We recommend a careful approach to interpreting NMA results and the validity of an NMA depends on its underlying statistical assumptions and the quality of the evidence used in the NMA. The highest level of evidence for the comparative effectiveness of different clinical interventions generally comes from systematic reviews of randomized controlled trials (RCTs) –3. The mNetwork meta-analysis (NMA) is a statistical method that extends the principles of pairwise meta-analysis to the evaluation of multiple interventions in a single process, which is achieved by combining both direct and indirect evidence , 5, 7, 8Network meta-analysis is a statistical method for synthesizing direct and indirect evidence from a network of clinical trials to concurrently compare multiple clinical interventions in a single process , 5, 7–9.NMA has matured over the recent years and NMA models are available for different types of individual-level and trial-level data and summary effect measures  and are being implemented in both frequentist and Bayesian frameworks , 13, 14.95% credible interval (CrI), and the ranking probabilities of interventions [the 95% confidence interval (CI) for rejecting the research hypothesis based solely on present data [The Bayesian method combines the known information obtained in the past (prior information) with the present data (likelihood) to calculate the posterior (“post” data observation) probability where the research hypothesis holds . Therefoventions , 29, 30.ent data , 8, 17. ent data , 26, 31.95% probability of being between 0.04 and 0.82. The estimated OR of CRBD at 0 h after surgery of tramadol versus placebo is 0.26 with a 95% CrI of . Since this 95% CrI contains 1, OR of CRBD at 0 h after surgery of tramadol versus placebo has a 95% probability of not being different. A 95% CI under the frequentist approach does not have the same intuitive and practical interpretation, but can only conclude whether the two interventions are statistically different in the effect size at 5% level of significance [The Journal of Anesthesia has recently published several NMAs –36. We iificance , 38. A sWe illustrate the interpretation of results of a frequentist NMA through a study that examined the effects of individualized positive end-expiratory pressure (PEEP) combined with recruitment maneuver (RM) on intraoperative oxygenation during abdominal surgery . A frequIt is worthwhile to discuss the interpretation of ranking probabilities such as surface under the cumulative ranking area (SUCRA), since these often tend to be misinterpreted in the literature , 28, 39.NMA inherits all challenges present in a conventional pairwise meta-analysis, but with magnified complexity due to the large number of comparisons within the evidence network . To copeIn summary, violation of the similarity, homogeneity and consistency assumptions, as well as the presence of any risk of bias and publication bias, affect the overall confidence in the results of an NMA. Therefore, when reviewing a published NMA, one should examine if these issues were identified and how they were dealt with and base one’s confidence in the NMA on these factors. GRADE  is a transparent framework for developing and presenting summaries of evidence , 43. It Nowadays, as data become easier to collect and assess, we enter an era of “big data” with big data analysis emerging as a new analysis technique in clinical research . We can Network meta-analysis has become increasingly popular for synthesizing multiple sources of clinical evidence. It provides the ability to compare multiple clinical interventions where head-to-head trials are not always available by combining direct and indirect evidence from a network of clinical trials. By doing so, it produces less biased and more precise intervention efficacy estimates. While Bayesian and frequentist methods often yield similar results, the two approaches are fundamentally different in theoretical principles and more importantly require different interpretation of the results. The major limitation of NMA is that NMA results hinge on the inherent statistical assumptions of the NMA and the quality of the evidence used in the NMA. The inherent statistical assumptions are strict and often difficult to satisfy, and the quality of evidence used in the NMA are often difficult to uphold. Multiple requirements need to be met for the results to be sound and useful. Therefore, we recommend a thorough, careful, and conservative approach to interpreting and evaluating the results of an NMA. We also recommend using big data analysis techniques to integrate IPD into the NMA to improve the overall quality and precision of the NMA.


Wireless sensor network deployment should be optimized to maximize network coverage. The D-S evidence theory is an effective means of information fusion that can handle not only uncertainty and inconsistency, but also ambiguity and instability. This work develops a node sensing probability model based on D-S evidence. When there are major evidence disputes, the priority factor is introduced to reassign the sensing probability, with the purpose of addressing the issue of the traditional D-S evidence theory aggregation rule not conforming to the actual scenario and producing an erroneous result. For optimizing node deployment, a virtual force-directed particle swarm optimization approach is proposed, and the optimization goal is to maximize network coverage. The approach employs the virtual force algorithm, whose virtual forces are fine-tuned by the sensing probability. The sensing probability is fused by D-S evidence to drive particle swarm evolution and accelerate convergence. The simulation results show that the virtual force-directed particle swarm optimization approach improves network coverage while taking less time. Wireless sensor networks (WSNs) are commonly employed in harsh, high-risk, and distant unmanned environments. The sensor nodes in WSNs installed underwater  are sometimes difficult to identify, and often swing or drift with the influence of ocean currents, waves, weather, ships, fish, and other factors. Physical causes of displacement or failure also have an impact on underwater acoustic detection equipment. Furthermore, monitoring signals attenuate quickly in an underwater environment, and different types of noise interference might occur often, resulting in quite significant energy loss of sensor nodes. As a result, while deploying wireless sensor network nodes, it is important to consider not only network coverage, but also the node displacement distance.Many heuristic techniques, such as the genetic algorithm , the fisR around the sensor node, the D-S evidence theory is applied. When there is a considerable evidence conflict, the typical D-S evidence theory aggregation rule does not conform to the real circumstances, and the result is not convincing. To address the concern, the priority factor is introduced to reassign the sensing probability.A sensing probabilistic model is constructed. To aggregate the sensing probability in the circular area with radius A virtual force-directed particle swarm optimization algorithm with sensing probabilities aggregated by D-S evidence (VF-PSO-DS for short) is proposed. The approach improves the virtual force-directed particle swarm optimization (VF-PSO) algorithm by considering sensing probability. The sensing probability is aggregated by D-S evidence, which guides the particle swarm evolution process and accelerates convergence.In this paper, a novel node deployment optimization strategy for WSNs is suggested. The virtual force-directed particle swarm (VF-PSO) algorithm and the D-S evidence theory are employed in the strategy. We provide the following contributions:The rest of this paper is organized as follows. Recently, deployment optimization in WSNs has been discussed by many researchers. Most of them consider this problem under different constraints and deployment goals. Dou et al. , for exaThe virtual force (VF) approach has been a prominent WSN node deployment strategy for the past 15 years. In 2007, Wang et al.  investigThe application of D-S evidence theory in node deployment for WSNs has long been a hot topic. Zhao et al.  sought tThe Dempster–Shafer evidence theory, often known as the D-S evidence theory, is a widely used technique in the field of multi-sensor information fusion. D-S evidence theory, which is an inexact derivation of probability theory and is based on Bayesian reasoning, can work without a priori information and conditional probability . D-S eviLet The aggregation formula of n pieces of evidence isA basic probability assignment function of It can be seen from Formula (4) that if If According to Equation (5), the other If there is a conflict between one piece of evidence and other pieces of evidence, but the result of data fusion can be expressed, the denominator of zero will appear in the aggregation rules, resulting in the inability to use the D-S aggregation rules. Therefore, we should avoid the inconsistent results of the D-S evidence theory when it conflicts and synthesizes multiple pieces of evidence.The priority factor is usually a correction method used in system decision-making. In this paper, the priority factor method is introduced into evidence aggregation rules, and the evidence provided by information sources is divided according to the priority level, to further solve the problems of invalidation and conflict in the D-S evidence theory aggregation rules.If there are i = 1,2,…L) is defined to indicate the priority level of each piece of evidence. The priority factor is just a symbol. Whenever a piece of evidence is listed as the most important piece of evidence, the priority factor is assigned to i = 1,2,…L) is Considering that different sources of information provide different priority levels of evidence, the priority factor j = 1,2,…i − 1,i + 1,…, n), and its relationship is as follows:To determine the priority factor of evidence, it is necessary to define the amount of self-conflict in the evidence. The amount of self-conflict of the evidence is the sum of conflicts between evidence i = 1,2,…The steps to determine the priority factor are as follows: First, all evidence pieces should be collected, and the self-conflict amount i = 1,2,…L) is determined, and the value range of According to the above definition, the priority factor method of evidence synthesis is introduced. Firstly, the information about each piece of evidence is collected, and then the self-conflict quantity of evidence is calculated. According to the self-conflict quantity, the priority factor When the number of pieces of evidence This method uses conflicting evidence f(A), andIn Equation (7), the basic probability assignment of conflict evidence is expressed in By using a voting method, the basic probability assignment with highly conflicting evidence is weighted, so n, while the initial value of the threshold In Equation (8), all supported propositions are counted. If the given threshold According to the synthesis method determined above, the method for the synthesis of new evidence again is as follows:Step 1. The two pieces of evidence of priority factors Step 2. Calculate the conflict value between them.The amount of mutual conflict calculated by the evidence Therefore, the method of introducing the priority factor is used in evidence synthesis, which is shown asStep 3. Take the next piece of evidence Step 4. Use the above method to continuously synthesize the newly generated priority factor. When the priority factor is The synthesis method based on the evidence classification strategy is the evidence synthesis method with a priority factor, and its application process is shown in If the two-dimensional plane monitoring area By introducing the D-S evidence theory of the priority factor to synthesize perception probability, the probability that this pixel is perceived by a wireless sensor network is as follows:The evidence is aggregated only if the sensing probability There are Whether the In this paper, the coverage of the monitoring area By calculating the sensing probability of pixels in a circular area with the sensor node Assuming any pixel The force direction points from the pixel The resultant force in the X-axis direction is The goal of deployment optimization is to maximize aware coverageThe constraints are listed as follows:It can be seen from (21) that The distribution of pixel set {Particle swarm optimization (PSO) regards each sensor node as a massless particle flying at a certain speed in the search space. This speed is dynamically adjusted according to the flight experience of itself and its companions. Reference  discusseThe longitudinal moving distance The algorithm steps are as follows:Step 1. Each wireless sensor node in the area Step 2. If the wireless sensor node Step 3. By introducing the D-S evidence theory of the priority factor, the sensing probability Step 4. Using the particle swarm optimization algorithm, calculate the optimal distance Step 5. If the new position to be moved is outside the monitoring area Step 6. If the preset number of cycles is reached, the algorithm ends. Otherwise, go to step 1 and start the next moving process.Python language was used to program the above deployment model. It is assumed that The parameters of the VF algorithm are as follows: To prevent wireless sensor nodes from moving out of the monitoring area The fixed position of the sensor node after the initial random arrangement is shown in To further verify the performance of the three algorithms, the convergence rates were compared through experiments, as shown in One hundred independent simulation experiments were conducted. The average effective coverage, average calculation time, and average iteration times of the three algorithms are shown in To verify the adaptability of the proposed VF-PSO-DS strategy to hot spots and obstacles, it is assumed that the area to be measured contains one hot spot and several obstacles, and the wireless sensor network contains 16 fixed sensor nodes and 80 mobile sensor nodes, while other network parameters and algorithm parameters remain unchanged. The initial deployment of the network is shown in The stability of the deployment strategy was tested for the case of the number of sensor nodes increasing. At each of the values of In this paper, a sensing probability model of nodes is suggested and the sensing probability is aggregated by D-S evidence. Because the traditional D-S evidence theory is inconsistent with the actual situation and the result is invalid when there is a large evidence conflict, the priority factor is introduced to reassign the sensing probability in the part where the evidence conflict occurs. A virtual force-directed particle swarm optimization algorithm based on D-S evidence theory (VF-PSO-DS) for node deployment is proposed. In this approach, the effective coverage rate of the network is taken as the optimization goal, the global optimal scheme is searched for by particle swarm optimization, and the virtual force fine-tuned by sensing probability given by D-S evidence is employed to guide the particle evolution and speed up the convergence. Experimental results show that the virtual force-directed particle swarm optimization approach can quickly and effectively achieve global optimization. Compared to the virtual force algorithm and the particle swarm optimization algorithm, the virtual force-directed particle swarm optimization approach takes less time, converges faster, and increases the sensing coverage of wireless sensor networks.


Polypharmacy use in older adults is increasing and sometimes leads to poor health outcomes. The influence of health literacy in managing polypharmacy and making decisions about stopping medication has received limited attention.A mixed methods design was used to measure and investigate the influence of health literacy in the management of polypharmacy and decisions about deprescribing. Phase 1 involved two cross-sectional surveys, one with older adults using five or more medications and the other with general practitioners (GPs).Older adult health literacy was measured using the All Aspects of Health Literacy Scale. Phase 2 employed individual interviews with both older adults and GPs and further explored the reported use of health literacy in practice. SPSS version 24 was used to conduct descriptive statistical analysis of the Phase 1 survey responses and Phase 2 interviews were analyzed using thematic analysis with the assistance of NVivo 12.Phase 1 survey responses were received from 85 GPs and 137 older adults. Phase 2 interviews were conducted with 16 GPs and 25 older adults. Phase 1 results indicated that self-reported older adult health literacy was high, and that GPs believed older patients could engage in decisions about deprescribing. Phase 2 findings showed that older adults developed and employed complex health literacy practices to manage medications between consultations; however, few reported using their health literacy skills in consultations with their GPs. GPs noted that older adult involvement in decision-making varied and generally thought that older adults had low health literacy.HLRP: Health Literacy Research and Practice. 2023;7(1):e14–e25.]Older adults reported using health literacy practices in the management of their sometimes-complex medication regimens. However, the role of health literacy in deprescribing decision-making was limited. The mixed methods approach allowed greater insight into older adult and GP practices that influence the acquisition and use of health literacy. [This report explores health literacy in the use of multiple medications and decisions to stop using medication/s in older age. Older adults reported good heath literacy and practiced many health literacy skills in the management of their medications. However, they did not always report the use of their health literacy skills when discussing their medications with their family doctor. Polypharmacy, commonly described as the use of five or more medications, is increasingly prevalent, especially among older adults . PolyphaGeneral practitioners (GPs) do most medication prescribing in the community setting , are truMany factors influence older adult participation in shared decision-making including limited time in consultations, attitudes of deference to medical professionals, and GP training . It is kThe risk of low health literacy is high in older adults and is associated with lower cognition and male gender . Those wNutbeam  has defiThe criteria from O'Cathain et al.  for repoMixed methods research allows multiple types of evidence to be gathered and integrated resulting in a greater depth of knowledge . Chinn aFigure 1 describes the design of the study, which included GPs and older community living adults age 65 years or older who were taking five or more medications. Two phases were conducted, each with two study arms. In both study phases, older adults were included if they were living in the community and using five or more medications. There were no exclusion criteria applied to GPs.This study used a sequential explanatory approach . The finIn the first phase, two cross-sectional surveys were conducted, one with GPs and the other with older adults. Surveys were completed between October 2015 and November 2016. As no validated tool existed, a 21 item GP survey tool was developed to explore attitudes toward deprescribing. This was based on a review of the available deprescribing literature . The surThe older adult survey included two validated tools: the Patients' Attitude Towards Deprescribing (PATD) questionnaire  and the In the second phase, individual interviews were conducted with GPs and older adults. Separate interview question guides were developed for both GPs and older adults based on the analysis of the phase one results. Interviews, conducted between January 2018 and May 2019, were recorded with permission, transcribed verbatim, and pseudonyms were used to ensure participant anonymity.The study was carried out in an area covering both urban and semi-rural areas south of Sydney, Australia. The demographic profile of the study area regarding the percentage of people older than age 65 years and provision of and utilization of GP services is similar to Australia's as a whole .A convenience sample of both GPs and older adults was gathered for Phase 1. GPs were recruited via their practices while older adults were recruited via community pharmacies and seniors' community groups. In Phase 2, GPs were recruited via professional networks and snowballing. Older adults who had participated in the earlier survey and were willing to be interviewed were included and additional individuals were contact via snowballing. Sample size for the qualitative phase was not pre-determined. However, data collection ceased when saturation was achieved in both groups.The study was approved by the University of Wollon-gong and Illawarra Shoalhaven Local Hospital District and the Medical Human Research Ethics Committee (HREC No: 15_086).p value reached ≤ .05.Descriptive statistical analysis was conducted using SPSS version 24 for the Phase 1 responses. Statistical significance was assumed where a Thematic analysis  was usedThe final integrated analysis of Phase 1 and Phase 2 explored complementary, convergent, or divergent patterns or findings . Both foIn Phase 1, 85 GPs and 187 older adults responded to their respective surveys. Fifty older adult responses were excluded because respondents did not meet the inclusion criteria for age and/or number of medications used. Data quality was high across both surveys with missing responses for individual items ranging from 0.0% to 5.1%.Table 1 andTable 2. The median GP age was 52 years and 57% were male. Almost one-half (47.6%) were experienced practitioners. The median age of older adults was 76 years and 60.5% were female. They were taking a median of seven medications. Just more than one-half (51.1%) had completed a form of post-high school training. In phase 2, 16 GPs and 25 older adults were interviewed. Interviews ranged in length from 17 to 53 minutes.The characteristics of the survey respondents are described in GP and older adult survey results reporting attitudes toward deprescribing have been reported elsewhere Table 3).Most GPs (84.7%) agreed that their older patients could engage in decision-making. Almost two-thirds agreed that they would ask patients their preference when deciding to continue or deprescribe a medication. GPs' views on their older patients' preferences for involvement in decisions varied with almost one-half agreeing that their older patients preferred to defer decision-making to them. Most GPs reported that they could identify when their older patients found it difficult to understand the rationale for deprescribing. This suggests that they were assessing, in a practical sense what could be described as the health literacy level of their older patients. Many (68.2%) GPs believed they had the ability to provide and communicate risk and benefit information about deprescribing with their older patients (Table 4). Respondents with lower overall AAHLS scores were significantly more likely to be in the group using 10 or more medications .Older adults generally reported high health literacy scores, especially on the functional (18–20) and communicative health literacy domain questions (19–23). Scores for the critical health literacy domain were more varied (24–27)  and the summed overall scores . In contrast, there was a negative correlation between both functional  and communicative health literacy scores  and feeling that one or more medications may no longer be needed. Significant correlations were weak to moderate  correlated with higher scores on the critical health literacy subscale (moderate .Table 5). All of these factors are likely to be supportive of older adults' engaging with their health care provider to discuss deprescribing as a possible medication management strategy.It is notable that higher overall AAHLS scores were positively correlated with understanding the reasons for medications being prescribed, participating in the decision-making process, recalling a medication review, and believing there was enough time to discuss medication concerns during consultations (n = 9) had worked for more than 21 years. The majority were male (n = 10) and worked in low socioeconomic areas (n = 10). The age range of the older adults was 67 to 95 years with a median age of 79. Most participants used 5 to 9 medications (n = 16), and 14 lived in a low socioeconomic area.In phase 2, 16 GPs and 25 older adults participated. Most GPs .The health literacy demands placed on older adults were recognized by some GPs. For example, they acknowledged that generic medications often caused confusion and that medication information was limited when using blister packaged medications because the packs only included the name and dose of the medication. Also, patients did not always have medication changes explained to them during hospitalizations or specialist consultations.“I would imagine most people can juggle you know two or three medications reasonably easily but, obviously the more you have, the harder it potentially becomes and the more room for error there is.” –GP 13Most GPs expressed or implied concern about their older patients' health literacy across the range of health literacy domains: functional, communicative, and critical. They noted their older patients' difficulty in following instructions, difficulty in understanding risk versus benefit and lack of ability to discern appropriate medication information. This led some to be cautious about deprescribing because of concerns that medication changes would result in confusion.Three themes were identified when analyzing the older adult qualitative data. These included: information gathering about medications, the use of health literacy skills in everyday medication management, and health system barriers to health literacy accumulation and application.Information gathering about medications. Finding information was a significant task undertaken by older adult participants. This information increased medication-related health literacy. A range of sources were accessed including their GP, specialists, pharmacists, practice nurses, patient information leaflets, the internet and medication information available from health magazines and books, as well as from family and friends. They also gathered their own autobiographical information regarding the history of their medication use, what benefits they experienced, problems or side effects.The types of information sought included details about why each medication was required and how each medication worked and its potential side effects, including interactions between other medications. Participants also accessed information about administration such as when or how to take their medication, how best to store it and consequences associated with missing or changing dosages or changing medication types.“Well. . .you pick up Google, don't you? And you look for reliable—as far as you know—thing and read it. But it gives me the basis for asking a question if I wanted a question to ask. Not believing everything they say by any means but giving me a bit more understanding.” –Participant, age 87 yearsDespite demonstrating significant knowledge of their own medications, older adults understood their GPs held specific medical knowledge and expertise and valued this more than their own. This view sometimes undermined their confidence to share their own knowledge and most chose to remain passive and trusting in decision-making.“Well, I read up on things. I've got one or two medical books. . .I do not ask him if I should be stopping any minute. I trust that he knows best because I'm not a medical man, am I?” –Participant, age 86 yearsApart from their GP, the internet was the most common source of medication information accessed. This was especially notable given their median age (76 years) and that GPs did not believe their older patients were accessing the internet. Some demonstrated the use of critical health literacy skills to assess the credibility, suitability, and usability of information to their health context. One of the outcomes of this increased knowledge was the ability to engage with their doctors, using their understanding of their own information needs to frame further questions during consultations.Applying health literacy to manage medications.The participants noted the application of their health literacy skills in the many activities that they undertook to manage their medications daily. These included keeping their medications organized, getting prescriptions, collecting supplies, and setting reminders to take medications on time (often scheduling multiple doses per day and often around other activities). Additionally, participants actively monitored the effect of their medication use, including observing for side effects, improvements in their symptoms or outcomes following deprescribing, and communicated information back to their GPs. Because of their many medications and the frequent changes to regimens this was not an insignificant task. However, this monitoring and reporting back was important as it enabled their prescribers to find an acceptable balance of benefit versus side effects and had the potential to facilitate shared decision-making.“It depends on the extent of the side effects. I mean you have to decide whether it was better to change”. . .“cause the side effects may be just as fatal as the medicine sort of thing.” –Participant, age 88 yearsParticipants also indicated that they actively weighed up the current usefulness of their medications as a resource to advance their goals to maintain their health, independence and to help extend their lives, against their assessment of risk in terms of current or future side effects.Health system barriers to the accumulation and use of health literacy.The development of older adult health literacy skills was not always supported during consultations. Older adults reported not always being told relevant information about their medications or leaving consultations unsure of the meaning of explanations. They noted they did not always ask the questions they needed to about their medications because they trusted the expertise of their GP and specialists. Some did not believe that their GP had enough time to provide them with adequate information about medications. Those participants who were frail were no longer able to access their GP independently, resulting in less frequent access to their GP as a source of medication information.“I think they're time-poor. They're pushing you through, so you really don't get very much information. And they're the doctor . . . but I always do my own checking up.” –Participant, age 67 yearsThe participants' experience of medication reviews varied. Only six described regular, comprehensive reviews, where they were offered opportunities to discuss their medications. Others were unsure if their medications had ever been comprehensively reviewed and had limited opportunities to share their treatment preferences to inform decision-making.“We talk about the medications . . . she sort of runs through them all, but she doesn't necessarily talk about them all, you know what I mean? I think she just sort of seems to accept that if things are going okay that we'll stick with that.” –Participant, age 88 yearsThis study measured older adult health literacy and explored its use in the management of polypharmacy and decision-making about deprescribing. Self-reported health literacy was high, especially functional and communicative health literacy. The distribution of scores across the three domains was similar to that seen in a study of chronically ill patients conducted in the Netherlands using the Functional Communicative and Critical Health Literacy Scale .The subsequent qualitative Phase added further details about the type and sources of information accessed by older adults to inform their medication related health literacy. Despite older adults demonstrating significant medication related knowledge and practicing complex health literacy skills between consultations they were less likely to report using their health literacy skills in consultations and GPs generally doubted the health literacy capability of their older patients.Previously, the association between health literacy and older adult attitudes to polypharmacy use or deprescribing had been investigated finding no association . SchiøtzOlder adults demonstrated that they had built their health literacy capacity over time and from a range of professional and lay sources, including the internet. This finding supports broader conceptions of health literacy which acknowledge that health literacy is co-produced within social relations  and thatEarlier deprescribing studies suggested that GPs believed their older adults had a poor understanding of their medications due to old age or a lack of education . HoweverDespite demonstrating considerable knowledge of their own medications, many older adults trusted their GP's greater expertise and remained passive in consultations. Others demonstrated that their knowledge allowed them to be more empowered to enter discussions with their GP about their medications. Differences in critical health literacy capabilities may explain variability in involvement. GPs may interpret that older adults who are passive in consultations are lacking in health literacy skills. This, along with a general perception that older adults have deficient health literacy skills, may deter them from initiating shared decision-making, instead assuming decision-making authority.Exploring and explaining decisions are opportunities to improve older adults' acquisition of health literacy capacity. Most GPs reported that they were confident to explain risks and benefits of medication use and deprescribing to their older patients. However, qualitative studies conducted elsewhere have found that GPs are uncertain of the risks/benefits of ongoing use of medications, especially in the case of preventatives, such as statins and they find it difficult to communicate risk/benefit information to their older patients . FurtherThis study has demonstrated the usefulness of a mixed method approach to exploring health literacy. Viewed in isolation, the quantitative analysis would have suggested a much more positive view of the association of good health literacy with aspects likely to be supportive of deprescribing such as participation in shared decision-making. However, the qualitative analysis allowed insight into the practical application of health literacy, revealing many barriers to its application when considering deprescribing and polypharmacy use.The GP survey tool has not yet been validated and this may have affected the reliability of the findings in the initial GP quantitative arm. However, the GP survey results were also considered in light of the later findings from the qualitative arm and situated in relation to the current deprescribing literature. The samples across the quantitative arms of the study were small and based on convenience samples only. Most participants lived or practiced in an urban area, and it is not known if the results would be similar in rural or remote areas of Australia.Only two participants in this study appeared to be in a frail condition and their health literacy practices were constrained. Demands on health literacy skills are likely to increase as frailty increases because treatments become more complex and varied and the outcomes of decisions are less certain. Further research is required with a larger sample to understand how best to support older adults who are in a frail condition to maintain and apply their health literacy skills and to support GPs as they seek to support shared decision-making when considering deprescribing.Patient education materials are available to assist older adults as they consider deprescribing; however, the majority have been found to be difficult for average readers to comprehend and they do not present information in a way that would be sufficient to inform consent . This prAppropriate written and verbal communications about medications and providing information on trusted websites are important to empower all older adults to understand and access the information they require to participate in medication decisions. Involving older adults in regular reviews of the ongoing appropriateness of current medications may allow opportunities to discuss and plan deprescribing pro-actively without the time pressures of regular consultations.Significant demands are placed on the health literacy capabilities of older adults who use polypharmacy. A range of complex health literacy practices were used between consultations to independently manage their medications. However, health literacy was found to play a limited role in medication-related decisions. GPs were largely unaware of the extent of their older patient's health literacy capacity. Few opportunities were described to develop and to apply health literacy capabilities during discussions of polypharmacy use and/or deprescribing in consultations. These findings emphasize that GPs can do more to understand their older patient's health literacy capacity, refrain from assuming poor health literacy, and explore unmet information needs in the context of deprescribing.


To produce graduates with strong knowledge and skills in the application of evidence into healthcare practice, it is imperative that all undergraduate health and social care students are taught, in an efficient manner, the processes involved in applying evidence into practice. The two main concepts that are linked to the application of evidence into practice are “evidence‐based practice” and “evidence‐informed practice.” Globally, evidence‐based practice is regarded as the gold standard for the provision of safe and effective healthcare. Despite the extensive awareness of evidence‐based practice, healthcare practitioners continue to encounter difficulties in its implementation. This has generated an ongoing international debate as to whether evidence‐based practice should be replaced with evidence‐informed practice, and which of the two concepts better facilitate the effective and consistent application of evidence into healthcare practice.The primary objective of this systematic review was to evaluate and synthesize literature on the effectiveness of evidence‐informed practice versus evidence‐based practice educational interventions for improving knowledge, attitudes, understanding, and behavior of undergraduate health and social care students toward the application of evidence into practice. Specifically, we planned to answer the following research questions: (1) Is there a difference  between evidence‐informed practice and evidence‐based practice educational interventions? (2) Does participating in evidence‐informed practice educational interventions relative to evidence‐based practice educational interventions facilitate the application of evidence into practice ? (3) Do both evidence‐informed practice and evidence‐based practice educational interventions targeted at undergraduate health and social care students influence patient outcomes ? (4) What factors affect the impact of evidence‐informed practice and evidence‐based practice educational interventions ?We utilized a number of search strategies to identify published and unpublished studies: (1) Electronic databases: we searched Academic Search Complete, Academic search premier, AMED, Australian education index, British education index, Campbell systematic reviews, Canada bibliographic database (CBCA Education), CINAHL, Cochrane Library, Database of Abstracts of Reviews on Effectiveness, Dissertation Abstracts International, Education Abstracts, Education complete, Education full text: Wilson, ERIC, Evidence‐based program database, JBI database of systematic reviews, Medline, PsycInfo, Pubmed, SciELO (Scientific Electronic Library Online), and Scopus; (2) A web search using search engines such as Google and Google scholar; (3) Grey literature search: we searched OpenGrey (System for Information on Grey Literature in Europe), System for information on Grey Literature, the Society for Research on Educational Effectiveness, and Virginia Henderson Global Nursing e‐Repository; (4) Hand searching of journal articles; and (5) Tracking bibliographies of previously retrieved studies. The searches were conducted in June 2019.We planned to include both quantitative  and qualitative primary studies , that evaluate and compare the effectiveness of any formal evidence‐informed practice educational intervention to evidence‐based practice educational intervention. The primary outcomes were evidence‐informed practice and evidence‐based practice knowledge, attitudes, understanding, and behavior. We planned to include, as participants, undergraduate pre‐registration health and social care students from any geographical area.Two authors independently screened the search results to assess articles for their eligibility for inclusion. The screening involved an initial screening of the title and abstracts, and subsequently, the full‐text of selected articles. Discrepancies were resolved through discussion or consultation with a third author. We found no article eligible for inclusion in this review.No studies were found which were eligible for inclusion in this review. We evaluated and excluded 46 full‐text articles. This is because none of the 46 studies had evaluated and compared the effectiveness of evidence‐informed practice educational interventions with evidence‐based practice educational interventions. Out of the 46 articles, 45 had evaluated solely, the effectiveness of evidence‐based practice educational interventions and 1 article was on evidence‐informed practice educational intervention. Hence, these articles were excluded as they did not meet the inclusion criteria.There is an urgent need for primary studies evaluating the relative effectiveness of evidence‐informed practice and evidence‐based practice educational interventions targeted at improving undergraduate healthcare students' competencies regarding the application of evidence into practice. Such studies should be informed by current literature on the concepts  to identify the differences, similarities, as well as appropriate content of the educational interventions. In this way, the actual effect of each of the concepts could be determined and their effectiveness compared. We did not find any studies that met our inclusion criteria, therefore we cannot draw any conclusions regarding the relative effectiveness of the two approaches. The evidence is current to June 17, 2019.1.3The effective application of the best evidence into healthcare practice is strongly endorsed, alongside a growing need for healthcare organizations to ensure the delivery of services in an equitable and efficient manner. Existing evidence shows that guiding healthcare practice with the best available evidence enhances healthcare delivery, improves efficiency and ultimately improves patient outcomes. Nevertheless, there is often the ineffective and inconsistent application of evidence into healthcare practice.What is the aim of this review?This Campbell systematic review examines the effectiveness of evidence‐informed practice and evidence‐based practice educational interventions for improving knowledge, attitudes, understanding, and behavior of undergraduate health and social care students toward the application of evidence into practice.The two main concepts that have been associated with the application of evidence into healthcare practice are “evidence‐based practice” and “evidence‐informed practice.” This review assesses the relative effectiveness of these two approaches, specifically in relation to improving knowledge, attitudes, understanding, and behavior of undergraduate health and social care students. In addition, we aimed to assess the impact of evidence‐informed practice and/or evidence‐based practice educational programmes on patient outcomes. Examples of patient outcome indicators that we would have assessed had eligible studies been found are: user experience, length of hospital stay, nosocomial infections, patient and health practitioner satisfaction, mortality, and morbidity rates.1.4We planned to include both quantitative and qualitative studies aimed at improving knowledge, attitudes, understanding, and behavior of undergraduate pre‐registration health and social care students from any geographical area. Studies whose participants were registered health and social care practitioners and postgraduate students were excluded.We planned to include studies that were published between 1996 and June 2019. No limit was placed on the language of publication.1.5A total of 45 full‐text articles on evidence‐based practice educational interventions and one full‐text article on evidence‐informed practice educational intervention were screened for their eligibility for inclusion. However, we identified no studies examining the relative effectiveness of evidence‐informed practice versus evidence‐based practice educational interventions. As a result, we are unable to answer the question as to which of the two concepts better facilitates the application of evidence into healthcare practice.1.6Whilst evidence suggests that evidence‐informed practice can be effective (compared to a no‐intervention control) in improving student outcomes, we are unable to conclude which approach better facilitates the application of evidence into practice.1.7The review authors searched for studies published up to June 2019.22.1Over the past three decades, there has been increasing attention on improving healthcare quality, reliability, and ultimately, patient outcomes, through the provision of healthcare that is influenced by the best available evidence, and devoid of rituals and tradition .Two main concepts have been associated with the application of evidence into healthcare practice: “evidence‐based practice” and “evidence‐informed practice.” Evidence‐based practice is an offshoot of evidence‐based medicine; hence, the universally accepted definition of evidence‐based practice is adapted from the definition of evidence‐based medicine, which is “the conscientious, explicit and judicious use of the best evidence in making decisions about the care of the individual patient”  created by Tilson et al. . These aKnowledge referred to learners' retention of facts and concepts about evidence‐informed practice and evidence‐based practice. Hence, assessments of evidence‐informed practice and evidence‐based practice knowledge might assess a learner's ability to define evidence‐based practice and evidence‐informed practice concepts, list their basic principles, or describe levels of evidence.Attitudes referred to the values ascribed by the learner to the importance and usefulness of evidence‐informed practice and evidence‐based practice to inform clinical decision‐making.Understanding referred to learners' comprehension of facts and concepts about evidence‐based practice and evidence‐informed practice.Behavior referred to what learners actually do in practice. It is inclusive of all the processes that a learner would use in the implementation of evidence‐informed practice and evidence‐based practice, such as assessing patient circumstances, values, preferences, and goals along with identifying the learners' own competence relative to the patient's needs to determine the focus of an answerable clinical question.We planned that the mode of delivery of the educational program could be in the form of workshops, seminars, conferences, journal clubs, and lectures (both face‐to‐face and online). It was anticipated that the content, manner of delivery, and length of the educational program may differ in each of the studies that were to be included as there is no standard evidence‐informed practice/evidence‐based practice educational program. Evidence‐informed practice and evidence‐based practice educational interventions that are targeted toward health and social care postgraduate students or registered health and social care practitioners were excluded.2.3Most efforts to apply evidence into healthcare practice have either been unsuccessful or partially successful  between evidence‐informed practice and evidence‐based practice educational interventions?2.Does participating in evidence‐informed practice educational interventions relative to evidence‐based practice educational interventions facilitate the application of evidence into practice ?3.Do both evidence‐informed practice and evidence‐based practice educational interventions targeted at undergraduate health and social care students influence patient outcomes ?4.What factors affect the impact of evidence‐informed practice and evidence‐based practice educational interventions ?Specifically, the review aimed to address the following research questions:44.14.1.1https://doi.org/10.1002/cl2.1015.This review followed standard procedures for conducting and reporting systematic literature reviews. The protocol for this systematic review  was published in July 2019. The protocol is available at: In this review, we intended to include both qualitative and quantitative primary studies (a mixed‐methods systematic review) that compared evidence‐informed practice educational interventions with evidence‐based practice educational interventions.We planned to include quantitative primary studies that used both experimental and epidemiological research designs such as RCTs, non‐RCTs, quasi‐experimental, before and after studies, prospective and retrospective cohort studies.We also planned to include qualitative primary studies that had used descriptive epidemiological study designs. Examples include case series, individual case reports, descriptive cross‐sectional studies, focus groups, and interviews. Furthermore, we intended to include primary studies that have used qualitative approaches such as ethnography, phenomenology, and grounded theory. We planned to discuss the biases and limitations associated with any included study design in relation to the impact it may have on the effectiveness of the intervention.For the primary analysis, the intention was to follow the recommended steps by Sandelowski et al. : to firs4.1.2We intended to include undergraduate pre‐registration health and social care students in higher education (University) from any geographical area. We planned to include undergraduate pre‐registration students studying health and social care programs such as nursing, midwifery, dental hygiene, and dental therapy, dental nurse practice, diagnostic radiography, occupational therapy, operating department practice studies, paramedic practice, social work, and physiotherapy.We planned to exclude studies whose participants were registered health and social care practitioners and postgraduate students.4.1.3The intention was to include primary studies that evaluate and compare any formal evidence‐based practice educational intervention with evidence‐informed practice educational interventions aimed at improving undergraduate pre‐registration health and social care students' knowledge, attitudes, understanding, and behavior regarding the application of evidence into healthcare practice. Such interventions may be delivered via either workshops, seminars, conferences, journal clubs, or lectures (both face‐to‐face and online).Specifically, we planned to include any formal educational intervention that incorporates any or all of the principles and elements of McSherry 's evidenEvidence‐informed practice and evidence‐based practice educational interventions that are targeted at health and social care postgraduate students or registered health and social care practitioners were excluded. We intended to include, as comparison conditions, educational interventions that do not advance the teaching of the principles and processes of evidence‐informed practice, and/or evidence‐based practice in healthcare.4.1.4Outlined below are the primary and secondary outcome measures for this systematic review1.Participants' knowledge about evidence‐informed practice and/or evidence‐based practice.2.Participants' understanding of evidence‐informed practice and/or evidence‐based practice.3.Participants' attitudes toward evidence‐informed practice and/or evidence‐based practice.4.Participants' behavior toward evidence‐informed practice and evidence‐based practice.the use of a standardized questionnaire to evaluate knowledge, attitude, understanding, and behavior toward the application of evidence into practice. Examples of such questionnaires include (1) the Evidence‐Based Practice Belief (EBPB) and Evidence‐Based Practice Implementation (EBPI) scales developed by Melnyk et al. . The EBPunstandardized instruments include self‐reports from study participants and researcher‐administered measures.Since there is no uniform tool for evaluating the effectiveness of evidence‐based practice and evidence‐informed practice educational interventions, we planned that measurement of the above outcomes may be conducted using standardized or unstandardized instruments. Some examples of these instruments include:The intention was to include studies that measure the impact of evidence‐informed practice and/or evidence‐based practice educational programs on patient outcomes. We planned to assess patient outcome indicators such as user experience, length of hospital stays, absence of nosocomial infections, patient and health practitioner satisfaction, mortality, and morbidity rates.No limit was placed on the duration of follow‐up. The rationale was to give room for studies with either short‐ or long‐term follow‐up duration to be eligible for inclusion.We intended to include primary studies from any geographical area. However, due to language translation issues, we planned to include only studies written in English. We also planned that studies whose title and abstracts are in English and meet the inclusion criteria, but the full article is reported in another language would be included, subject to the availability of translation services.To qualify for inclusion in this systematic review, studies must have been published during the period from 1996 (the date when evidence‐based practice first emerged in the literature) , wildcards (such as?), phrase operators (e.g. “”), and truncations (including *). This was done to ensure search precision and sensitivity. In addition, we used three sets of terms for the search strategy: the population, intervention(s), and outcomes. We used limiting commands to narrow the results by date and type of study design. The search was limited to studies published from 1996, which is the year when evidence‐based practice first emerged in the literature. No limit was placed on the language of publication, however, due to language translation issues, eligible studies whose full texts were not in English might have been included only if there were available language translation facilities.Targeted population: nurs* OR physio* OR “occupa* therap*” OR “dental Hygiene” OR “undergraduate healthcare student*” OR “undergraduate social care student*” OR baccalaureat* OR “social work” OR dent* OR BSc OR student* OR “higher education” OR “undergrad* nurs* student*”Intervention: evidence‐informed* OR evidence‐based* OR “evidence‐informed practice” OR “evidence‐based practice” OR EBP OR EIP OR “evidence‐informed practice education” OR “evidence‐based practice education” OR “evidence into practice” OR evidence‐informed near. practice teaching learning OR evidence‐based near. practice teaching learningOutcomes: “knowledge, attitudes, understanding and behavio* regarding EBP” OR “knowledge near. attitudes understanding behavio* regarding EIP OR “Knowledge of evidence‐informed*” OR “knowledge of evidence‐based*” OR “patient outcome*” OR outcome*Study design/type: trial* OR “randomi?ed control trial” OR “qua?i‐experiment*” OR random OR experiment OR “control* group*” OR program OR intervention OR evaluat* OR qualitative OR quantitative OR ethnograpy OR “control* study” OR “control* studies” OR “control* design*” OR “control* trial*” OR “control group design” OR RCT OR rct OR “trial registration”Below are examples of the search terms used in the current review:4.2.2We exported the full set of search results directly into an Endnote X9 Library. Where this was not possible, search results were manually entered into the Endnote Library. The Endnote library made it easier to identify duplicates and manage references.4.2.3The search to identify eligible studies was initially carried out in June 2018, and then a repeat search was conducted in June 2019. We utilized a number of strategies, to identify published and unpublished studies that meet the inclusion criteria described above. These strategies are outlined below.1.An electronic database search was conducted using the following databases.Academic Search completeAcademic search premierAMEDAustralian education indexBritish education indexCampbell systematic reviewsCanada bibliographic database (CBCA Education)CINAHLCochrane LibraryDatabase of Abstracts of Reviews on EffectivenessDissertation Abstracts InternationalEducation AbstractsEducation completeEducation full text: WilsonERICEvidence‐based program databaseJBI database of systematic reviewsMedlinePsycInfoPubmedSciELO (Scientific Electronic Library Online)ScopusThe following electronic searches were conducted to identify eligible studies.2.A web search was conducted using the following search engines.GoogleGoogle Scholar3.A gray literature search was conducted using the following databases.OpenGrey (System for Information on Grey Literature in Europe)System for information on Grey LiteratureThe Society for Research on Educational EffectivenessVirginia Henderson Global Nursing e‐RepositorySupporting Information Appendix Worldviews on Evidence‐Based Nursing Journal, the British Medical Journal, and the British Journal of Social Work.Hand searching: the table of contents of three journals were hand‐searched for relevant studies. The journals include the Tracking bibliographies of previously retrieved studies and literature reviews: we screened the reference list of previously conducted systematic reviews, meta‐analysis, and primary studies for relevant studies.The following strategies were also used to identify eligible studies.4.34.3.1First, we exported search results from the various databases and search engines into an Endnote X9 software. Second, we searched for and removed duplicates using the Endnote software. Third, we exported search results from Endnote into Covidence (a web‐based software platform that streamlines the production of systematic reviews) for the screening of the search results. Two authors independently screened the titles and abstracts for relevant papers. The full text of potentially relevant papers was subsequently assessed by two independent authors for inclusion. Disagreements were resolved through discussion. Where disagreements persisted, a third reviewer was contacted. We selected papers that were published during the period from 1996 (the date when evidence‐based practice first emerged in the literature)  would have assessed its methodological validity using standardized critical appraisal instruments from the Joanna Briggs Institute Meta‐Analysis of Statistics Assessment and Review Instrument (JBI‐MAStARI). We would have used the JBI‐MAStARI checklist for case‐control studies, the checklist for case reports, the checklist for cohort studies, the checklist for quasi‐experimental, the checklist for RCTs, and the checklist for analytical cross‐sectional studies. Disagreements between authors would have been resolved through discussion; if no agreement could be reached, a third author was to be consulted.Data would have been extracted from included papers using standardized data extraction form for intervention reviews for RCTs and non‐RCTs developed by the Cochrane Collaboration. We would have extracted information relating to study design, interventions, population, outcomes that are of significance to the review questions, and specific objectives, and methods used to assess the impact of evidence‐informed practice/evidence‐based practice educational interventions on patient outcomes. Supporting Information Appendix 4.3.3Two authors (either R. M. and S. H. or E. A. K. and J. B. S.) would have independently assessed eligible studies for risk of bias. This would have been done using the Cochrane Collaboration's Risk of Bias tool , no attrition, or no evidence of differential attrition would have been considered as having a low risk of bias. We planned to record the use of Intention to Treat (ITT) analysis and methods of account for missing data .We intended to assess studies for reporting bias to determine whether there are inconsistencies between measured outcomes and reported outcomes. Studies would have been considered as having a low risk of bias if the results section of publications clearly show that all pre‐specified outcomes are reported.4.3.4g, which is calculated using the following formula:For continuous data, where outcomes on the same scale are presented, we planned to use mean difference, with a 95% confidence interval. However, where outcome measures differ between studies, we would have used the standardized mean difference as the effect size metric based on Hedges' For dichotomous data, we planned to calculate the risk ratio  for the occurrence of an event. For meta‐analysis, we planned to convert risk ratios to the standardized mean difference, using David Wilson's practical effect size calculator. We intended to use meta‐regression to assess the impact of moderator variables on the effect size of interventions. We planned to conduct moderator analysis if a reasonable number of eligible research articles were identified and if the required data is presented in the report.For studies with one control group versus two or more intervention groups, and all the interventions are regarded as relevant to the study, we planned to use the following options: (1) if the intervention groups are not similar, we would have divided the sample size of the control group into two (or more based on the number of intervention groups), and then compared with the intervention groups (2) if the intervention groups were similar, we would have treated the two groups as a single group. Therefore, we would have provided two effect size estimates in this study. This was to ensure that participants in the control group were not “double‐counted”  of participants as units of analysis. we planned that in the event that cluster‐randomized trials  are identified as eligible, we would have used standard conversion criteria as recommended in the Cochrane Handbook  × ICC, where M is the average cluster size and ICC is the Intra‐cluster Correlation Coefficient.The Cochrane Handbook , data analysis would have been conducted based on the available data.However, if data is missing not at random , we planned to impute the missing data with replacement values, and treat these values as if they were observed .4.3.7I2 statistic, which describes the approximate proportion of variation that is due to heterogeneity rather than sampling error. This would have been supplemented by the χ2 test, where a p value < 0.05 indicates heterogeneity of intervention effects. In addition, we planned to estimate and present τ2 along with its CIs, as an estimate of the magnitude of variation between studies. This would have provided an estimate of the amount of between‐study variation. We also planned to use sensitivity and subgroup analyses to investigate possible sources of heterogeneity.We intended to assess heterogeneity through the comparison of factors such as participant demographics, type of intervention, type of control comparators, and outcome measures. We would have assessed and reported heterogeneity visually and by examining the 4.3.8We planned to assess studies for reporting bias to determine whether there were inconsistencies between measured outcomes and reported outcomes. Studies would have been considered as having a low risk of bias if the results section of publications clearly showed that all pre‐specified outcomes are reported.4.3.9We planned to use narrative and statistical methods to synthesize included studies. The synthesis would have focused on calculating the effect sizes of the included studies. We planned to conduct meta‐analysis if our search yielded sufficient  eligible studies that can be grouped together satisfactorily. A logical approach would have been used when combining studies in meta‐analysis. Had we found any eligible studies, decisions on combining studies in meta‐analysis would have been based on two reasons: (1) a sufficient number of eligible studies with similar characteristics, (2) similar characteristics shared by those eligible studies may include the type of intervention and the targeted outcome of the intervention. Had we conducted a meta‐analysis, we planned to use the comprehensive meta‐analysis software developed by Borentein et al. . We woul4.3.10Where there was significant statistical heterogeneity, we planned to conduct subgroup analysis to consider the effects of variables, such as participant's age, geographical area, mode of delivery and content of the educational intervention, and type of study design.4.3.11Unpublished studiesStudies with outlier effect sizesStudies with high risks of biasStudies with missing information Had we found any eligible studies, we planned to conduct sensitivity analysis to determine whether the overall results of data analysis was influenced by the removal of:4.3.12Two authors (either S. H. and R. M. or E. A. K. and J. B. S.) would have independently assessed included qualitative studies for methodological validity using the JBI Qualitative Assessment and Review Instrument (JBI‐QARI). Any disagreements between authors would have been resolved through discussion, if no agreement could be reached, a third author would have been consulted.We planned to extract data from included papers using the Cochrane Collaboration's Data Collection form for Intervention Reviews for RCTs and non‐RCTs . We intended to achieve integration by translating findings from the quantitative review into qualitative results using Bayesian conversion to generate synthesized results.4.3.13Not applicable to the current review.55.1The findings of the current systematic review are presented below.5.1.1Database searches were conducted by a librarian (JH) and another review author (EK). Terms and keywords specific to the Participant (P), Intervention (I), Comparative intervention (C), and the Outcome (O) were used to generate relevant articles. A total of 14,411 citations were identified through a database search (completed on 17 June 2019). A search of additional records including google and google scholar yielded 178 citations. After duplicates were removed, 10708 records were retained for screening. After the screening, a total of 46 full‐text studies were assessed for eligibility. Among the 46 studies, 45 had assessed solely evidence‐based practice educational interventions and 1 paper had assessed the effectiveness of evidence‐informed practice interventions. However, we found no evidence of primary studies that had evaluated and compared the effectiveness of evidence‐informed practice educational interventions with evidence‐based practice educational interventions. A summary of the search results is presented in Figure 5.1.2We did not identify any qualitative nor quantitative study that was eligible for inclusion in this review.5.1.3We evaluated and excluded 46 full‐text articles. The rationale for exclusion was that none of the 46 studies had evaluated and compared the effectiveness of evidence‐informed practice educational interventions with evidence‐based practice educational interventions. Out of the 46 articles, 45 had evaluated solely, the effectiveness of evidence‐based practice educational interventions and 1 article was on evidence‐informed practice educational intervention. Hence, these articles were excluded as they did not meet the inclusion criteria. Refer to the Summary of findings table for further details regarding the characteristics of excluded studies.5.2We evaluated no study for methodological quality.5.2.1We found no study eligible for inclusion in this review.5.2.2We found no study eligible for inclusion in this review.5.2.3We found no study eligible for inclusion in this review.5.2.4We found no study eligible for inclusion in this review.5.2.5We found no study eligible for inclusion in this review.5.3We found no study eligible for inclusion in this review.66.1We did not identify any evidence on the relative effectiveness of evidence‐informed practice and evidence‐based practice educational interventions for improving knowledge, attitudes, understanding, and behavior of undergraduate pre‐registration health and social care students toward the application of evidence into practice.6.2This review is considered an empty review, as we found no study that met all the inclusion criteria. This notwithstanding, the review offers vital evidence concerning the gaps in the literature on the effectiveness of evidence‐informed practice versus evidence‐based practice educational interventions, and whether the two concepts act together or individually to facilitate the application of evidence into healthcare practice. There are a plethora of studies that have evaluated the effectiveness of evidence‐based practice educational interventions on undergraduate pre‐registration health and social care students (this review identified 45 primary studies), however, there is limited evidence on the effectiveness of evidence‐informed practice educational interventions as well as a comparison of the effectiveness of the two concepts.Evidence‐informed practice is evolving, as understanding and expertise increase WHO, . In receTo prepare health and social care students to effectively apply evidence into practice, it is imperative that undergraduate health and social care curricula are designed to enhance progressive knowledge development on the application of evidence into clinical practice. Moreover, students' experiences in the clinical setting must produce opportunities for them to use evidence in patient care decisions as well as at the organizational level to impact patient outcomes.The findings of this systematic review reveal the need for rigorously designed studies that are informed by empirical and theoretical literature on evidence‐informed practice versus evidence‐based practice educational interventions. The characterizations of the processes involved in implementing evidence‐informed practice and evidence‐based practice given in this systematic review offer intervention designs based on existing empirical and theoretical literature, which could be considered in the design of evidence‐informed practice and evidence‐based practice educational interventions.6.3We searched the literature for both quantitative and qualitative studies on the effectiveness of evidence‐informed practice versus evidence‐based practice educational interventions on the knowledge, attitudes, understanding, and behavior of undergraduate pre‐registration health and social care students. Although there is some evidence of studies that have assessed the effectiveness of either evidence‐based practice or evidence‐informed practice educational interventions, our search revealed no evidence of studies that compared the effectiveness of evidence‐informed practice with evidence‐based practice educational interventions. Thus, we are unable to draw any conclusions regarding the effectiveness of these two concepts when compared to each other. The evidence is current to June 17, 2019.6.4We made every effort to minimize bias in this review. A comprehensive search of multiple health and social care databases was conducted, and no limit was applied to publication status or language. The search strategy was developed by an experienced health and social care librarian, in consultation with experts of the review team. The search for papers was done by two review authors . Screening of the search results was also conducted by two independent review authors (E. A. K. and J. B. S.). Disagreements were resolved by another review author (R. M.).The main potential bias of this review is the unlikely event that we have missed any study that evaluated and compared the effectiveness of evidence‐informed practice to evidence‐based practice educational interventions on the knowledge, attitudes, understanding, and behavior of undergraduate health and social care students. However, given our extensive search of the current literature, it is unlikely that we have missed any eligible studies. A limitation of our review is that we are unable to answer our research questions since we did not identify any eligible studies.6.5Currently, there is no primary study that has evaluated and compared the effectiveness of evidence‐informed practice to evidence‐based practice educational interventions. In addition, there is no previous systematic review that has compared the effectiveness of these two concepts. We are, therefore, unable to compare our results with other studies.77.1We are unable to make firm conclusions about the effectiveness of evidence‐informed practice versus evidence‐based practice educational interventions in improving undergraduate health and social care students' knowledge, attitudes, understanding, and behavior toward the application of evidence into practice.Our review reveals extensive research work conducted on the effectiveness of evidence‐based practice educational interventions (see the Summary of findings table). Besides, several educational programs and interventions on evidence‐based practice have been developed for undergraduate health and social care students. However, there is limited evidence on evidence‐informed practice educational interventions and the effects of such interventions. Our finding reaffirms the importance attached to the evidence‐based practice concept and its use in decision‐making regarding health care practice and policy.The purpose of evidence‐based practice to healthcare practice is to provide appropriate healthcare in a timely and effective manner to the patient WHO, . Evidenc7.2There is a need for primary studies evaluating the effectiveness of evidence‐informed practice versus evidence‐based practice educational interventions. Future research should reflect the differences and similarities between these two concepts in the educational interventions to be evaluated. In this way, the actual effect of each of the concepts could be determined and, compared with each other to determine the differences in the effect size.Due to the limited literature on evidence‐informed practice interventions, future research in this area should be informed by a systematic map of the wider literature to determine models of evidence‐informed practice.Content and Systematic Review methodology: Ms. Elizabeth Adjoa Kumah is a registered nurse who has worked mainly in the critical care setting as a nurse supervisor and patient advocate. She has been actively engaged in teaching healthcare students in the clinical setting and serving as a mentor. She has recently completed a Ph.D. Health program, with evidence‐informed practice and evidence‐based practice educational interventions as the area of research focus. She brings knowledge about the content both in terms of teaching healthcare students about the application of evidence into practice and theoretically for improving knowledge of evidence‐informed practice and how it enhances evidence‐based practice skills, attitude, and behavior in the educational setting. Elizabeth is passionate about improving the standard of patient care and patient outcome, which she believes could be achieved by effective and consistent implementation of evidence‐informed practice. She will also contribute to the methodological aspects of the systematic review.Content and Systematic review methods: Professor Robert McSherry will bring both methodological as well as content expertise relating to evidence‐informed practice and the development of teaching programmes to the team. His area of expertise is around evidence‐informed practice, patient safety, quality, and clinical governance using practice development. Practice development is about promoting person‐centered care and approaches, which Rob has integrated effectively within both educational and research programs. He is the co‐author of a book on systematic reviews and has over 30 years of experience as a registered nurse. Robs educational and professional expertise has been recognized and rewarded internationally and nationally. He was awarded the highly prestigious National Teaching Fellow award in the UK in 2011.Content and systematic review methods: Dr. Josette Bettany‐Saltikov will bring significant expertise of Systematic review methods and content to this systematic review, both in terms of knowledge about evidence‐based practice and knowledge about developing educational programs. She has taught systematic review methods to university students at all levels for over 15 years. She has also published a book on how to conduct a systematic review and has been involved in three Cochrane reviews, one of which she led. She has authored a number of systematic reviews on diverse topics published in other journals and has significant experience in developing educational programs from her teaching experience as a university Senior lecturer for 23 years.Content and systematic review methods: Professor Sharon Hamilton will bring expertise in systematic reviewing. She is the director of the Teesside Centre for Evidence‐Informed Practice: A Joanna Briggs Institute Centre of Excellence, and has conducted a number of qualitative and quantitative reviews. Sharon is a registered nurse and has research expertise in the evaluation of clinical interventions.Information retrieval: Mrs. Julie Hogg brings Information retrieval expertise to the team. Julie is an Academic Librarian at Teesside University and will carry out a thorough and systematic search of the literature.Statistical analysis: Mrs. Vicki Whittaker is a very experienced statistician with over 18 years of experience in teaching and advising students and academics on their research projects and clinical trials. She has been involved in data analysis and meta‐analysis of numerous research projects and systematic reviews.Characteristics of excluded studiesThe review team declares no potential conflicts of interest.In this review, we made every effort to identify eligible studies by following the methods outlined in the protocol.In the protocol, we planned to assess whether evidence‐informed practice compared to evidence‐based practice educational interventions improve knowledge, attitudes, understanding, and behavior of undergraduate health and social care students toward the application of evidence into practice. In addition, we aimed to assess the impact of evidence‐informed practice and/or evidence‐based practice educational programs on patient outcomes. Examples of patient outcome indicators that we planned to assess include user experience, length of hospital stay, nosocomial infections, patient and health practitioner satisfaction, mortality, and morbidity rates. However, we could not explore these objectives in the final review because we did not identify any eligible studies for inclusion.Teesside University, UKInternal sourcesThis review forms part of a Ph.D programme, which is supported and funded by the Teesside University, Middlesbrough.Not applicable to the current systematic review, UKExternal sourcesThis systematic review did not receive any form of external support.Supporting information.Click here for additional data file.