Integrated traditional Chinese medicine (TCM) and Western medicine (WM) is a new medical science grounded in the knowledge bases of both TCM and WM, which then forms a unique modern medical system in China. Integrated TCM and WM has a long history in China, and has made important achievements in the process of clinical diagnosis and treatment. However, the methodological defects in currently published clinical practice guidelines limit its development. The organic integration of TCM and WM is a deeper integration of TCM and WM. To realize the progression of "integration" to "organic integration", a targeted and standardized guideline development methodology is needed. Therefore, the purpose of this study is to establish a standardized development procedure for clinical practice guidelines for the organic integration of TCM and WM to promote the systematic integration of TCM and WM research results into clinical practice guidelines in order to achieve optimal results as the whole is greater than the sum of the parts.The online version contains supplementary material available at 10.1186/s40779-023-00481-9. Integrated traditional Chinese and Western medicine is unique to China, and it is also a major feature and advantage of China's medical and health system. The synthesis of integrated traditional Chinese medicine (TCM) with the knowledge of Western medicine (WM) provides the Chinese people with a powerful medical and healthcare system with "Chinese characteristics" , 2.In 2022, the General Office of the State Council of China issued the “14th Five-Year Plan” for the Development of TCM, which clearly named a number of integrated TCM and WM diagnosis and treatment programs that can reflect the "appropriate Chinese and Western medicine" . AccelerIn the past 5 years, more than 50 guidelines or expert consensus statements on integrated TCM and WM have been published each year. Researches show that these still have many methodological defects , 7. TherIntegrated TCM and WM focuses on integrating the advantages of TCM with WM and avoiding their disadvantages, reflecting the combination of a holistic view and systems biology, individualization and precision medicine. The discipline of integrated TCM and WM uses modern language to clarify the scientific connotation of TCM, and provides the possibility for the evidence-based optimization of "standardized production of evidence" and "accurate differentiation of syndromes", as well as the language translation of organic integration of TCM and WM and the mutual recognition of results , 8, 9. TA standard research group was set up, consisting of 17 members from the following centers: Center for Evidence-Based and Translational Medicine, Zhongnan Hospital of Wuhan University , Dongzhimen Hospital, Beijing University of Chinese Medicine, Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Hubei Province TCM Hospital, Center for Evidence-Based Medicine, Lanzhou University, The Chinese Cochrane Center, West China Hospital of Sichuan University, Guangdong Province TCM Hospital. This research team approved the "methodologies and procedures for developing clinical practice guidelines of organic integration TCM and WM" in the China Standardization Association, aiming to standardize the development process of guidelines for integrated TCM and WM, and develop guidelines that can reflect its diagnosis and treatment practice and solve its clinical problems, so promoting the systematic integration of TCM and WM research results into guidelines to achieve optimal results as the whole is greater than the sum of the parts.Inclusion criteria: The full text of the guidelines for integrated TCM and WM published at home and abroad was obtained. Among the titles, "integrated TCM and WM (integrative TCM and WM)" and/or "Combination of WM diseases and TCM syndrome differentiation" words or titles include the proper terms of TCM and WM diagnosis and treatment. For example, "Guidelines for combining TCM with antibiotics to treat diseases". In addition, although the titles of some articles are the integration of TCM and WM, the recommendations only cover TCM, which does not meet the requirements, and such articles need to be excluded.Search strategy: The databases used included PubMed, Web of Science, CNKI, Wanfang Data and VIP database, and the search time limit was from inception to July 2022. The search was carried out using a combination of subject words and free words and adjusted according to the characteristics of each database. References from the included studies were also searched to track the literature that met the inclusion criteria. The Chinese search terms are traditional Chinese medicine, Western medicine, integrated traditional Chinese and Western medicine, Combination of Western medicine diseases and traditional Chinese medicine syndrome differentiation, guidelines, clinical practice guidelines, recommendations, etc. The English search terms are as follows: traditional Chinese medicine, TCM, Western medicine, integrative medicine, integrated traditional Chinese and Western medicine, guide*, clinical practice guideline, recommend*.Data extraction and screening: Two researchers independently screened the literature, extracted the data, and cross-checked it. Disagreements were resolved through discussion or consultation with a third party. Data extraction included: first author, year of publication, journal of publication, type of disease, whether a population, interventions, comparisons, outcomes (PICO) problem was raised, whether evidence was researched, whether evidence quality evaluation was carried out, whether a suitable model of integrated TCM and WM was proposed, whether recommendation suggestions included TCM and WM diagnosis and treatment methods, whether suggestions of integrated TCM and WM were provided, whether the application of TCM and WM recommended suggestions were clarified, the specific content of the integrated TCM and WM and the application of the relationship.A qualitative design with purposive sampling was used to select experts in evidence-based medical methodology and researchers or clinicians who develop clinical practice guidelines for TCM/integrated TCM and WM. The included experts are required to have a doctoral degree, a senior professional title, and at least 10 years of work experience. A semi-structured interview format was used for data collection in an online meeting via Tencent Meeting (a Chinese-based online conference facility). The interviews lasted 40–60 min, and were all completed over a 2 months’ period. Data information saturation was used as a condition for the termination of sample collection.An outline for the interview was developed from the literature review, including the following questions: What do you think are the methodological or reporting flaws of the existing guidelines for integrated TCM and WM? Do you think that the existing clinical questions in clinical practice guidelines for the integration of TCM and WM are in line with the principles and methods of clinical questioning in guidelines? Do you think that the existing grading system of evidence for TCM is fully applicable to the development of guidelines for integrated TCM and WM? How do you think the quality of evidence of TCM expert opinion/ancient literature can be objectively evaluated? How do you think the guidelines for integrated TCM and WM can fully integrate the recommendations of TCM and WM in order to closely fit the clinical practice of integrated TCM and WM?NVivo software  was used for data coding and content analysis was used to analyze the data. The themes were refined after joint analysis and discussion by the research team to avoid bias.In the standard research group, the professional field covered clinical integrated TCM and WM, methodology of evidence-based medicine (including development and writing of guidelines). The systematic review of previous literature and results of the qualitative research were used as the content of a preliminary discussion for brainstorming. All participants spoke freely and fully on the topic of how to promote the development of guidelines for the organic integration of TCM and WM through methodological guidelines. The research group recorded the opinions of all participants verbatim, and the core research members integrated the information after the meeting by using the recording facility in Tencent Meeting. Based on the information integrated by the above-mentioned methods and steps of guidelines development, "issues to be considered in the guidelines for organic integration of TCM and WM" and "precautions for the guidelines for organic integration of TCM and WM" were developed, so as to improve methodology such as the formulation of clinical problems of the organic integration of TCM and WM, the grading of evidence and the formulation of recommendations, the mode of integrated TCM and WM, the organic integration methods of recommendations, and finally developed the draft of the methodologies and procedures for developing clinical practice guidelines of organic integration of TCM and WM.All members of the standard-setting group were invited to discuss, revise, and improve the basic content of the draft through consensus meetings to form the final draft of the methodologies and procedures for developing clinical practice guidelines of organic integration of TCM and WM.A total of 23 multidisciplinary experts from across the country covering the fields of WM, TCM and integrated TCM and WM, evidence-based medicine, pharmacy and nursing were invited to conduct the Delphi consultation. The consultation questionnaires were sent by e-mail and included the terminology, method protocols, diagrams and annexes in the draft. Experts were free to question any content of the exposure draft. Where experts disagreed with the content, they needed to give reasons and specific suggestions for changes. Moreover, the Chinese Society for Standardization conducted a web-based consultation to solicit input from a wide range of developers and users of guidelines for integrated TCM and WM.The research team collated the returned consultation drafts and a further consensus meeting was held by the standard development team to discuss each standard individually and decide whether to make changes to their content.Guidelines developers and methodologists in the field of integrated TCM and WM who had not participated in the development and consultation of this standard were then invited to participate in an external review. The review comments were summarized and discussed to make the necessary changes and improvements, and the standard was then published. The methodological flow chart is shown in Fig. A search obtained 54 guidelines for integrated TCM and WM. Only 3 (5.6%) of the included studies clearly presented clinical problems based on the idea of integrated TCM and WM. Only 26 (48.1%) papers described how to use the TCM and WM treatments to provide clinical ideas related to integrated TCM and WM, i.e., analyzing why TCM and WM can be integrated, and the advantages and disadvantages of each side. Only 22 (40.7%) papers gave the relationship between integrated TCM and WM, i.e., they specified how to use the treatment methods of TCM and WM in different situations  and how to make trade-offs, and indicated the order of priority and preference between TCM and WM treatments, including the combined use, recommendation of TCM, recommendation of WM, no recommendation of TCM, and no recommendation of WM. In the remaining guidelines TCM and WM recommendations are given separately, and the recommended intervention protocols are not sufficiently qualified to determine the relationship between the TCM and WM recommended intervention protocols, and give limited guidelines.A total of 20 experts in evidence-based medical methodology, researchers or clinicians with experience in developing guidelines in TCM/integration of TCM and WM were interviewed. The analysis yielded six themes and twelve sub-themes, as shown in Table In the internal discussion of the project team, researcher Hong-Cai Shang pointed out that TCM and WM should not only be integrated, but also go further to achieve organic integration of TCM and WM. "Integration" refers to the superficial connection. Which often involves "incompatibility", "organic integration" means that materials enter each other and become one. Organic integration is really the combination and integration of TCM and WM to form a new system. The guidelines for organic integration of TCM and WM should not be a simple superposition or collection of evidence and recommendations of both WM and TCM. It is the deep organic integration of the advantages of TCM and WM based on the clinical diagnosis and treatment pathways. Therefore, after repeated discussion, this standard has been named "methodologies and procedures for developing clinical practice guidelines for organic integration of traditional Chinese and Western medicine".Starting from each step of guideline development, "Specific conceptual problems of clinical practice guidelines for organic integration of TCM and WM" were introduced by asking questions, to guide and standardize the process of guideline development for organic integration of TCM and WM, and concurrently develop the "notes" in the guideline steps see Additional file (1) Whether there is a relationship between WM diseases and TCM diseases or a relationship between WM diseases and TCM syndrome differentiation and whether the scope of integrated TCM and WM involved in the theme of the guidelines for organic integration TCM and WM has been determined through extensive professional consensus when selecting the guideline theme?(2) According to the guidelines theme determined in question 1, it should be further determined whether the guideline is named using WM disease names, TCM disease names or TCM syndrome differentiation.(3) Does the practice of diagnosis and treatment of the disease fully reflect the characteristics of organic integration of TCM and WM? Instead of simply focusing on TCM or WM, or simply adding the two. For example, when treating a viral cold or a certain stage of the cold’s development, a certain sub-type, a certain syndrome or a certain symptom of a viral cold, some Chinese medicine or traditional Chinese herbs may have the effect of replacing and/or supplementing WM, while some may exert attenuating and/or enhancing effects on WM or other WM therapies; and vice versa.(1) Does the proposal of the problem come from the practice of integrated TCM and WM, and include the idea of integrated TCM and WM treatment?(2) Are TCM clinicians and WM clinicians, as well as clinicians combining Chinese and Western medicine all selected for the questionnaire?(3) Guideline developers should give full consideration to the characteristics of combined treatment of diseases with TCM and WM treatment and include specific outcomes for TCM before ranking the outcomes’ importance.Do the guidelines explicitly use integrated TCM and WM models of diagnosis and treatment? .(1) Has there been a complete retrieval of evidence sources for TCM, WM and integrated TCM and WM ?(2) Has there been a collection of characteristic evidence of TCM, such as TCM ancient books and expert experience?(3) Has there been a collection of evidence of patients' preferences, willingness, and values for accepting TCM or WM treatments?(4) Besides the above considerations, has there been a complete retrieval of information on the "body of evidence" for a specific clinical question? First, the "body of evidence" can be derived from a synthesis of pieces of evidence from the best research design, or it can be composed of evidence from multiple research methods and multiple sources. For example, when real-world evidence exists, ancient documentary evidence and expert opinion evidence can be corroborated with real-world evidence to form a "body of evidence". Again, for example, the "body of evidence" from randomized controlled trials provides evidence of intervention effectiveness, while studies such as observational studies or the registration and recording platforms of adverse reactions can provide evidence of intervention implementation and safety, and the two types of evidence together support the designation of scientific and rational decision-making.(1) Is it necessary to classify the quality of evidence with TCM characteristics such as TCM ancient books and expert experience? Or is there a custom classification standard of the quality of evidence and recommendations for the guidelines for organic integration of TCM and WM?(2) How should an appropriate grading system for the quality of evidence and recommendations be chosen in order to achieve a reasonable classification of included TCM and WM literature? In this process, is it necessary to make appropriate adjustments to the existing grading system of the quality of evidence and recommendations ? Has the adjustment method passed the consensus of the guidelines’ development group?(1) When forming the recommendations for the organic integration of TCM and WM, are clinical experts from TCM, WM and integrated TCM and WM all invited to agree on the recommendations?(2) Does the generated recommendation explain the relationship between TCM treatment, WM treatment and integrated TCM and WM treatment?(1) Does the title of the guidelines clearly state "clinical practice guidelines for organic integration of TCM and WM?"(2) Has a recommendation implementation pathway map or other visual form been developed to present the recommendation, which reflects the full organic integration of TCM and WM recommendations?(3) Is the report of clinical practice guidelines for organic integration of TCM and WM based on the pre-set integrated TCM and WM model?The existing guidelines for integrated TCM and WM often use the combined diagnosis of dual disease names of TCM and WM. In TCM and WM combined with double disease diagnosis, WM diagnosis of disease based on WM diagnostic criteria, and then based on the primary symptom of disease using TCM diagnosis, to determine the guidelines contain diseases and disease categories of TCM and WM common to the three kinds of situations. In this standard, the correlation diagram of disease names of organic integration of TCM and WM is designed to show the three common situations: the scope of WM diseases includes the category of TCM disease names, the category of TCM disease names includes a variety of Western diseases and the scope of Western diseases includes a variety of TCM disease names. It is used to guide the guidelines developer to consider the relationship between WM diseases and TCM diseases, and the relationship between WM diseases and TCM syndrome differentiation, so as to rationally determine the theme of the guidelines.The clinical problems in the guidelines of organic integration of TCM and WM should be generated from the diagnosis and treatment practice of integrated TCM and WM, especially for the most confusing diagnoses and treatment problems for clinicians, and should reflect the characteristics and ideas of integrated TCM and WM diagnosis and treatment.After the theme of the guideline is determined, the guidelines developer determines the clinical problem, and it is necessary to consider whether the question raised is specific and clear. The relationship between WM diseases, TCM diseases and the corresponding syndrome differentiation or staging types must have been analyzed when determining the theme. Clinical problems need to be raised and reflect the needs of clinical diagnosis and treatment practice after the above relationship is clarified. This standard uses graphics to show PICO problems and the relationship between disease name and syndrome differentiation, which can not only guide the guidelines developer to refine PICO clinical problems, but also help make the clinical problems more specific and clearer.Evidence retrieval for guidelines for organic integration of TCM and WM should reflect a full search of the evidence sources of TCM, WM and integrated TCM and WM, and have considered the retrieval and inclusion of TCM characteristic evidence. In addition, the relationship between multi-source evidence and the development of guidelines should be clarified. This standard, emphasizes the importance of ensuring that ancient TCM books and modern TCM expert experience have also undergone validation and evaluation by clinical research , so that it can enter the summary and evaluation of secondary studies and become the evidence source for guidelines. That is, the TCM classical evidence can be inherited through the verification of ancient and modern evidence. We have integrated the above content into a schematic diagram of the evidence source level and structure of the guidelines of organic integration of TCM and WM.In order to make it easier to refer to this standard, we have designed seven appendices  to understand many details of the guidelines’ development process, such as the contributors and their role in guidelines development, literature retrieval and systematic review of the guidelines development team. Moreover, the inclusion of this standard before each step can help guidelines developers to formulate guidelines with full organic integration of TCM and WM recommendations in the practice of diagnosis and treatment of TCM and WM.Dual diagnosis is often used in the existing guidelines for integrated TCM and WM. In dual diagnosis, the WM disease name is based on Western medical diagnostic criteria, and then the TCM disease name is based on the main symptoms of the disease. Therefore, it can determine the scope of both the Western medical disease and the Chinese medical disease included in the guidelines. In this standard, we have listed three patterns in general and the diagram with examples to help the readers to confirm the appropriate guidelines’ theme and PICO questions. The development of guidelines for organic integration of TCM and WM is often faced with difficulties because of insufficient evidence, especially the lack of high-quality studies on the comparison of integrated TCM and WM and non-integrated TCM and WM. At this point, the guidelines development group needs to gather expert evidence that includes empirical feedback from long-term practice provided by people who are knowledgeable or technically skilled in a particular field on the clinical issues that urgently need to be addressed, and how to use and evaluate evidence from ancient TCM literature and the experience of modern TCM experts. This standard has given detailed suggestions. The guidelines panel needs to read this standard before writing the guideline protocols to make sure the specific issues about guideline methodologies and practice about organic integration of TCM and WM are fully considered. After the guideline is completed, the guidelines panel should also review the standard again to determine the completion of each step, especially regarding clear and operable presentation of organic integration of TCM and WM recommendations.With the development of evidence-based medicine (EBM), guidelines play an increasingly important role in healthcare decision-making both in China and worldwide. High-quality guidelines development depends on the complete and rigorous guidelines development methodology ."Integration" refers to a superficial connection, which often still has areas of incompatibility. Organic integration means the cooking gas comes out", "Organic Integration" means that materials enter into each other and become one [It is not difficult to acknowledge the achievements and develop the guidelines for integrated TCM and WM. At the present stage, integrated TCM and WM typically remains at the level of the combination of TCM and WM diagnosis and treatment methods, and the diagnosis and treatment methods of the two modes are relatively separate, and this can't actually guide the clinical diagnosis and treatment practice of TCM and WM. The development of integrated TCM and WM should focus on clinical practice, analyze the relationship between TCM and WM, find the confluence, and seek new concepts and methods of integrating TCM and WM . Origin come one . Organiccome one .The establishment of this standard is strictly in accordance with the construction rules of the group standard of China Association for Standardization to conduct project approval and development and solicitation of opinions on standards. The development process includes the literature review and expert interview, information extraction, discussion and analysis, collection of opinions from consensus meetings and subsequent revision. Participants in the standard development process include methodologists, clinical experts, and targeted users of the guidelines developed for TCM, WM and integrated TCM and WM. The purpose of this standard is to establish a method for developing guidelines in the field of integrated TCM and WM. The content provides: 1) Basic steps and procedures of guidelines development; 2) The important issues in developing guidelines for organic integration of TCM and WM. At each step, these are presented as "problems that need to be considered" with eye-catching titles, which very easily attract the attention of the guidelines’ developers. This allows them to consider the unique methodological problems of TCM and WM organic integration and helps them to raise clinical problems from the practice of TCM and WM diagnosis and treatment. They can then define the integrated TCM and WM model, so achieving complete evidence grading and rational evaluation of evidence quality. This can, in turn, guide the writing of recommendations and opinions and the construction of diagnosis and treatment procedures based on the integration of TCM and WM in the hope of achieving implementation of the guidelines from their development; 3) It provides the points for attention in the development of various guidelines steps from the perspective of organic integration of TCM and WM, which can enhance their prompting and guiding functions; 4) Seven appendixes are provided for reference and to aid operability. However, this standard still has obvious limitations, one of the main reasons for the low quality of integrated TCM and WM guidelines is the lack of high-quality and convincing evidence that integrated TCM and WM is superior to WM alone. There is less evidence available to carry out controlled studies given the realistic timing of TCM and WM treatment and their respective characteristics and advantages. When the evidence is seriously insufficient it is difficult to form clinical recommendations even with highly operational methodological guidelines.Establishing a standardized development procedure for guidelines for the organic integration of TCM and WM can provide a feasible method for systematically integrating TCM and WM, promote the communication and cooperation between TCM and WM, and form a comprehensive treatment model for disease diagnosis and treatment.This standard has specified the method for developing guidelines in the field of integrated TCM and WM, so it is mainly applicable to those who develop these guidelines. We believe that they are not confined to China and not confined to the integration of TCM and WM.As a genre of traditional medicine, TCM has gained popularity worldwide. At present, more than 180 countries and regions in the world recognize TCM and use TCM to treat diseases to a greater or lesser extent. In the Asian region, TCM is widely used in China, Japan, Korea, Vietnam, Malaysia, and other countries. Moreover, TCM has been gradually recognized and accepted in Europe, America, and other regions. Many countries have begun to introduce TCM into their medical systems, combining it with modern medicine to provide a more comprehensive treatment plan . In 2014Several surveys in the USA, Denmark, Sweden and Norway indicate a high consumption of complementary and alternative medicine (CAM) . An inteFor guidelines developers, by including international guidelines developers and guidelines developers in the WM professional field, when collecting and formulating clinical problems, it is possible to encounter clinical questions about the use of integrated TCM and WM or other types of CAM combined with WM in clinical practice. Both guidelines’ developers and health professionals should consider the current stage of the disease and how these modalities should be integrated. What are the potential interactions and so on? This standard can be applied in many ways.When formulating recommendations, considering the generalization, suitability, and potential resource utilization, while also balancing clinical advantages and disadvantages is quite necessary and helps to form rigorous and reasonable recommendations. This view and method have been considered and practiced in many published guidelines , 20. ConThe organic integration of TCM and WM is better at solving clinical problems, such as improving efficacy and detoxification, reducing the drug dosage, early intervention, avoiding surgery, and reducing complications. It is an international trend of future medical development. This standard aims to further promote the organic integration of TCM and WM by developing high-quality guidelines for organic integration of TCM and WM and understanding diseases and life from multiple perspectives and world views. Finally, it promotes integrating and developing Chinese and Western cultural concepts and clinical practice internationally.Additional file 1: The formulation methods and procedures for clinical practice guidelines for organic integration of traditional Chinese and Western medicine.


EdTech is an umbrella term to encompass apps, learning platforms and online courses designed with the explicit purpose to educate and advance learning. The availability and variety of these tools expanded significantly after the COVID19 school closures but only 16% of 1058 educators surveyed by EdWeek (2023) described EdTech as very effective in accelerating learning. Indeed, converging evidence shows that although EdTech has the potential to provide highly individualized and advanced learning options, it is not meeting its potential (yet) to positively impact children’s learning4. The U.S. Food and Drug Administration and similar agencies in other countries review and approve therapies offered on the market, including game-based digital therapeutic devices. However, despite repeated calls, there is no equivalent certification and approval agency for EdTech5. There are various and complex reasons for this, including the rapid development and often uncritical adoption of technologies that outpace the global research capacity for rigorous testing of the effects of these technologies; the misalignment of incentive mechanisms for EdTech developers and researchers to collaborate on product development research; the lack of scientifically trained EdTech entrepreneurs and dedicated EdTech training for scientists and the lack of international, EdTech-specific evaluation standards.Mental health and learning outcomes are closely related and both are affected by students’ use of EdTech6. We are an interdisciplinary research team and aim to advance the field with an initial, easy-to-apply guidance for evaluating EdTech’s evidence claims based on scientific standards. Based on general principles of the science of learning in terms of methodological plurality and quality assurance criteria, we outline a simple evaluation routine to facilitate discussions of EdTech evidence among diverse stakeholders.Disciplinary differences in how the quality of an EdTech product is evaluated, further complicate the assessment efforts. For example, in psychology, the focus on measuring learning outcomes and assessing instructional features through media comparison research studies is pertinent for gauging EdTech’s impact on academic performance7. Despite being very popular and widely used by children, EdTech products often lack research-based insights on how we learn, which has negative consequences for early education8. For example, Meyer et al. (2021) analyzed the 124 most-downloaded EdTech mobile apps and reported that most of them were judged to stimulate repetitive, distracting, and meaningless experiences with minimal learning value9.Evidence-based EdTech has been called for but is in short supply, as shown in recent government and industry reports. Out of a hundred most popular EdTech in US schools, only a quarter had evidence of research and positive impact11There are several reasons why a majority of EdTech ventures do not rely on evidence-based, scientifically rigorous research to evaluate and drive their impact. One is that EdTech ventures, by virtue of being part of a competitive marketplace, are driven by Key Performance Indicators, level of funds raised, retention, profit margins, or product scalability. When sales take over evidence, learning outcomes are not reached. This problem leads to products being deployed in learning environments that may or may not be effective and may even have negative effects. Indeed, the negative effects, such as lower or no learning after the introduction of EdTech into public classrooms, were noted by recent governmental reports assessing the state-of-art in EdTech after the pandemic 12. Furthermore, popular EdTech advertised to young children contain manipulative design features such as pressures for children to complete a game within a short time, difficulty to navigate the screen or artificially prolong children’s app use13.Furthermore, there is the issue of EdTech companies using data for monetization and commercialization purposes. Many EdTech advertised to children use data with persuasive design intended to motivate children to use the app for as long as possible and engage them in repetitive use without advancing their learningA related issue impeding a system-wide orientation towards evidence is a disconnect in the EdTech funding and development. While the investor and funding community typically value impact metrics that are guided by scientific research principles, they do not have a unified approach to guide these efforts. Some use national standards of evidence available in individual countries , while others have their own internal assessment criteria that they apply as part of due diligence process. Others employ commercial consultants to gauge the scientific basis of companies seeking investment with their own, often non-transparent, assessments.can have a strong positive impact on educational outcomes if there are certain conditions in place, including that the technologies are designed with learning principles in mind. Evidence for this proposition has been provided in meta-analyses of apps for early learning or digital reading apps15. One of the key reasons that commercial EdTech have a low evidence base is that they are often not developed by, or with, researchers. The misalignment between latest scientific evidence and EdTech design is a methodological one and a practical one16.The scientific consensus is that EdTech Practically, the advancement of ethical, evidence-based EdTech is a complex task that requires collaboration between EdTech funders, producers, scientists as well as users (teachers and children/adolescents in classrooms). EdTech products should provide a full disclosure on the stage of development/level of maturity in their design, development, implementation, and evaluation process for the respective product. In the evaluation process, schools, procurement teams and funders need to know how to assess EdTech’s evidence base. What criteria for the quality of provided evidence should be used in the assessment ? What questions should be asked in determining how EdTech developers view and apply evidence in their work ?https://ies.ed.gov/ncee/wwc)) apply different evaluative standards and draw divergent recommendations about which educational programme is evidence-based17. This can be confusing for EdTech and should be routinely addressed with an evaluation approach, spanning foundational research, practice-informed basic research, and user-oriented research with direct applicability to policy and practice.These questions do not have straightforward answers, but they can be systematically reflected upon with some guiding frameworks. There are many analysis questions to consider when making a conclusion about “what works” in education - even the largest educational clearinghouses  as a simple guide to be applied in the evaluation of the evidence base of existing EdTech solutions and to guide the EdTech companies in growing their products’ evidence base. Table EVER can be applied to the development of EdTech solutions, the evaluation of existing or planned products, and the investment in products. Thereby, products with poor or no evidence can be filtered out and conversely, more quality products will enter and/or remain in the EdTech market. Our intention is to encourage this cycle with EdTech created for assessment, intervention or edutainment (i.e. education coupled with entertainment) in K-12 education.Indeed, EVER can be used for EdTech of any type, including those that are designed to promote foundational skills in literacy and math, those that aim to change learners’ behaviour, as well as those that combine assessment and intervention. EVER can be used at various stages of an EdTech’s lifecycle, including the pre-company stage as part of an accelerator or when mature companies look for additional funding. The strength of each of the criteria should be rated on a 0–5 point-scale for each of the cells, including the cells where the company has no activity.Methodological quality denotes whether the evaluation methods used are appropriately executed, described and justified, and what the results show. It helps to answer questions such as “Is the rationale sound or logically flawed?”, “Can the chosen methodology speak to whether the EdTech works as intended?” and “Has the EdTech been tested in a sufficiently large target population?” Outcome strength denotes whether the EdTech has a sizable impact or predictive value. Impact is usually quantified as a significance measure or an effect size, which is a quantitative measure of the magnitude of the effect on a particular external measure. It helps to answer questions such as “How much of an effect does the EdTech have?” and “How accurate is the tool?” Predictive value can be quantified by sensitivity/specificity predictive validity and classification accuracy, which are quantitative measures of how good a tool is at correctly distinguishing groups/categories .Generalizability can be defined as the extension of research findings and conclusions from a research study conducted on one selected sample population to the population (or a target population) at large. While a larger sample typically comes with a higher generalizability, it still needs to match the target population in terms of demographic characteristics, socio-cultural values, skills and abilities . It helps to answer questions such as “Can I be sure that the tool works for my students?” and “Will the tool be well-received in my market?” or “Who will the product be helpful for?”.Ethics and Transparency ensure that the questions asked or the design of the EdTech and its purpose are ethical, as well as ensuring users’ well-being as well as broader contributions to social justice. It includes culturally-responsive approaches and a transparent use of participants’ data. It helps to answer questions such as: “Do users know which personal data are collected, used, or otherwise processed?”, “What are the data protection standards?”, and “Are users treated respectfully and is their dignity preserved?” There are different criteria for assigning scores in each of the quality assurance aspects with different types of evaluation methods. For example, the criteria to assess methodological quality of conceptual studies can be different from generalisability criteria in quantitative or qualitative studies.Finally, The proposed EdTech Evaluation Routine can be used as a prompt for reflection when evaluating the evidence portfolio of diverse EdTech products, processes and initiatives. The synergistic model proposed through the evaluation process takes into account the benefits and limitations of different methodological approaches and can be applied in conjunction with local quality assurance assessments of EdTech (for example those applied at district or school level) as well as by EdTech developers in iterative product development. EVER is best used as part of formative evaluations; it is not intended to determine “good” or “bad” solutions but rather to offer a constructive template for addressing the current lack of EdTech evidence in the ecosystem.The advent of generative AI, and the current lack of accountability measures that ensure the implementation of evidence-based criteria in children’s EdTech, mobilised international governments into action. Organisations offering rapid evaluations and research consultancy services for EdTech have emerged alongside increased academia-industry partnerships. The evaluation routine can be seen as a first step toward an international, open-access benchmark of EdTech evidence in various partnership models between researchers and the EdTech community. EVER can be used alongside internal company or non-profit research and national evaluation standards and should be supplemented with other frameworks that target cost-effectiveness, data privacy and teachers’ usability evaluations.In conclusion, the Science of Learning is an interdisciplinary field of study with many diverse methodologies. The open-ended nature of EVER is intentional in that we wish to promote an equitable approach to EdTech evidence that acknowledges the limited access some, notably smaller start-ups from low and middle-income countries, have to research teams and testing possibilities in schools. We hope that the guidance within our preliminary EdTech Evaluation Routine can be used as a prompt for discussions about EdTech evidence across various stakeholder groups and be part of the mind shift necessary for promoting greater integration of science into EdTech design and thereby, better learning outcomes for our students.


Paediatric pulmonary diseases are the leading causes of mortality amongst children under five globally. Economic evaluations (EEs) seek to guide decision-makers on which health care interventions to adopt to reduce the paediatric pulmonary disease burden. This study aims to systematically review economic evaluations on different aspects of the inpatient management of paediatric pulmonary diseases globally.We systematically reviewed EEs published between 2010 and 2020, with a subsequent search conducted for 2020–2022. We searched PubMed, Web of Science, MEDLINE, Paediatric Economic Database Evaluation (PEDE) and the Cochrane library. We extracted data items guided by the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. We collected qualitative and quantitative data which we analysed in Microsoft Excel and R software.Twenty-two articles met the inclusion criteria. Six of the articles were cost-effectiveness analyses, six cost-utility analyses, two cost-minimisation analyses and eight cost analyses. Twelve articles were from high-income countries (HICs) and ten were from low- and middle-income countries (LMICs). Eight articles focused on asthma, eleven on pneumonia, two on asthma and pneumonia, and one on tuberculosis.Conducting more EEs for paediatric pulmonary diseases in LMICs could allow for more evidence-based decision-making to improve paediatric health outcomes. Paediatric pulmonary diseases are the leading causes of morbidity and mortality amongst children under the age of five, especially in low- and middle-income countries (LMICs) . AccordiOver the years, economic evaluations (EEs) have increased in availability and have gained more acceptance in priority setting . EEs areWe conducted a systematic review of EEs focusing on alternative approaches for inpatient management of paediatric pulmonary diseases within five electronic databases: PubMed, Web of Science, MEDLINE, Paediatric Economic Database Evaluation (PEDE), and the Cochrane library. We made use of keyword searches, MeSH terms, truncation, and Boolean operators. We had three search categories, namely: paediatrics, pulmonary disease, and EEs. We also set parameters for the year of publication to include 2010–2020. An updated search was also conducted for 2020–2022 to identify any additional articles.We included full EEs cost-effectiveness analysis (CEA), cost-utility analysis (CUA), cost–benefit analysis (CBA), cost-minimisation analysis (CMA) and partial EEs . We included articles reported in the English language, which were published between the years 2010 and 2020. The EEs included in the review were specific to paediatric pulmonary diseases, comprising inpatients aged from zero to six years old. Our focus was on interventions delivered within the inpatient setting, including alternative medications, diagnostics and screening, medical devices, and additional support such as supplemental oxygen. Countries were included irrespective of income level.In the first stage of the selection process, we removed duplicates in EndNote X9 Software . We did this both electronically and manually. The screening of the papers was done in three stages: title screening, abstract screening and full-text screening. These stages are represented diagrammatically in the PRISMA diagram, Fig. After the selection process, we stored the articles which met the eligibility criteria in a shareable folder in EndNote X9 Software, Clarivate Analytics.health economic evaluations are identifiable, interpretable, and useful for decision making” [We developed an extraction tool in Microsoft Excel using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist as a guide to identify the data items . The CHE making” . The extWe adopted a convergent mixed-methods approach , combiniWe conducted our quantitative data analysis in Microsoft Excel and R software  using the RStudio interface. We analysed the volume of publications, the hospitalisation costs, and ICERS. All costs which were not reported in United States Dollars (USD) were converted to USD using the reported exchange rate for the study year. We inflated the costs to 2019 USD using World Bank Consumer Price Indices [The study is a secondary analysis which did not involve human subjects; however, we obtained ethical approval from the Human Research Ethics Committee (HREC) at the University of Cape Town (UCT), reference number HREC 587/2020.Our search retrieved 1470 articles. After duplicates were removed, both manually and electronically, 1159 articles remained. Following the screening by title and by abstract, 945 of the 1159 articles were excluded. We then screened the full text of the remaining 214 articles and 20 articles met the full inclusion criteria  [ 22, 24,, 37–40 oWe also used our data analysis framework to identify the outcome measures, which were reported in the articles as natural units, quality adjusted life years (QALYs) and disability adjusted life years (DALYs). The natural units were emergency department (ED) visits averted (5) , 37–40, The importance of EEs being readily available to inform health care priority setting must be underscored. In this regard, systematic reviews such as this can synthesise large amounts of economic evaluation data and make these more accessible . The finOur systematic review identified both partial EEs  and full EEs . Cost analysis is the most basic form of  EE as it assesses only the costs of the intervention and provides no information on the outcomes . The resWe also highlight some notable differences in the geographical distribution of diseases for which the EEs were conducted. This could be attributed to asthma being a disease of affluence . In the Interestingly, we only found one TB study for the zero to six age group, yet TB incidence is high in LMICs . A possiWe were also interested in understanding the different perspectives adopted in the articles. There were more articles which adopted the provider perspective than the patient perspective. A possible explanation is one similar to the adoption of full vs partial EEs where the resources for societal costing might not be available.Our review also summarised the methodological approaches that were employed in the different articles included in the review. From our findings, not all eligible studies reported discounting their costs and outcomes. The studies which discounted their costs and outcomes discounted at a 3% discount rate , 18 whicIt is worth highlighting the study strengths and limitations. This systematic review is unique in that it focused on the different types of economic evaluations conducted for paediatric pulmonary diseases in a global context. Unfortunately it only considered EEs conducted in a hospital setting or that assessed inpatients. Therefore, our findings would not be generalizable to other service delivery platforms. The inclusion of only published literature and not grey literature is another limitation. Additionally, screening for eligibility and data extraction was not done by two independent reviewers.There was also a missed opportunity to analyse the trends in methodological approaches over a longer duration as the review only included published literature between 2010 and 2022 due to practicality.The study set out to summarise EEs that have been conducted for paediatric pulmonary diseases globally. There were fewer EEs conducted in LMICs than in HICs, yet children from LMICs are disproportionately affected by pulmonary diseases. Conducting more EEs, of good quality for paediatric pulmonary diseases in LMICs could allow for more evidence-based decision-making to improve paediatric health outcomes.


The enlargement of the liver and spleen  is a common manifestation of Gaucher disease (GD). An accurate estimation of the liver and spleen volumes in patients with GD, using imaging tools such as magnetic resonance imaging (MRI), is crucial for the baseline assessment and monitoring of the response to treatment. A commonly used method in clinical practice to estimate the spleen volume is the employment of a formula that uses the measurements of the craniocaudal length, diameter, and thickness of the spleen in MRI. However, the inaccuracy of this formula is significant, which, in turn, emphasizes the need for a more precise and reliable alternative. To this end, we employed deep-learning techniques, to achieve a more accurate spleen segmentation and, subsequently, calculate the resulting spleen volume with higher accuracy on a testing set cohort of 20 patients with GD. Our results indicate that the mean error obtained using the deep-learning approach to spleen volume estimation is 3.6 ± 2.7%, which is significantly lower than the common formula approach, which resulted in a mean error of 13.9 ± 9.6%. These findings suggest that the integration of deep-learning methods into the clinical routine practice for spleen volume calculation could lead to improved diagnostic and monitoring outcomes. Gaucher disease (GD) is a hereditary lysosomal storage disorder that is characterized by the accumulation of glucocerebroside in various organs, such as the spleen, liver, kidneys, lungs, brain, and bone marrow ,2,3,4. C3) = 30 + 0.58 × L × D × T, or equivalent formulae [A precise estimation of the liver and spleen volumes is a crucial aspect of the baseline assessment and monitoring of the treatment response in patients with GD. In this regard, magnetic resonance imaging (MRI) is a widely utilized imaging modality ,11. Radiformulae ,10. In sformulae ,13. Howeformulae ,15,16, aformulae ,18. ConsIn recent years, the use of deep-learning methods in numerous and various fields has expanded, due to the increase in data availability, computing capability, and algorithmic improvements . “Deep lDeep learning in biomedical applications has been utilized successfully, in various studies, for abdominal organ segmentation ,37,38,39We estimated the accuracy of the proposed method in comparison with the commonly used spleen volume formula method. To the best of our knowledge, this is the first time such a comparison has been carried out for MRI scans. Additionally, while there are only a few studies on spleen segmentation in splenomegaly cases, we further established the necessity of applying deep-learning methods for spleen volume calculation in a patients’ group with spleen disorders. To the best of our knowledge, this is the first work applying deep-learning methods for patients with GD. We show that the deep-learning method outperforms the commonly used spleen volume formula, and leads to a much higher accuracy.Abdominal MRI scans of 1.5 Tesla  of 30 patients with GD were included in the study. All patients were followed in the Gaucher Unit, Shaare Zedek Medical Center (SZMC), and had an abdominal MRI assessment of spleen volume performed at the Department of Radiology, SZMC, as part of clinical routine practice. The cases included in the study were randomly selected from a cohort of 100 cases. Abdominal MRI scans that had low-quality images due to technical issues, or that included additional abdominal pathologies not related to GD, were excluded from the study.Picture Archiving and Communication System (PACS) was utilized to extract the axial out-of-phase T1 sequence with DICOM and TIFF image files. The MRI slice thickness was 3 mm. The boundaries of the spleen were marked on the TIFF file images by a radiology resident (A.W.), with the help of a computer science student (M.W.). All MRI scans were anonymized and coded prior to transferring them to the SAMPL Lab. The study was approved by the SZMC IRB; protocol number: 0520-20-SZMC. A waiver was received for the signing of informed consent.Among the 30 patients who underwent MRI, it was determined that a total of 20 patients’ MRI scans contained all the required slices, rendering them suitable for use as the test set. Two MRI scans had missing slices from the beginning or end of the spleen. Hence, volume evaluation could not be performed, and these MRIs were disqualified from inclusion in the testing set. In three MRIs, some of the outlined spleen slices were not accurate enough, or there were technical coding difficulties in extracting the masks from them. This created “sequence holes” in the scan, which could lead to an inaccurate estimation of the spleen volume. As a result, these MRIs were also disqualified from the testing set. Five additional MRI scans were provided at different resolutions and in different file settings to the other MRIs. As different resolutions and file settings may affect differently the accuracy metrics, it was decided to exclude these scans, so that the reported accuracy metric would not be affected positively by this bias.A manual segmentation process was employed to outline the boundaries of the spleen in the MRI slices  and normalized , to ensure that the image tissue features were comparable across the different MRI patient scans, and scaled consistently. Importantly, the patient scan mean and standard deviation were calculated for the body scan regions with the background excluded, in order to reduce the body size effect, and eliminate the background effect. Secondly, the slices were cropped  to have a size of 256 × 256 pixels. This helped to reduce redundant data, and enhance the modeling efficiency. Finally, the slices were augmented  to generate more synthetic data, which has been demonstrated to improve accuracy, and mitigate overfitting .The pre-processed dataset was utilized to train a deep-learning model, with the spleen masks as the target. A total of 30 patient scans were used, with one patient scan excluded at a time for the testing set, while the remaining patient scans were used for training, utilizing a leave-one-out cross-validation scheme  pipelines are an essential part of the deep-learning pipeline, to correct inaccurate model predictions (MP). In this study, we integrated several PP algorithmic steps into our pipeline. The first step involved filling holes in the MP segments that contained holes. The second step was connected component analysis. This operation enables the detection and labeling of all segments predicted by the model for a given slice as different components. The third step was the component size evaluation from the previous step. This operation enabled us to keep the most probable segment, which was the largest component (in pixel area) in the given slice (which should be associated with the spleen), and to remove noisy and inaccurate predictions . The PP pipeline ensured the correct segmentation of the spleen, leading to a more precise volume calculation. Importantly, in this study, most of the MP slices were valid, and the PP did not have any effect on it, as in The spleen volume calculation is performed as follows. For each slice, the area of the spleen is calculated by summing the pixels associated with the spleen, and multiplying the result by the pixel size conversion factor. This gives the area of the spleen in the given slice. Then, the areas of all the slices are summed to yield the total volume of the spleen. For accurate summation, a trapezoid integration method is applied to integrate the areas with respect to the depth–size factor, which can be obtained from the DICOM file of each slice.At the beginning and end of an MRI scan, the spleen is not presented. When a deep-learning model is applied to a slice where the spleen is not presented, ideally, it returns zero values for all pixels, signifying the absence of spleen detection. However, it is noteworthy that the model can predict artifacts in empty slices. That may decrease the accuracy of the calculated spleen volume, especially when the number of slices in which spleen in not presented can be large (more than 50% of the MRI scan). In order to improve the model performance, the region to which the spleen is confined (the perpendicular axis to the plane of the slices in the scanning direction), should be detected. Subsequently, the slices in which the model predicts artifacts outside of the confined region of the spleen should be nullified. One approach to this is to calculate the area of the predicted segment in each slice (slice plane). This returns a profile of the slices area as a function of the scanning direction. The ideal shape of such a profile starts with the line at the zero value (indicating spleen absence), then, when the spleen is detected, the value increases, up to a maximum value (when the spleen within a given slice is largest), and then decreases again, until there is no spleen, and ends with a line of zero values (spleen absence). Practically, in the zero-value lines, outside of the confined region of the spleen, some noise appears, due to the model artifact predictions. Moving from the maximum value on the profile to the right/left of the scanning direction will result in the first zero prediction. Then, from that point to the right/left, all the slices for which the model predicts artifacts should be nullified. This approach may be problematic if the first zero occurs far from the true confined region of the spleen. Alternatively, the edges of the region where the spleen is confined  can be detected via low-value thresholding on the area profile. This yields a set of slice indices corresponding to the locations of the right and left boundaries of the spleen in the scanning direction. Clustering these indices into two clusters yields two indices, corresponding to the right and left boundaries of the spleen, and representing the confined region of the spleen. Then, all the slices outside of the confined region should be nullified. We applied this approach in this study.In addition to the spleen volume calculation, we also calculated an accuracy metric known as the DC. The DC provides an indication of the overlap between the GT spleen segments and the MP segments, and is given by:When the Dice coefficient equals 1, it represents the best matching between the GT and MP, which means that the model has succeeded in accurately segmenting the spleen, and when the Dice coefficient equals 0, it means there is no matching at all between the GT and MP, and indicates a low model performance.All the code used in this study was implemented using Python 3 . In addiOur modeling pipeline consists of pre-processing the MRI scans , inputtiIn order to quantify the accuracy of the model, ground truth (GT) segmentation of the spleen was performed by a radiologist, and was compared to that obtained via the deep-learning model. We used the Dice coefficient  to indicate the amount of overlap between the GT and model segmentation. For a perfect segmentation, when the GT exactly matches the model prediction, a value of 100% is obtained. In the opposite case, for a completely inaccurate segmentation, where there is no overlap at all, a value of 0% is obtained. Values that are closer to 100% indicate a high accuracy performance of the model. The DC for the 20-patient testing set cohort was almost perfect .3 , the error in the volume calculation using the formula method was relatively larger, more than 34%, while the error was much smaller—at most, 3.6%—using the MP methods. This suggests that the MP method is also valid for the smaller spleen volumes. Even after excluding these patients, the mean relative error was still large for the formula method, at 11.2%, compared to the MP method, demonstrating the advantages and consistency of the deep-learning-method predictions.The results show that the MP yielded a mean relative error that was much smaller than the mean relative error obtained via the formula method. The standard deviation error was also smaller using the MP method. For two patients , where the spleen is not presented at the beginning and end of the scan, may serve as a true indication of the MP’s performance. Only in one patient (number 13, The proposed method has the potential to be applied and utilized by physicians for the accurate diagnosis of spleen disorders, and to monitor the response to treatment. Furthermore, the results of this study demonstrate the potential of deep-learning methods in accurate and efficient spleen segmentation and volume calculation in Gaucher patients. While the development of the deep-learning model and algorithmic solution may require significant time and labor, the automatic and efficient nature of the method makes it a valuable tool for clinical routine and in-house applications. However, it is important to note that these methods need to be updated and re-trained, to account for changes in MRI data, and to work with different MRI machines.


However, most eHealth literacy definitions and frameworks are designed from the perceptive of the individual receiving eHealth care, which do not include health care providers’ eHealth literacy or acceptance of delivering eHealth resources.To identify existing research on eHealth literacy domains and measurements and identify eHealth literacy scores and associated factors among hospital health care providers.Individual eHealth literacy, (2) Interaction with the eHealth system, and (3) Access to the system. Pairs of authors independently assessed eligibility, appraised methodological quality and extracted data.This systematic review was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 checklist. A systematic literature search was conducted in MEDLINE, Cinahl, Embase, Scopus, PEDro, AMED and Web of Science. Quantitative studies assessing eHealth literacy with original research, targeting hospital health care providers were included. Three eHealth literacy domains based on the eHealth literacy framework were defined a priori; (1) individual eHealth literacy. A minority of the studies covered domains such as interaction with the eHealth system and access to the system. The mean eHEALS score in the studies ranged from 27.8 to 31.7 (8–40), indicating a higher eHealth literacy. One study reported desirable eHealth literacy based on the Digital Health Literacy Instrument. Another study reported a relatively high score on the Staff eHealth literacy questionnaire. eHealth literacy was associated with socio-demographic factors, experience of technology, health behaviour and work-related factors.Fourteen publications, of which twelve publications were conducted in non-Western countries, were included. In total, 3,666 health care providers within eleven different professions were included, with nurses being the largest group. Nine of the included studies used the eHealth literacy scale (eHEALS) to measure eHealth literacy, representing the domain of interaction with the eHealth system and access to the system. Furthermore, most studies were conducted in Eastern and Central-Africa, and more research is thus needed in a Western context.Health care providers have good individual eHealth literacy. However, more research is needed on the eHealth literacy domains dependent on PROSPERO International Prospective Register of Systematic Reviews (CRD42022363039). Morelth care .In the last decade, digital solutions of service provision and working methods have changed health care providers’ competence requirements . With ththe individual’s ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to address or solve a health problem. The concept was built on the well-known Lily model and measured with the tool eHealth literacy scale (eHEALS) [Domain dependent on the basic individual eHealth literacy; (2) Domain dependent on how the individual interacts with the eHealth system; and (3) Domain dependent on the system (e.g. access to hardware or an internet connection when needed). The interaction between the individual and the system is where unique aspects of the revised concept of eHealth literacy start to unfold [The concept of eHealth literacy was introduced by Norman and Skinner in 2006, defined as (eHEALS) . The Lil(eHEALS) . However(eHEALS) . Therefo(eHEALS) –14. This(eHEALS) , 12. FurSelf-rated competencies containing the concept of eHealth literacy was assessed by five out of 26 studies and all five used the eHEALS which represents the domain dependent on the basic individual eHealth literacy [In its broadest sense, eHealth is concerned with improving the flow of information, through electronic means, to support delivery of health services and the management of health systems . Despiteliteracy . HoweverTo identify existing research on different eHealth literacy measurements and domains among hospital health care providers.To identify eHealth literacy scores among hospital health care providers.To identify factors associated with eHealth literacy among hospital health care providers.A systematic review with a narrative synthesis was used. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed to minimise potential sources of bias . The proThe specific eligibility criteria defined a priori are presented in Table The systematic search strategy was designed to locate eligible studies published in English and Scandinavian languages. A team of clinical researchers  and a librarian (LBH) agreed on a search strategy for MEDLINE, which was adapted for use in Cinahl, Embase, Scopus, PEDro, AMED and Web of Science. The timeframe was from inception to November 18th, 2022, we sat no limit on the year of publication, as we wanted to describe the entire range of research relevant for our research questions. The search strategy was made available through DataverseNO . ArticleThe search results from the different electronic databases were combined in a single EndNote library by the librarian (LBH). The librarian identified and removed duplicates. All search results were subsequently uploaded to Rayyan (Rayyan Systems Inc) for storage and facilitation of blinding during the screening process. According to the pre-defined inclusion and exclusion criteria all selected titles and abstracts were scanned independently by two researchers (MHL) and (AKW). The full-text versions of potentially relevant articles were obtained and assessed independently for eligibility by two researchers (MHL and AKW). Any disagreements were resolved through discussion with a third researcher (GB). Three researchers  verified the final list of included studies. The reasons for the exclusion of full text publications were recorded using the PRISMA 2020 flow diagram . An overOne researcher (GB) extracted the data to a standardized data collection form that included the following data: year of publication, country of origin, the aims, study design, time period, sample size, health care profession, context, definitions of health literacy, eHealth literacy domains defined a priori, eHealth literacy measures, and findings related to the research questions of the review. A second researcher (AKW) assessed the data extraction for accuracy. Disagreements were resolved through discussion with a third researcher (MHL).The quality appraisal was systematically assessed using the Joanna Briggs Institute (JBI) critical appraisal checklist for analytical cross-sectional studies . The cheDomains dependent on the basic individual eHealth literacy; (2) Domains dependent on how the individual interacts with the eHealth system; and (3) Domains dependent on the system (e.g. access to hardware or an internet connection when needed). These three major eHealth literacy domains were based on the eHLF [A descriptive summary of the included publications was performed. To answer the first aim, the eHealth literacy scales used in the included studies were operationalised into three major domains defined a priori; (1) the eHLF . To answA total of 1,212 publications were identified from the systematic literature search after removal of duplicates. After the first screening, 26 publications were assessed in full text, of which 14 met the inclusion criteria. One study identified from citation searching was assessed as eligible. During the quality appraisal and the data extraction process, one more study was excluded as it did not meet the inclusion criteria for the population. Figure In total, 14 publications from 10 different studies were included. The characteristics of the included studies are presented in Table Two publications did not define the concept of eHealth literacy , 35, fouOverall, the mean eHEALS score in the studies ranged from 27.8 to 31.7 (8–40). The studies assessing eHealth literacy among nurses in South Korea , Turkey Protecting privacy, Operational skills, Navigation skills and Information searching. Furthermore, they had desirable level in Adding content, Determining data relevancy and Evaluating data reliability.The study, which used the Digital Health Literacy Instrument (DHLI) reported that the health care providers have very desirable literacy in the scales Using technology to process health information, Understanding of health concepts and language, and Ability to actively engage with digital services), while they reported a lower score on the Staff eHLQ scale 4 to 7 . Physicians scored higher on Staff eHLQ2 , but lower on Staff eHLQ scale 6 and eHLQ scale 7  compared to medical secretaries, nursing assistants and nurses.The study that used the Staff eHealth literacy questionnaire (Staff eHLQ) reported a relatively high score on the Staff eHLQ scale 1 to 3 (I know how to use the internet to answer my questions about the COVID-19 pandemic). They reported that 40 to 50% of health care providers had a higher eHealth literacy score [Two studies assessed eHealth literacy in relation to COVID-19 by using a questionnaire tailored for this purpose  compared to females. Moreover, the Staff eHLQ scale 1–4  was negatively correlated with age. Finally, the Staff eHLQ was positive correlated with experience of quick and easy access to information, sharing of data to reduce double registration and stability of IT systems [The mean eHealth literacy measured using DHLI was significantly different based on health care providers’ level of education, hospital affiliation and job category . The stu systems .The studies assessing eHealth literacy in relation to COVID-19 reported that higher eHealth literacy was associated with a higher educational level, higher income, access to a smartphone, high computer literacy and perception of digital tools as useful and easy to use. These participants also had a favourable attitude to eHealth, good knowledge of eHealth and a higher frequency of internet use , 29.To enhance trustworthiness, all studies were quality appraised using the JBI critical appraisal checklist for analytical cross-sectional studies. Overall, on average, the studies reported an adequate score (Yes) in four of the included six questions, however with some variety. One study reported an adequate score in six out of six questions , and oneOnly half of the studies presented an established definition of eHealth literacy, which is important for how the concept is operationalised and measured . The findings showed that health care providers have good individual eHealth literacy, primarily measured with the first-generation patient reported outcome measurement for eHealth literacy (the eHEALS) representing the domains dependent on the basic individual eHealth literacy. Assessments on the eHealth literacy domain dependent on how the individual interacts with the eHealth system and domain dependent the system itself (e.g. access to hardware that work) were almost absent. Moreover, the studies were conducted in six different countries, primarily non-Western countries. This, as well as the low number of studies being published on this topic in the era of eHealth development, indicate that eHealth literacy among hospital health care providers is underexplored globally, specifically in Western countries. This is important as eHealth literacy is a key to seeking health information online for appropriate decision-making . TherefoAs the eHEALS is based on the Lily model, which was developed for the first generation of eHealth back in 2006, these publications may not provide enough knowledge important to implementing eHealth resources that enable digital interactions with patients , 14. AppFurthermore, a discrepancy was identified between the definitions and measurements used in very many of the publications. A previous systematic review has also reported this unfortunate absence of established eHealth literacy definitions . AdditioThe hospital health care providers reported an eHEALS score of between 27.8 and 30.7 (8–40) which represents a high self-perceived eHealth literacy, compared with eHEALS scores reported by younger adults in South Korea who are active online users (eHEALS score 28.06)  and by aInterestingly, eHealth literacy was higher among physicians, than other health care providers. This could be related to physicians’ prior digital training and higher level of education, which may be associated with higher eHealth literacy . These rIt is a paradox that the countries that have measured eHealth literacy among hospital health care providers are those assumed to have lower awareness and utilisation of eHealth technology, such as Ethiopia, and not the countries that have implemented eHealth on a larger scale. Importantly, hospital health care providers in Ethiopia with a higher eHealth literacy score also had better computer access and knowledge, and perceived digital tools as useful and easy to use , 34, 35.The strength of our systematic mixed studies review lies in its methodological rigour. A systematic approach to collecting data, including a broad search strategy in six databases, developed in close collaboration with an experienced research librarian. The PRISMA guidelines were used to minimise potential sources of bias. In addition, study selection, quality assessment and data extraction were conducted systematically and in parallel by two independent researchers. One limitation could be that the current systematic review only includes quantitative studies, and qualitative studies might have added valuable knowledge to the field.Furthermore, the measurements used are based on Western conceptualization of eHealth literacy and operationalisation, which may not harmonise with the worldviews of all participants in the studies using these instruments . TherefoThe result from this systematic review shows that health care providers have good individual eHealth literacy. However, more research is needed on the eHealth literacy domains dependent on how the individual interacts with the eHealth system and on the system itself, using more comprehensive measures. Studies are also needed to measure multidisciplinary professions’ eHealth literacy in the hospital context. Furthermore, most of the studies were conducted in Eastern and Central-Africa, and more research of higher methodological quality is needed in a Western context.


Arabidopsis thaliana as maternal contributor and A. arenosa or A. lyrata as pollen donors result in an endosperm based post-zygotic hybridization barrier that gives rise to a reduced seed germination rate. Hybrid seeds display opposite endosperm cellularization phenotypes, with late cellularization in crosses with A. arenosa and early cellularization in crosses with A. lyrata. Stage specific endosperm reporters display temporally ectopic expression in developing hybrid endosperm, in accordance with the early and late cellularization phenotypes, confirming a disturbance of the source-sink endosperm phase change. We demonstrate that the hybrid barrier is under the influence of abiotic factors, and show that a temperature gradient leads to diametrically opposed cellularization phenotype responses in hybrid endosperm with A. arenosa or A. lyrata as pollen donors. Furthermore, different A. thaliana accession genotypes also enhance or diminish seed viability in the two hybrid cross-types, emphasizing that both genetic and environmental cues control the hybridization barrier. We have identified an A. thaliana MADS-BOX type I family single locus that is required for diametrically opposed cellularization phenotype responses in hybrid endosperm. Loss of AGAMOUS-LIKE 35 significantly affects the germination rate of hybrid seeds in opposite directions when transmitted through the A. thaliana endosperm, and is suggested to be a locus that promotes cellularization as part of an endosperm based mechanism involved in post-zygotic hybrid barriers. The role of temperature in hybrid speciation and the identification of distinct loci in control of hybrid failure have great potential to aid the introduction of advantageous traits in breeding research and to support models to predict hybrid admixture in a changing global climate.Speciation involves reproductive isolation, which can occur by hybridization barriers acting in the endosperm of the developing seed. The nuclear endosperm is a nutrient sink, accumulating sugars from surrounding tissues, and undergoes coordinated cellularization, switching to serve as a nutrient source for the developing embryo. Tight regulation of cellularization is therefore vital for seed and embryonic development. Here we show that hybrid seeds from crosses between  A specibidopsis , CapsellCapsella , rice . Cellularization of the endosperm marks a transition in seed development, as up to this point, the endosperm functions as a nutrient sink. At this developmental time point, the endosperm concurrently switches from nutrient sink to source for the developing embryo , and manArabidopsis has been widely used for studying evolutionary questions  shifted to be maternally expressed genes (MEGs)  . Importas (MEGs) . A majorArabidopsis is reliant on the FERTILIZATION INDEPENDENT SEED-Polycomb Repressive Complex 2 (FIS-PRC2)  . FIS-PRCIS-PRC2) . DeregulIS-PRC2) . One excbidopsis . The clebidopsis , with mubidopsis , whereasrization . AGL62 mion rate .A. thaliana mothers with A. arenosa or A. lyrata pollen donors result in diametrically opposed endosperm phenotypes, both giving rise to reduced seed germination rate, albeit caused by late cellularization in crosses with A. arenosa and early cellularization in crosses with A. lyrata. We demonstrate that the hybrid barriers are under the influence of abiotic factors, and show that a temperature gradient leads to opposed cellularization phenotype and seed viability in hybrid endosperm with A. arenosa or A. lyrata as pollen donors. In addition, A. thaliana accession genotypes also influence seed viability in the two hybrid cross-types in opposite directions. Using stage specific endosperm reporters, we demonstrate that the source-sink endosperm phase change is delayed or precocious in seeds of the two hybrids. Our data suggests an A. thaliana type I MADS-BOX family locus to act as a promoter of endosperm cellularization, affecting the germination rates of A. arenosa or A. lyrata hybrid seeds in opposite directions.Here we report that hybrid seeds from crosses between 22.1A. thaliana accessions  and mutant lines were obtained from the Nottingham Arabidopsis Stock Center (NASC). The A. arenosa population MJ09-4 originates from Nízke Tatry Mts.; Pusté Pole  and the A. lyrata subsp. petraea population MJ09-11 originates from lower Austria; street from Pernitz to Pottenstein  (agl35-1 (SALK_033801), agl40-1 (SALK_107011) and mea-9 (SAIL_724_E07) were in Col-0 accession background  and proAT4G00220>>H2A-GFP (TE1-GFP) were in Col-0 accession background (15.9755) . Mutant ckground . The aglckground .A. thaliana) or 10 days  before germination at 22°C with a 16h/8h light/dark cycle. After two weeks, seedlings were transferred to soil and cultivated at 18°C . A. arenosa strain MJ09-4 was previously demonstrated to be diploid  and wash solution (0.001% Tween20) for 5 min each step . Sterili diploid . A. lyra2.2A. thaliana plants were emasculated 2 days before pollination. Crossed plants were placed at experimental growth temperature until silique maturity/harvesting. For each cross combination, 4-8 different individual plants were used as pollinators and 15-85 siliques  were harvested individually . These rare events occurred at a frequency less than 1% and self-plants were removed from the analyses if present.vidually . All see2.3ggplot2 (dplyr (car package (ggpubr package (R-studio (version 2023.03.1 + 446)  was usedggplot2  and dply2 /LOG(2)).Feulgen stained seeds were harvested at 6 days after pollination (DAP), stained using Schiff’s reagent (Sigma-Aldrich S5133), fixed and embedded in LR White (London Resin) . Imaginghttps://imagej.nih.gov/ij/).Crosses with EE-GFP and TE1-GFP markers were imaged using the Andor DragonFly as described. Whole-mount imaging of seeds was performed using an Axioplan2 imaging microscope after 24 h/4°C incubation in 8:2:1 (w/v/v) chloral hydrate:water:glycerol . Mature 33.1A. thaliana crossed with A. lyrata or A. arenosa, we first observed the seed-set. Hybrid crosses with A. lyrata fathers had significantly reduced seed set per silique compared to crosses with A. arenosa fathers or A. thaliana (Col-0) self crosses  hybrid seeds with A. arenosa or A. lyrata fathers, we used genetic markers of endosperm development that are expressed before and after endosperm cellularization in A. thaliana  and no visible downregulation could be observed  mothers and A. lyrata or A. arenosa fathers at 4°C temperature windows ranging from 14°C to 26°C. Interestingly, the germination rate of A. thaliana × A. arenosa hybrid seeds was significantly enhanced by progressive lowering of the temperature, and contrasted by the germination rate of A. thaliana × A. lyrata hybrid seeds that was significantly enhanced by progressive increase of the temperature  germination rate in the A. lyrata hybrid cross  was not significantly different from wildtype (Col-0) when crossed to A. arenosa. Both homozygous (agl35-1 -/-) and heterozygous agl35-1 +/- plants crossed to A. arenosa displayed significantly lower germination rate than Col-0 crossed to A. arenosa, indicating that the increased strength of the hybrid barrier was caused by mutation of AGL35. Furthermore, the observation that the strength of the hybrid barrier can be caused by heterozygous (agl35-1 +/-) plants crossed to A. arenosa indicates that the observed phenotype is caused by genetic interaction occurring in the fertilization products, the embryo or the endosperm.To test if cryptic genetic variation in the 4Recent advances in elucidating molecular mechanisms and genetic networks in hybrid endosperm lethality suggest that imprinted genes and genetic variation in the hybrid parents are important factors that can enhance or repress the frequency of the endosperm-based barrier . HoweverA. lyrata or A. arenosa as the paternal parent to A. thaliana mothers show antagonistic endosperm cellularization phenotypes, with late cellularization in crosses with A. arenosa and early cellularization in crosses with A. lyrata. In both cases, cellularization failure results in an endosperm-based hybrid barrier and reduced viability of germinating seeds. This compares to previous studies where timing of endosperm cellularization is influenced by the paternal species in reciprocal A. arenosa and A. lyrata interspecies crosses and in crosses within the genus Capsella  can bypass the cellularization phenotype in paternal excess inter-ploidy crosses . The same temperature range has an opposite effect in Col-0 × A. lyrata seeds resulting in increased germination rates (more than 50%). This demonstrates a temperature dependent genetic mechanism that acts antagonistically when A. thaliana is crossed to A. arenosa or A. lyrata and produces diametrically opposed cellularization phenotype responses in the hybrid endosperm.We found that by increasing the temperature from 14°C to 26°C, Col-0 × Brassica oleracea, temperature affects abscisic acid (ABA) levels specifically in the endosperm and cooler temperatures obstruct the breakdown of ABA in the desiccating endosperm , and even larger than the interploidy effect. While the Col-0 and Ws-2 A. thaliana accessions resulted in a generally higher germination rate when hybridized with A. lyrata compared to A. arenosa, the C24 accession had the opposite effect. The way the accessions affected hybrid seed viability in opposite directions may point at a similar mechanism as observed in the temperature experiment. However, our results do not readily explain the observed germination rates by cellularization phenotype alone, and further investigations are required to resolve these observations.Notably, the effect of using different A. thaliana Wa-1 to diploid A. arenosa increases hybrid seed survival  type I MADS-box TFs are highly expressed in the seed, specifically during endosperm cellularization . Their indosperm  and upre crosses . Our resnd AGL35 . These And AGL35 . Paterna failure . AGL62 i complex  whereas 4.4The findings in this study introduce a rigorous model system for the dissection of the influence of abiotic and genetic parameters in hybrid admixture, and have a large potential to support breeding and climate research. Further examination and usage of these approaches could help pinpoint genes, networks or gene dosage balances that are involved in overcoming the endosperm-based hybridization barrier. Species previously thought to be unable to hybridize due to postzygotic seed lethality may be able to do so given favorable conditions, and a similar effect could also apply to interploidy hybrids.Currently, it is not known if the temperature effect on hybridization success is mediated by the same genetic network that is operated by changes in ploidy or genetic variation. Phenotypically, the temperature effect restores defects in timing of cellularization, but it is not known if the trigger is upstream or downstream of the causative genetic network. Elucidation of the genetic, epigenetic and mechanistic basis for this cross talk between the genic and environmental factors is therefore essential for our understanding of the plasticity of endosperm-based hybridization barriers.The original contributions presented in the study are included in the article/PG, RA, KB, and AB designed the research; RA, IM, and KB performed the experiments; PG, RA, KB, and AB analyzed and discussed the data; PG, RA, KB, and AB wrote the article; All authors contributed to the article and approved the submitted version.


Its importance relies on the highlight of the logical fallacy where a hypothesis is assumed to be true or false before being scientifically and satisfactorily investigated. With that in mind, in this editorial, we comment on how this premise directly applies to clinical practice.“Absence of Evidence is not Evidence of Absence” is a quote by Carl Sagan, an American astronomer and one of the leading science communicators of the 20 However, several treatments lack sufficient evidence to justify an upfront recommendation without further consideration. In this case, we must understand what kind of “absence of evidence” we are dealing with. Two scenarios may apply.When a certain treatment modality has already been investigated and proven superior to another one or even to traditional therapy, the clinical decision is straightforward. As a result, we should select this modality over others and apply it to our patients, as its benefits are unambiguously supported by reliable scientific proof. whereas other antibiotic protocols have demonstrated to be more effective than doxycycline. Therefore, this treatment should not be selected over others, as it has already been tested and showed no benefits to justify its use.The first one is when we are dealing with a therapy that has been extensively tested but failed to show beneficial results when compared with others. In this case, the recommendation should be not to use it. Let’s consider a practical example related to periodontitis. Several studies have evaluated the effects of systemic doxycycline as an adjunct in the treatment of patients with severe periodontitis. Nevertheless, these studies have failed to show a consistent benefit of this agent,The second scenario – the one addressed in the quotation in question – is when we lack reliable evidence for recommending or not a specific therapy because it has not yet been comprehensively evaluated by clinical trials. In this case, we cannot simply assume that the treatment is effective or not. In other words, lacking evidence does not imply any benefit conclusions of a specific therapy or even the existence of a benefit itself. Here, the decision should also be not to recommend the treatment.Neither few, nor poorly designed studies, nor investigations enrolling small samples, nor research resulting in clinically irrelevant differences should ever support clinical decision-making. Yet, in some circumstances, such as clinical conditions that may incur risk for the patient’s general health  or harmless therapies supported by moderate evidence , the clinical decision may be guided by three main factors: (1) “Biological Plausibility,” (2) Risk/Benefit Analysis, and (3) Cost/Benefit Analysis. Here, the “biological plausibility” for prescribing the antibiotics should consider (i) the type and duration of the surgical procedure, (ii) if the surgery involved bone removal and extension of the osteotomy, and others.For (1) “biological plausibility,” examples of clinical situations include using or not an antibiotic after conducting a connective gingival graft or a periodontal regeneration procedure, or after the placing of a single implant. Finally, one should also evaluate (3) the cost/benefit balance before choosing a certain treatment, that is, what is the investment for both patient and practitioner to apply a therapy that has not been proven to have a real clinical benefit – even for unhazardous treatments. The cost-benefit analysis should be applied to any clinical question, especially when dealing with high-cost therapies, such as those involving some biomaterials or expensive laser equipment.Regarding (2) the risk/benefit analysis, one should consider (i) the direct risk for the patient, (ii) the risk for the population, and (iii) the risk for the environment. For example, when dealing with antibiotics, we must consider the risk to the patient , as well as to society as a whole. The indiscriminate use of antibiotics may increase bacterial resistance to these drugs in the general population and decrease their effectiveness, including for lethal diseases.– some of them even being recommended as “established protocols.” On the other hand, protocols that have been already supported by reliable scientific evidence may take too long to be incorporated into clinical practice.The aforementioned reasoning entails how to approach every situation when deciding which is the best treatment protocol for our patients. Our concern about the dental field is that we often see treatments that are not evidence-based being widely used in clinical practice Finding, discriminating, and interpreting scientific literature is one of the most relevant abilities of a health care professional, and we should pursue this goal!Researchers and clinicians must understand which therapies are proven effective or ineffective, and differentiate them from those that still need to be properly investigated. This will help support effective clinical practice and identify gaps in knowledge. To achieve this goal, we must advance our understanding of clinical trials and systematic/scoping reviews of clinical and laboratory data.