Non-Fungible Tokens (NFTs) are digital assets that are verified using blockchain technology to ensure authenticity and ownership. NFTs have the potential to revolutionize healthcare by addressing various issues in the industry.The goal of this study was to identify the applications of NFTs in healthcare. Our scoping review was conducted in 2023. We searched the Scopus, IEEE, PubMed, Web of Science, Science Direct, and Cochrane scientific databases using related keywords. The article selection process was based on Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).After applying inclusion and exclusion criteria, a total of 13 articles were chosen. Then extracted data was summarized and reported. The most common application of NFTs in healthcare was found to be in health data management with 46% frequency, followed by supply chain management with 31% frequency. Furthermore, Ethereum is the main blockchain platform that is applied in NFTs in healthcare with 70%.The findings from this review indicate that the NFTs that are currently used in healthcare could transform it. Also, it appears that researchers have not yet investigated the numerous potentials uses of NFTs in the healthcare field, which could be utilized in the future. With the pressure of long-term chronic disease, rising costs of services, aging population, an inadequate health workforce, and finite resources, we have long known that the present healthcare system is unsustainable. With the recent growth of digital technologies, the healthcare industry is poised to shift toward digital health as a means of addressing the various challenges it faces –4. DigitFor several years running, Gartner a technological research and consulting firm has ranked blockchain as one of the top 10 strategic technology trends , 11–14. According to Peres et al. , NFTs arBao and Roubaud  conducteAlthough there is much evidence for the use of NFTs in the cited fields, there has not been much study on its use in healthcare. Recently, three major studies on the value of NFTs and its part in the future of healthcare were published. In the first recent study published in Nature Medicine, researchers mentioned that NFTs in healthcare might revolutionize how health data is handled. They discussed transparency, accountability, privacy, and patient ownership as being central to NFTs, which also important ideals in the field of healthcare. The authors of the study concluded that if NFTs are to revolutionize digital health, more improvements to protect data security, streamline operations, and promoting accessibility for all will be necessary . AnotherDespite the growing interest in exploring the potential applications of NFTs in healthcare , 45, theThe primary objective of this investigation was to delineate the applications of NFTs within the domain of healthcare. In the year 2023, we executed a comprehensive scoping review, meticulously querying prestigious scientific databases, namely Scopus, IEEE, PubMed, Web of Science, Science Direct, and Cochrane. Our search was guided by relevant keywords, and the selection of articles adhered to the guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Following the application of predefined inclusion and exclusion criteria, a meticulous curation process led to the inclusion of a total of 13 articles. Subsequently, the collated data underwent meticulous summarization and reporting. Further elucidation of the methodological aspects is presented in given below.RQ1: What are the main applications of NFTs in healthcare?RQ2: What is the current state of NFTs usage in healthcare?The search started in February and ended on the first of March 2023. Searches were done in scientific databases, including Scopus, PubMed, Web of Science, IEEE, Science Direct, and Cochrane. Given the interdisciplinary nature of our research, we extended our search to encompass databases and knowledge bases in the fields of health sciences, engineering, and management.The combination of related keywords is demonstrated in i. Only studies that explicitly mention the applications of NFTs are included.ii. Only studies that show any use cases of NFTs in healthcare with health promotion goals included.iii. Only original research included.iv. Only English publications included.v. Only journal articles and conference papers included.The process of article selection was based on PRISMA and 13 papers were selected . All theAccording to predefined inclusion and exclusion criteria, the present study incorporated a total of 13 research investigations from an initial pool of 188 studies identified in the initial database search. The comprehensive delineation of these selected studies, including the various article categories, is displayed in Subsequent to data extraction, the study's outcomes were summarized and depicted in the form of tables and figures, aligning the presentation of results with the overarching study objectives. Also, to enhance clarity and facilitate comprehension, the results have been thoughtfully segregated for presentation, focusing on the primary columns of Most of the papers (77%) were published in 2022 . 62% of As presented in the fourth column of As showed in the fifth column of As presented in the sixth column of The seventh column of As shown in the eighth column of As presented in the tenth column of The eleventh column of NFTs have gained a lot of attention in recent years as a means of establishing ownership and authenticity of digital assets , 21, 23.The findings of this study indicate a significant increase in the utilization of NFT-based solutions within the healthcare industry over the past 2 years, particularly in 2022. Specifically, the investigation revealed that 10 out of the 13 studies included in the analysis were conducted in 2022. These results align with other studies that have identified a growing trend in publications that focus on the applications of NFTs worldwide . CollectAnother noteworthy finding is that 62% of the studies conducted in this domain were classified as developmental studies, with the remaining 38% falling into the category of conceptual research. An important related fact is that the applications of NFTs in healthcare extend beyond the realm of theoretical and conceptual frameworks, with researchers actively working to implement these solutions in developmental and practical contexts within healthcare. This observation is consistent with our earlier findings, which indicated that most investigations focused on the applications of NFTs, in general, were not restricted to conceptual frameworks alone .Given that most of the studies included in this analysis were classified as developmental, it is noteworthy that the system development life cycle emerged as the primary applied methodology across these investigations. SDLC is a methodology used in software engineering to guide the development of information systems, software applications, and other technology-based projects. It provides a framework for the development of software and systems from inception to retirement, with the aim of producing high-quality, efficient, and effective solutions that meet the needs of stakeholders .Our analysis reveals that approximately 70% of the researched or proposed projects in this study utilized the Ethereum blockchain, while 20% employed Solana, and 10% chose Hyperledger Sawtooth. Notably, Ethereum stands as the dominant choice for NFT applications in healthcare due to factors such as its robust developer community, market value, inherent support for Decentralized Applications (DApps), tokenization capabilities , and backing for non-fungible digital assets. It's important to note that while Ethereum typically follows the ERC-20 standard , which pAside from the mapped blockchain solutions capabilities, limitations that were not addressed or needed to be better addressed for the successful application of NFTs for healthcare are presented in Although many of the limitations listed above are equally relevant, interoperability, standardization, and centralization were found to be of high importance across all technologies for the healthcare domain. For example, centralization regards imbalances in consensus algorithms that would impact NFTs minting and sales, impacting data owners. Standardization regards the implementation of technologies, standards, frameworks, and policies agreed upon by a committee. If standardization is not present at some level to regulate how NFT are created, the risk for users is increased . FinallyBased on the Technology Readiness Level (TRL) scale , the preThe classification of NFTs applications in healthcare utilized in this study was primarily based on the work of Musamih et al. . Based oNFTs can offer several benefits for health data management, such as enhancing the security and privacy of sensitive patient information through encryption and decentralized storage on a blockchain network. NFTs can also enable seamless and tamper-proof tracking of medical records, facilitating more accurate diagnoses and treatments. Moreover, NFTs can provide patients with greater control over their health data by empowering them to grant permission for access to their records and receive compensation for its use , 64, 65.NFTs can be utilized in healthcare supply chain management to improve transparency, traceability, and accountability. By creating a secure and tamper-proof record of the transportation of medical supplies and equipment, NFTs can help to prevent counterfeiting and improve efficiency in the distribution process. Additionally, NFTs can enable better tracking of the temperature and conditions in which medical products are stored and transported by attaching a unique digital certificate to each item that can be tracked on the blockchain to ensuring that they remain viable and safe for use , 54, 66.NFTs can have potential applications in genomics by providing secure and efficient management of genetic data. By creating a tamper-proof record of genomic information, NFTs can enable more accurate and personalized healthcare interventions. NFTs can also facilitate secure sharing and storage of genetic data for research purposes while preserving patient privacy. Furthermore, NFTs can allow for equitable and transparent distribution of financial benefits arising from the use of genetic data .NFTs can be applied to create digital twins, which are virtual models of patients that can be used to simulate medical scenarios and optimize treatments. By creating a tamper-proof record of a patient's health data, NFTs can enable more accurate and personalized digital twin models. Additionally, NFTs can allow for secure and transparent sharing of digital twin data for research purposes while protecting patient privacy. Furthermore, NFTs can enable patients to have greater control over the use of their digital twin data and receive compensation for its use .NFTs can safeguard the tracking of the spread of infectious diseases by creating tamper-proof records of patient health data and facilitating secure sharing of information between healthcare providers. Generally, NFTs can help to improve infection tracking in healthcare by creating a tamper-proof record of the movement of people and equipment within healthcare facilities. By attaching NFTs to equipment or even patient wristbands, healthcare providers can track the movement of people and things within their facilities, making it easier to trace the spread of infections. NFTs can also be used to record the temperatures of equipment used in patient care, such as ventilators or IV pumps, ensuring that they are within safe operating ranges. This information can be used to quickly identify potential sources of infection and promote more effective disease control measures .NFTs can potentially accelerate the drug discovery process by enabling secure and transparent sharing of research data and intellectual property among researchers and organizations. By creating an immutable record of ownership and usage rights, NFTs can facilitate collaborations while also protecting the intellectual property of researchers. This can lead to more efficient drug discovery and development processes, ultimately leading to the creation of new treatments for a variety of health conditions. Additionally, NFTs can be used to tokenize the data generated during clinical research, making it easier for researchers to analyze and share data, leading to faster and more accurate drug development .NFTs can be used to create information models for designing healing and therapeutic interventions, allowing for the development of more personalized and effective treatment plans. By creating digital representations of patient health data, NFTs can enable the analysis of large amounts of information and support data-driven decision making for healthcare providers .According to the findings of this study, the green circles represent current applications of NFTs in healthcare, while the red circles denote potential future use cases of NFTs within the healthcare industry.Another noteworthy discovery from this investigation is that most studies included in the analysis incorporated evaluation phases. This finding is significant in that it highlights the importance of assessing the performance of NFT-based solutions and draws attention from researchers and stakeholders in this domain. Additionally, the results of this study revealed that the primary dimension evaluated in these investigations was security. One potential explanation for this phenomenon may be attributed to the significant advantage of utilizing NFT-based solutions, which is the enhancement of security. This finding is consistent with other investigations .The present study identified the main value propositions of NFT-based solutions in healthcare which primarily centered around enhancing security, privacy, confidentiality, transparency, and accessibility. The present study aligns with previous research that demonstrates the significant benefits of NFTs-based solutions, including unique digital ownership, enhanced security and privacy, the potential for increased value, traceability, accessibility, and interoperability , 27, 31.While the potential applications of NFTs in healthcare such as improve data security, track the provenance of pharmaceuticals and medical devices, and streamline the supply chain for medical products are promising, however, it is important to acknowledge and address the associated challenges and concerns. First and foremost, the integration of NFTs in healthcare systems may introduce security vulnerabilities, as healthcare data is highly sensitive and susceptible to cyberattacks , 49, 68.Also, it is essential to differentiate between the applications of NFTs and the broader blockchain technology in the context of healthcare due to their unique contributions to the industry. While both NFTs and blockchain offer secure and transparent data management, NFTs stand out for their distinct ability to represent ownership, authenticity, and provenance of digital assets. This differentiation becomes crucial when addressing healthcare challenges, as certain applications may require the specific features that NFTs provide. By understanding when and where NFTs are most advantageous, healthcare stakeholders can make informed decisions about technology adoption, potentially leading to more effective, efficient, and patient-centric solutions. Moreover, a clear distinction between NFTs and blockchain allows for a more precise analysis of how each technology contributes to resolving healthcare issues, ensuring that the appropriate tools are applied to the task at hand. In our ongoing research, we aimed to elucidate the precise roles of NFTs and broader blockchain technology in healthcare applications, shedding light on the distinct advantages and synergies that drive innovation in the field , 49, 61.While this scoping review provides valuable insights into the burgeoning utilization of NFTs in healthcare, it is essential to acknowledge certain limitations in our study. Firstly, the relatively limited number of selected papers (13) may not encompass the entirety of NFTs applications in healthcare, and there might exist additional relevant research that was not included. Furthermore, the rapidly evolving nature of both blockchain and NFTs technologies implies that the field is subject to continuous developments and advancements that may not be fully captured in our review. Additionally, the potential biases inherent in the selection and analysis of the included studies, as well as the use of predetermined search criteria, could influence the comprehensiveness of the findings. Therefore, while our review underscores the transformative potential of NFTs in healthcare, it is important to consider these limitations as areas for further research and exploration in this dynamic and innovative domain.Future research in NFTs-based solutions for healthcare should focus on several areas to address gaps in the existing literature. First, more studies are needed to explore the ethical, social, and cultural implications of using NFTs in healthcare. This includes examining potential issues related to data privacy, ownership, and accessibility, as well as the impact on healthcare providers and patients. Second, future research should investigate how NFTs can be integrated into existing health systems and workflows, including exploring potential barriers to adoption and identifying strategies for effective implementation. Third, there is a need for more empirical studies to evaluate the effectiveness and efficiency of NFTs-based solutions in healthcare. Finally, researchers should explore the potential for using NFTs in other areas of healthcare beyond those currently identified. By addressing these areas of research, we can better understand the potential benefits and challenges of using NFTs in different areas of healthcare and develop strategies to maximize their value for patients, providers, and other stakeholders in the healthcare ecosystem.This scoping review highlights the emerging use of NFTs in healthcare. The study searched for and reviewed 13 relevant papers, and the results provide valuable insights into the current state of NFTs in healthcare. The study also explored various variables, including type of study, methodology, technical aspects of NFTs-based solutions, TRL levels, applications of NFTs in healthcare, evaluation type and results, and the value proposition of NFTs. The results revealed that NFTs have potential applications in healthcare supply chain management, data management, clinical trial management, digital twin, genomic, and other areas. In addition, this study has uncovered further potential uses for NFTs in healthcare. More specifically, three of the reviewed studies proposed NFT-based solutions for drug discovery, tracking infections, and utilizing information modeling to design healing and therapeutic interventions. Additionally, increased security, privacy, confidentiality, transparency, and accessibility were found to be the most common value propositions of NFTs-based solutions. Overall, the results of this scoping review suggest that NFTs have the potential to revolutionize the healthcare industry; however, further research is needed to explore their full potential.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.SA: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Writing—original draft. PMo: Conceptualization, Project administration, Supervision, Writing—review & editing. PMi: Conceptualization, Data curation, Formal analysis, Validation, Writing—original draft. AG: Data curation, Formal analysis, Validation, Writing—original draft. TH-P: Conceptualization, Writing—review & editing. DC: Funding acquisition, Resources, Writing—review & editing.


The incorporation of the 1,2-disubstituted bicyclo[2.1.1]hexane core into the structure of fungicides boscalid (BASF), bixafen (Bayer CS), and fluxapyroxad (BASF) gave saturated patent-free analogs with high antifungal activity.Bicyclo[2.1.1]hexanes have been synthesized, characterized, and biologically validated as saturated bioisosteres of the  ortho-substituted benzene ring.1,2-Disubstituted bicyclo[2.1.1]hexanes have been synthesized, characterized, and biologically validated as saturated bioisosteres of the  It is the most popular ring in natural productsortho-substituted benzene ring, in particular, is found in the structure of more than three hundred drugs and agrochemicals , bixafen (Bayer CS), and fluxapyroxad (BASF) to provide saturated patent-free analogs with high antifungal activity.In this work, we have synthesized, characterized, and biologically validated 1,2-disubstituted bicyclo[2.1.1]hexanes as a new generation of saturated bioisosteres of 6H6; M.W. = 78) is an aromatic molecule. Among the conformationally rigid bi(poly)cyclic saturated scaffolds, – bicyclo[1.1.1]pentane,16 bicyclo[3.1.1]heptane,17 cubane,18 and bicyclo[2.1.1]hexane  at the 1- and 2-positions of the core . The compound contained ca. 10% of the isomeric diene, as the undesired alkylation at the methyl group also took place  after distillation. Saponification of the ester group in 1a followed by crystallization from hexane-tBuOMe gave pure carboxylic acid 1b as a white crystalline solid in 70% yield. The structure of compound 1b was confirmed by X-ray crystallographic analysis , meta (11–14), and ortho (15), – gave the corresponding bicyclo[2.1.1]hexanes 4a–15a in 67–73% yield. The photocycloaddition was also compatible with the fluorine atom directly attached to the diene structure (3 and 10). Various medicinal chemistry-relevant heterocycles, such as pyrazole (16 and 17), imidazole (18), thiophene (19), furane (20), and pyridine (21–23), provided the desired bicyclo[2.1.1]hexanes 16a–23a in 57–76% yield. All products 3a–23a were purified by distillation. Saponification of the ester group in 3a–23a followed by crystallization gave solid carboxylic acids 3b–23b in 63–76% yield.We studied the scope of the developed method. The photochemical reaction tolerated various substituents at the aromatic core . Among t23All final products were synthesized in gram quantities. The structures of bicyclo[2.1.1]hexanes 1b, 3b, 4b, 10b, and 12b were confirmed by X-ray crystallographic analysis .232H). We aimed to perform some representative modifications of these bicyclo[2.1.1]hexanes to obtain linkers for medicinal chemistry – compounds with two functional groups.Compounds 1b and 3b–23b possess one functional group (–CO4/RuCl3 gave linker 24 (etc.).24Esterification of carboxylic acid 1b, followed by oxidation of the phenyl group with NaIOinker 24 . The fluortho-substituted benzene ring. For this, we employed the exit vector plot tool. In this method, two substituents on the scaffold were simulated by two exit vectors n1 and n2  and φ2 (between vectors n2 and C–C), and the dihedral angle θ defined by vectors n1, C–C and n2. An additional representative parameter – the distance d between two carbon substituents  vs. 3.04–3.10 Å (ortho-benzene).The corresponding parameters for misartan .25 Analyr in bicyclo[2.1.1]hexane was ca. 0.1–0.2 Å longer than that in the ortho-benzene ring: 1.56 Å  vs. 1.39–1.41 Å (ortho-benzene).The distance φ1 and φ2 were similar in both scaffolds: 61–65°  vs. 55–57° (ortho-benzene). The difference in planarity, however, between the saturated scaffold and the benzene ring was significant: while ortho-benzene was almost flattened, bicyclo[2.1.1]hexanes had a significant three-dimensional character:/θ/= 56–59°  vs. 7–8° (ortho-benzene).Angles r and d and angles φ1 and φ2 of 1,2-disubstituted bicyclo[2.1.1]hexanes were similar to those of the ortho-substituted benzene ring.In general, both distances ortho-substituted benzene ring: agent for hyponatremia treatment conivaptan, lipid-lowering agent lomitapide  increased the solubility by threefold: 5 μM (conivaptan) vs. 14 μM (26) (vs. 18 μM (27).Replacement of the  μM (26) . An analortho-benzene ring by bicyclo[2.1.1]hexane in agrochemicals boscalid, bixafen, and fluxapyroxad produced conflicting results. In boscalid, such a replacement led to a significant threefold increase in solubility: 11 μM  vs. 35 μM (28). In bixafen, the opposite effect was observed, and the solubility was reduced: 30 μM (bixafen) vs. 4 μM (29).26 In fluxapyroxad, such a replacement resulted in a slight increase in solubility: 25 μM (fluxapyroxad) vs. 27 μM (30).Replacement of the ortho-benzene ring by bicyclo[2.1.1]hexane led to an enhanced water solubility.In conclusion, in four out of the five bioactive compounds, the replacement of the ortho-benzene ring with bicyclo[2.1.1]hexane on lipophilicity, we used two parameters: calculated lipophilicity (c log P)27 and experimental lipophilicity (log D).To estimate the influence of the replacement of the ortho-benzene ring in conivaptan, lomitapide, boscalid, bixafen, and fluxapyroxad with bicyclo[2.1.1]hexane (26–30) led to a decrease in c log P by 0.7–1.2 units.Replacement of the ortho-benzene ring with bicyclo[2.1.1]hexane had only a small effect on the logD index. In four bioactive compounds  such a replacement almost did not affect log D. Only in lomitapide, the saturated analog 27 had a significantly lower log D index: >4.5 (lomitapide) vs. 3.9 μM (27).Replacement of the ortho-benzene ring by bicyclo[2.1.1]hexane led to a decrease in calculated lipophilicity (c log P) by 0.7–1.2 units; and in four of them it had little effect on the experimental lipophilicity (log D).In summary, in all five bioactive compounds, replacement of the int (μL min−1 mg−1) = 31 (conivaptan) vs. 12 (26). In lomitapide, bixafen, and fluxapyroxad the incorporation of bicyclo[2.1.1]hexane  dramatically decreased the metabolic stability by two (30) to three  times, as measured by using t1/2 (min). In boscalid, such a replacement led to a slight decrease in metabolic stability: CLint (μL min−1 mg−1) = 26  vs. 29 (28)  increased the metabolic stability: CL 29 (28)  and 4.ortho-benzene ring with bicyclo[2.1.1]hexane decreased the metabolic stability.In brief summary, in four out of the five bioactive compounds, replacement of the ortho-benzene ring in bioactive compounds?9 Therefore, we measured the antifungal activity of the marketed fungicides boscalid (BASF), bixafen (Bayer CS), fluxapyroxad (BASF), and their saturated analogs 28–30. In strict contrast to medicinal chemistry, the use of racemic mixtures in agrochemistry is common;5 therefore for the validation of the proof-of-concept, we directly studied the biological activity of the available racemic compounds 28–30  on a multigram scale (ortho-substituted benzenes (In this work, we have synthesized, characterized, and biologically validated 1,2-disubstituted bicyclo[2.1.1]hexanes as saturated bioisosteres of the am scale  and 2. Pbenzenes . The incWe believe that given the commonplace of the phenyl group in chemistry, its saturated bioisosteres developed here will become common in medicinal chemistry in the coming years.The ESIA. D. and P. K. M. designed the project. A. D., P. G., Y. M., O. S., G. A.-M., and R. K. carried out the experiments. I. V. S. and P. K. M. wrote the manuscript and all authors provided comments.A. D., P. G., Y. M., O. S., G. A.-M., R. K., I. V. S., and P. K. M. are employees of a chemical supplier Enamine.SC-014-D3SC05121H-s001SC-014-D3SC05121H-s002


A positive, prosocial classroom climate is associated with improved social competence and academic achievement, as well as with decreased internalizing problems and antisocial behavior in children. It is expected that motivation to behave prosocially is needed to achieve a prosocial climate in the classroom, and that such motivation can be enhanced through three components of self-determination theory (SDT): competence, relatedness, and autonomy. The goal of this protocol is to describe the design of a study aiming to evaluate the effectiveness of a classroom-based program based on SDT components promoting a prosocial classroom climate.A cluster randomized controlled trial (RCT) will be conducted to examine the effectiveness of the classroom-based program Meaningful Roles, aiming to improve prosocial classroom climate through increasing children’s intrinsic prosocial motivation, stimulated by increasing social autonomy, social competence, and social relatedness. A multi-informant  and multi-method  approach will be used to assess primary outcomes  motivation, social autonomy, social competence, and social relatedness) and secondary outcomes , as well as moderators .The current study will provide information on the effectiveness of a classroom-based program promoting a prosocial classroom climate. It is of crucial importance that the school environment can provide a positive, prosocial classroom climate in which children feel safe and can achieve optimal social and academic competence and wellbeing.NCT05891067).ClinicalTrials ( A positive, prosocial classroom climate is crucial in promoting children’s school and psychological wellbeing, as well as in optimizing an effective learning environment , 2. PrevWhat is needed to create a prosocial classroom climate? According to the self-determination theory (SDT), behaviors are only encouraged when children experience intrinsic motivation for these behaviors. Moreover, such motivation (and behavior) is stimulated when children perceive that their environment fulfills their three fundamental needs: for autonomy, competence and relatedness  , 8. EnhaThe program is based on Meaningful Roles, an antibullying intervention program offering bullies prosocial alternatives for their behavior (by increasing autonomy and relatedness) . ResultsIn sum, there is both theoretical evidence and there are empirical suggestions that prosocial classroom climate can be improved through enhancing children’s motivation to behave prosocially by targeting SDT components. Moreover, empirical research on the effectiveness of school- or classroom-based programs in primary school demonstrated promising results for improving prosocial classroom climate, increasing prosocial behavior and wellbeing, and decreasing antisocial behavior. However, previous knowledge can be extended in several ways. First, about half of the studies performed a (cluster) RCT, which means that in non-randomized studies there might be confounding factors. Second, most meta-analytic reviews and studies included several age cohorts , not only primary school students, which precludes strong conclusions about this specific age group. Third, most studies were conducted in the United States, precluding conclusions about non-US samples. Hence, there is still not sufficient evidence for the effectiveness of classroom-based programs promoting prosocial behavior, specifically for programs based on STD components in primary school students. In addition, little is known about potential moderators of program effectiveness and the working elements of such programs.Moderators are characteristics that are likely to affect the effectiveness of intervention programs, such as child, teacher, school, and program characteristics. Child characteristics that may affect program effectiveness include demographic characteristics  and other factors . For instance, previous meta-analytic reviews showed that school-based interventions with older adolescents , a higheTeacher characteristics that may affect program effectiveness include demographic characteristics  and other factors . Previous meta-analytic studies examining the associations between student–teacher relationships and school adjustment showed that teachers’ gender and ethnicity moderated these associations , 28. ForProgram characteristics that may affect program effectiveness include program integrity and working elements. Program integrity, whether the intervention is implemented as originally planned, is important to report and to investigate as moderator of effectiveness, as it is necessary to draw conclusions about an intervention program. Negative or nonsignificant results might be due to incorrect program implementation, and not because of an ineffective program . In addiIn sum, the current protocol describes a study that will be conducted to examine the effectiveness of a classroom-based program promoting a prosocial classroom climate . Figure A cluster RCT will be conducted. Schools will be randomized (stratified for number of participating classrooms) into the intervention condition in which the program starts immediately or into a control condition that receives the program one school year later see Fig. , using aOnline questionnaires assessing demographics , primary outcomes  and secondary outcomes  will be completed by children prior to the start T1) of the program. Halfway through the program (T2) and after completion of the program in the intervention condition (T3), children complete questionnaires about program content  in addition to questionnaires assessing primary and secondary outcomes. Teachers complete questionnaires about demographics , attitudes, and civic skills at T1 and questionnaires about attitudes, civic skills, program content (to assess program integrity), and program evaluation at T2 and T3. Additionally, one teacher from each school will participate in the focus groups , to evaluate the implementation of the program . Table  of the pThe design of this study has been approved by the Ethical Committee of Utrecht University (number 23–0082) and is registered in the ClinicalTrials registry (NCT05891067).N = 42 schools to ensure there is enough power for the analyses (N = 21 schools for both the intervention and waitlist control condition). This sample size is sufficient to examine the effectiveness of Meaningful Roles and potential moderators, with a power of 0.80, an alpha of 0.05 and a medium effect size of 0.21 [We aim to include  of 0.21 .Participants are children from the upper grades of primary school (grades 5–8) in the Netherlands, their teachers (who will implement the program) and school leaders. The children will mostly be between 8 and 12 years old.Primary schools will be approached in person, via email and/or via telephone (mainly based on contacts of the intervention and research team). If schools show interest in participating in the Dutch adaptation of Meaningful Roles, they will receive (written) information about the project and what participation entails. Schools are also invited to an online orientation meeting. In this meeting the intervention and research team will clarify information about the program and research study, as well as answer questions from schools. If schools agree to participate, they will sign a cooperation form and will be randomized into the intervention or waitlist control condition. Parents/caregivers of children will be informed about the project  by schools and will be asked to give active informed consent for participation of their child in the study . Children aged 12 years or older will be asked for active informed consent themselves before they start the online questionnaire, in addition to their parents/caregivers. All children will be notified that participation is voluntary and that they can stop at any moment. Teachers and school leaders will be asked for active informed consent for their own participation in the study via a digital consent form, right before they start the online questionnaire. Children without permission/consent do not receive the research questionnaire, but they do receive the Meaningful Roles program.The Dutch adaptation of Meaningful Roles is a classroom-based program implemented by teachers. Teachers receive two three-hour training sessions from trained and experienced educational professionals. Meaningful Roles is an approach (throughout the schoolyear) for primary school groups 5 to 8 (children between 8–12 years old). The aim of Meaningful Roles is to improve the classroom climate and mutual relationships in the group, in a positive, prosocial way.The program consists of three main components: meaningful roles, classroom meetings, and compliments . Schools in the control condition are given the option to receive the materials and training to implement the intervention at a later stage, after the data collection is finished.Prosocial behavior will be measured by several instruments measuring different domains .not true) to 3 (definitely true). Higher scores reflect a higher injunctive norm of the group towards prosocial behavior.Prosocial behavior as an injunctive norm  will benever) to 4 . The SDQ-Dutch has sufficient psychometric properties in a children sample, with a Cronbach’s alpha of 0.78 for the total scale and 0.62 for the prosocial subscales [Individual prosocial behavior will be measured with the subscale prosocial behavior of the Strengths and Difficulties Questionnaire (SDQ), Dutch version , adaptedubscales . Higher Prosocial behavior according to children’s classmates will be measured with sociometric nominations. For each question, children see a list with the names of all classmates and select the classmates that fit the question best according to them . Two itenot true) to 4 (completely true). The cooperation subscale consists of four items assessing the extent to which children experience positive behavior . The conflict subscale consists of four items assessing the extent to which children experience negative behavior. For the current study, two items are used . The CPCQ has adequate psychometric properties in a children sample, with a Cronbach’s alpha of 0.79 and 0.83 for the cooperation and conflict subscale, respectively [Cooperation and conflict in the classroom will be measured with the Classroom Peer Context Questionnaire (CPCQ), subscales cooperation and conflict, respectively , 40, ratectively . Higher Intrinsic  motivation will be measured with four items specifically developed for this study , rated on a 3-point scale ranging from 1 (not true) to 3 (definitely true). Higher scores reflect more intrinsic  motivation.Social competence will be measured with items based on the Rosenberg Self-Esteem Scale (RSES), Dutch version [not true) to 3 (definitely true). The Dutch RSES showed high internal consistency in adults [ version , 42. Fivn adults  and suffn adults , 44. HigSocial autonomy will be measured with items based on the School as a Caring Community Profile-II ) and the Child Development Project [never) to 4 . Higher scores reflect more social autonomy. Project , 45. Fivocial relatedness will be measured with items of the Classroom Peer Context Questionnaire (CPCQ), subscales comfort and cohesion [not true) to 4 (completely true). The comfort subscale consists of four items assessing the extent to which children feel at ease in the classroom . The cohesion subscale consists of four items assessing the extent to which the children in the classroom spend time together. For the current study, two items are used . The CPCQ has adequate psychometric properties in a children sample, with a Cronbach’s alpha of 0.83 and 0.68 for the comfort and cohesion subscale, respectively [cohesion , 40, ratectively . Higher School wellbeing will be measured with eight items specifically developed for this study, assessing school belonging , rated on a 4-point scale ranging from 1 (never) to 4  and school safety , rated on a 5-point scale ranging from 1 (very unsafe) to 7 (very safe). Higher scores reflect positive school wellbeing.Social position will be measured with sociometric nominations. For each question, children see a list with the names of all classmates and select the classmates that fit the question best according to them [ to them . Questio to them .Bullying and victimization will be measured by assessing bullying and victimization with a selection of items of the Olweus Bully-Victim Questionnaire, Dutch version , rated on 5-point Likert scale ranging from 1  to 5 . One additional item asks where the child is bullied . Three items relate to the perceived behavior and attitudes of the teacher . Answer options will be recalculated so that higher scores reflect more bullying, victimization, and negative attitudes/behavior of the teacher. The OBVQ showed adequate psychometric properties in children samples, with a Cronbach’s alpha above 0.80 [n  and socove 0.80 –49. Regaove 0.80 . For eacove 0.80 .Civic skills will be assessed at all three timepoints, for teachers only. Specific questions were developed for this study, based on the questionnaire measuring citizenship . Four dTeacher attitudes will be assessed at all three timepoints, with specific questionnaires developed for this study. Teachers are asked what they find important in their job as a teacher to stimulate, for example: “that children help each other in the classroom”. Higher scores reflect a positive attitude of stimulating prosocial behavior and STD components.Child, teacher, and school characteristics will be collected at baseline. Child characteristics include age, gender, romantic attraction , grade, and ethnic identity. Teacher characteristics include age, gender, years of experience  and experience with implementing social safety programs, as well as attitudes of the teachers. School characteristics include location, number of participating grades and students in the classrooms, and years of experience with a social safety program .Working elements will be assessed halfway through the program (T2) and after completion of the program (T3), with specific questions developed for this study. Children and teachers fill in questionnaires related to the three main components of the program: 1) meaningful roles ”, “Which roles were the most popular?”, respectively), 2) classroom meetings , and 3) compliments . Higher scores reflect a more positive or greater adherence of the working elements.Program integrity will be assessed halfway through the program and after completion of the program, with specific questions developed for this study. Teachers and school leaders fill in (open and closed) questionnaires related to the implementation of the program, divided into the domains training and material , child questionnaires , time investment , and global impression . Higher scores reflect a more positive or greater adherence of the program.In addition to the questionnaires, focus groups for teachers will be organized at T1, T2, and T3 to evaluate the implementation of Meaningful Roles. One delegate teacher per school will be asked to participate in these focus groups. Delegates can talk about their experiences with the intervention program and research, and are encouraged to make suggestions about important questions that should be included in the data collection waves for teachers and school leaders (at T2 or T3), in order to understand how feasible the implementation has been and what should be improved about the program. Semi-structured interviews will be used for the focus groups. The purpose of these focus groups is thus threefold: 1) information can be used as an indicator of the process evaluation  to understand strengths and difficulties in the execution of the intervention and evaluation of the research. This information can also help to understand why stronger or weaker effects may be found in different schools ; 2) information can be used to give suggestions for improving the program in the future; and 3) information can be used to add any questions to the intermediate measurement (T2) and final measurement (T3) that, in the experience of the teachers, seem important to measure regarding the implementation (not the outcomes). These findings are used to clarify the findings in the context of the implementation process.Before the analyses, possible baseline (T1) differences in demographics and outcome variables (baseline assessment) between children in the intervention program and the control condition will be checked with ANOVA (continuous variables) and chi-square analyses . To investigate the effectiveness of the intervention program, regression analyses will be performed. The dependent variables are the outcome measures at intermediate (T2) or posttest (T3), condition is the predictor variable , and baseline (T1), potential differences between the experimental and control group at T1, or intermediate measures (when dependent variables are the outcome measures at T2) are the covariates. The effect of potential moderators  on the outcomes will be examined by adding interaction effects to the models.This study protocol presents the design of a cluster RCT to investigate the effectiveness of a classroom-based program, Meaningful Roles, to promote a more prosocial classroom climate in primary schools, through improving intrinsic motivation to behave prosocial, by enhancing SDT components . This is important, because a positive, prosocial classroom climate is associated with an optimal learning environment, as well as positive school quality and wellbeing in children , 2. PrevThere are several challenges in this study, such as the recruitment of a sufficient number of schools (and students) and the collaboration with multiple parties. First, it may be difficult to recruit sufficient schools with a diverse and representative sample of students and maintain the involvement of schools, teachers, students, and parents over the course of the study. Recruitment of schools may be difficult because we would preferably like schools to participate with all their upper classes (groups 5–8), which requires involvement and participation of multiple teachers . To recruit schools, we will mainly try to contact schools who already have connections or collaborations with the research or implementation team , making the first contact easier. These schools are located all throughout the Netherlands. As such, it is likely that diverse schools, with a diverse sample of students, will be represented in our study. Additionally, flyers will be distributed via mail and newsletters, and information meetings will be organized to give more information about the program and study participation.Second, since participation of students requires active informed consent from their parents, it is crucial that all parents of participating classes are reached and are provided with clear information about study participation. Participating schools make sure that parents receive information about the project, and teachers will be responsible for contacting parents . In order to make giving permission as easy as possible, parents can also give digital consent. Researchers and the contact person of the schools will be in constant contact to increase response rates. These precautions are expected to increase the likelihood of participation of students.Third, another challenge in this study is the participation and involvement of multiple parties, including the researchers, the implementation team, trainers, schools, teachers, parents, and students. To strengthen the collaboration between parties, we will sign multiple agreement forms . Moreover, we will organize regular meetings (between the research and implementation team) and give weekly updates about recruitment rates (to schools and teachers). Moreover, we will be available for schools for consultation at any time during the recruitment phase and implementation process. It is of crucial importance that we invest time in clear and timely communication with all parties in order to collaborate as effectively as possible. Furthermore, we try to work according to a standardized research protocol as much as possible, but will tailor it to the needs of the specific schools . This way, we aim to avoid miscommunications and will ensure a positive and fruitful collaboration.In sum, the current protocol describes a study that will examine the effectiveness of a classroom-based program, Meaningful Roles, in promoting a prosocial classroom climate in primary schools. It is of crucial importance that classrooms are a safe and positive environment for children to be in, to improve social competence and achievement, and prevent the development of mental health problems and dysfunction later in life.


The mean awareness score significantly improved from 80.83 ± 11.94 before the training to 85.45 ± 7.08 after the training . The mean competence score significantly increased from 67.31 ± 14.75 before the training to 79.38 ± 10.39 after the training . The tabletop exercise program effectively enhanced the nurses’ competencies in responding to emerging infectious diseases. The training program developed in this study may be utilized in addition to a simplified theoretical lecture to train nurses to improve their competency in responding to emerging infectious diseases.This study developed and evaluated a tabletop program for training nurses working in small- to medium-sized hospitals in areas with poor healthcare access for emerging a tabletop program for training emerging infectious disease responses in nurses working at small- to medium-sized hospitals in areas with poor healthcare access and evaluated infectious disease responses. A tabletop program for training responses to emerging infectious diseases was provided to 29 nurses at a general hospital with <300 beds in a region without a tertiary general hospital or healthcare institution with nationally designated inpatient wards for patients with infectious diseases. The 180 min program consisted of an online theoretical lecture (Phase 1), one-on-one drills (Phase 2), and a scenario-based tabletop exercise (Phase 3). To evaluate the program’s effect, pre- and post-knowledge, awareness, and competencies related to responses to emerging infectious diseases were assessed. The mean knowledge score significantly improved from 11.41 ± 2.33 before the training to 16.69 ± 2.48 after the training (z = −4.529,  The continuation of the coronavirus disease COVID-19) pandemic has adversely affected the field of healthcare worldwide 9 pandemi and commIn South Korea, patients with emerging infectious diseases are transferred to and treated at hospitals with nationally designated inpatient wards, where the establishment and operation of isolation rooms are funded by the government. Thus, the burden of care for patients with emerging infectious diseases has decreased in general healthcare institutions. However, in Korea, only 38 healthcare institutions operate nationally designated inpatient treatment beds, and there are regional disparities in their distribution. For example, Chungcheongnam-do Province, which is divided into 15 administrative regions (eight cities and seven counties), has only two hospitals that provide healthcare services . AccordiIn particular, nurses are at high risk of infection spread because, while providing nursing care, they are in close contact with patients with emerging infectious diseases that they have never encountered previously. Nurses constitute the highest proportion of healthcare workers  and playA tabletop exercise is a training approach aimed at improving nurses’ competencies in this regard . A tableTabletop exercises have been utilized in preparation for the occurrence of emerging infectious diseases ,16,17,18The current study aimed to develop a tabletop exercise program for training nurses at small- to medium-sized hospitals in areas with poor healthcare access to respond to emerging infectious diseases and evaluate the program’s effect. The specific objectives of the study were as follows:First, to develop a tabletop exercise program for nurses in small- to medium-sized hospitals in areas with poor access to healthcare in response to emerging infectious diseases.Second, to investigate the effect of the program on the nurses’ knowledge of emerging infectious diseases, awareness of measures for responding to emerging infectious diseases, and their competencies in responding to emerging infectious diseases.Hypothesis 1 (H1).Program participants’ knowledge of emerging infectious diseases will significantly increase after the implementation of the program.Hypothesis 2 (H2).Program participants’ awareness of how to respond to emerging infectious diseases will increase significantly after the implementation of the program.Hypothesis 3 (H3).Program participants’ competency scores in responding to emerging infectious diseases will significantly increase after program implementation.This was a pre-experimental study with a one-group pre-test–post-test design.Convenience sampling was conducted at a single general hospital with fewer than 300 beds  in a region without a tertiary general hospital or any healthcare institution with nationally designated inpatient wards for patients with infectious diseases. A flyer was posted at the hospital after obtaining permission to recruit study participants. Nurses who were informed of the study had an understanding of the study purpose and voluntarily consented to participate were selected as the study subjects.https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower, accessed on 1 July 2022). Under the assumptions of a significance level of 0.05, an effect size of 0.50, and a power of 0.80, the minimum sample size required for the paired t-tests was n = 28. Thirty nurses were recruited, considering the dropout rate. One nurse dropped out of the study; thus, data from 29 participants were analyzed.The sample size was estimated using G*power 3.1.2  in response to emerging infectious diseases, receiving training on emerging infectious diseases, level of on-the-job exposure to COVID-19, and on-the-job experience of contact with patients with emerging infectious diseases were included.To assess the knowledge regarding emerging infectious diseases, the instrument used by Choi  for nursThe instrument used by Choi  to evaluThe instrument, a revision of that proposed by Choi , was useData were collected between 14 November and 25 November 2022. The specific study procedure is as follows.In this study, a tabletop exercise program for emerging infectious disease response was developed to increase nurses’ competency for the accurate assessment of a situation and crisis response when encountering patients with emerging infectious diseases in a hospital ward . The program was designed to provide relevant education and skills training to help nurses understand emerging infectious diseases and respond appropriately when emerging infectious diseases are encountered in inpatients, as well as to teach them how to address problems with the help of a case study.To develop the program, focus group interviews were conducted online with three infectious disease nurses working in small- to medium-sized hospitals and three infectious disease nurses working in the office of infection control in tertiary general hospitals with a nationally designated inpatient ward. The interview lasted approximately 1 h. Based on the findings of the focus group, MERS, COVID-19, Ebola virus, and monkeypox were identified as emerging infectious diseases likely to be encountered in small- to medium-sized hospitals. Furthermore, how to put on and take off PPE used to stop viral transmission and protect medical staff and how to respond to the occurrence of emerging infectious diseases were identified as topics of lectures and drills. With respect to the training mode, a scenario-based small-group tabletop exercise was developed so that the training would be applicable in practice. To reflect the characteristics of the study institution, the content of the scenario-based tabletop program was revised with the help of a hospital infectious disease nurse.The program consisted of lectures, drills, and scenario-based tabletop exercises. The details are presented in The pre-test survey was performed 10 days before the start of the program. The pre-test survey collected data on general and emerging infectious disease-related characteristics and assessed knowledge, awareness, and competence regarding emerging infectious disease responses. The survey took approximately 15–20 min to complete. A post-test on knowledge, awareness, and competence was performed upon completion of the program. Anonymous IDs were assigned to the surveys, and participants were instructed to place the completed surveys in a survey collection box.The experiment proceeded in a sequence of lectures, followed by drills, and finally, by the scenario-based tabletop exercise .In Phase 1, online lectures were conducted 1 week before the drills. The lecture topics included the status of the occurrence, etiology, transmission routes, symptoms, diagnosis, treatment, and infection prevention and control of MERS, COVID-19, Ebola virus infection, and monkeypox. Each topic was presented using PowerPoint slides incorporating text, graphics, images, and a recording with the instructor’s voice. Approximately 25 min long videos for each topic were uploaded to a personal YouTube channel, allowing individual participants to access and listen to them.® PRO+8038 Respirator FIT tester , and test results were explained to participants so that they could select a more appropriate type of mask.In Phase 2, participants performed drills of putting on and taking off level-D PPE  and performed an N95 mask-fit test in a one-on-one setting. In the PPE drill, participants were instructed to wear and take off the PPE after a nurse from the infection control office who had 5 or more years of experience demonstrated how to put the PPE on and take them off. Participants were provided with immediate feedback and, if necessary, instructed to practice this again. In the N95 mask fit test, cup-shaped masks  and folded masks  were used. The mask-fit test was performed using a PORTACOUNTIn Phase 3, a scenario-based tabletop exercise was led by an infection control expert using the blueprint of the patient ward of the study institution and a tabletop exercise kit . The infection control expert worked in the infection control office of a healthcare institution with nationally designated inpatient wards, had more than 15 years of experience, and was a licensed infection control nurse. In the tabletop exercise, a scenario was presented in which a patient with an emerging infectious disease was present in a multi-bed ward. The following questions were discussed, and feedback was provided to encourage the study participants to participate actively in the exercise: 1. What is the very first thing you need to do? 2. To whom do you report the event? 3. How do you transport the patient? 4. Which PPE do you need? 5. What supplies do you require in the isolation room? 6. How extensively do you decide to trace contacts? 7. How do you manage the contacts? 8. How do you handle the medical devices used in the isolation room? 9. How do you handle the supplies used in the isolation room? 10. How do you handle the laundry used by the emerging infectious disease patient? 11. How do you treat the waste generated in the isolation room? and 12. How do you clean and disinfect the isolation room? The exercises were performed in an in-person setting. Three to four participants were assigned to each team so that decision-making would be possible based on lively interactions among team members. The tabletop exercise was conducted for a duration of 60 min per group by an infection control expert.This study was approved by the Institutional Review Board of Sun Moon University (IRB No. SM-202208-035-2). Prior to data collection, the participants were informed of the study’s purpose and procedure, anonymity, and confidentiality. They were also informed that participation in the study was voluntary and that they could withdraw from the study at any time without any consequences. The participants were given small gifts to show their appreciation for participating in the study. All documents pertaining to the study (including returned surveys) were stored in a locked place in the research’s office, and general and survey data were codified for the researchers’ information only. Finally, participants were informed that the data would be shredded and completely discarded at the end of the 3-year storage period after completion of the study.Data analysis was performed using SPSS , and general and emerging infectious disease-related characteristics were examined by computing frequencies, percentages, means, and standard deviations. Scores for knowledge, awareness, and competence regarding emerging infectious disease responses were examined by computing means and standard deviations. The internal consistency of the study instruments was assessed by computing Cronbach’s α values. Pre–post changes were analyzed using paired t-tests and Wilcoxon signed-rank tests.The general and emerging infectious disease-related characteristics of the participants are shown in n = 27) of the study sample were female. The proportion of staff nurses was 93.1% (n = 27). The mean length of total nursing experience was 73.38 ± 24.30 months, and nurses with less than 1 year of experience comprised 37.9% (n = 11) of the sample and constituted the highest proportion among the groups classified based on the length of nursing experience.The mean age of participants was 33.97 ± 9.45, and 93.1% (n = 22) received training on how to put on and take off PPE, and 72.4% (n = 21) received education on emerging infectious diseases. Regarding on-the-job exposure to COVID-19, 72.4% (n = 21) had direct exposure to patients with a confirmed COVID-19 diagnosis, which is the most common type of exposure to emerging infectious diseases. Regarding on-the-job exposure to patients with (or suspected of having) an emerging infectious disease, 96.6% (n = 28) had been exposed to COVID-19, and 3.4% (n = 1) had been exposed to patients with novel influenza.With respect to emerging infectious disease-related characteristics, 75.9% .The results supported Hypothesis 1: “Program participants’ knowledge scores regarding emerging infectious diseases will significantly increase from before to after the implementation of the program”. The mean pre-program score for knowledge was 11.41 ± 2.33, whereas the mean post-program score was 16.69 ± 2.48, indicating a significant increase .Hypothesis 2, “Program participants’ scores for awareness of how to respond to emerging infectious diseases will significantly increase from before to after implementation of the program”, was also supported. The mean pre-program score of awareness was 80.83 ± 11.94, whereas it significantly improved to 85.45 ± 7.08 after completion of the program .Similarly, Hypothesis 3, “Program participants’ scores for competence in responding to emerging infectious diseases will significantly increase from before to after implementation of the program”, was also supported. The mean pre-program score for competence was 67.31 ± 14.75, whereas the mean post-program score was 79.38 ± 10.39, which constituted a significant increase , wherein public health workers showed increased knowledge of SARS after completing the program . Park 2 mentioneThe mean awareness score significantly increased after the training program. This demonstrates that the tabletop exercise program for emerging infectious disease responses positively impacted the nurses’ awareness. Few intervention studies have examined the awareness of emerging infectious disease responses; thus, intervention studies should continue to be performed. Awareness of the seriousness of emerging infectious diseases may be useful for predicting the attitude and behavior of emergency room nurses in the event of emerging infectious disease encounters . Yi and The mean post-program competence score was also significantly higher than the pre-program score, confirming that the tabletop exercise program for emerging infectious disease response effectively improved nurses’ competencies. No previous studies have used the same program, participants, or research instruments, making direct comparisons difficult. However, our study findings were consistent with previous findings that public health workers’ skills pertaining to SARS were enhanced after they had received tabletop exercise-based training for responding to SARS  and thatAccording to a previous study , the perTo the best of our knowledge, this study is the first to evaluate the effect of a scenario-based tabletop exercise program targeting nurses at small- to medium-sized hospitals in areas with poor healthcare access amidst the global pandemic of COVID-19. The tabletop exercise program effectively improved nurses’ knowledge, awareness, and competence regarding emerging infectious diseases. We suggest that the education/training program developed in this study be utilized if training is planned to enhance nurses’ competencies in responding to emerging infectious diseases.A limitation of this study is that the tabletop exercise program for emerging infectious disease response was administered to a single group, thereby limiting the comprehensive evaluation of the program’s overall effectiveness. It was difficult to select study institutions because of the prolonged COVID-19 pandemic and South Korea’s continuing and thorough COVID-19 policies, as external infection control experts required access to healthcare facilities to conduct in-person training sessions with nurses. Consequently, the research team was unable to find approved medical institutions, resulting in the inability to establish a control group.Future research should include experimental or multi-institutional studies to compare and analyze the effectiveness of the tabletop exercise programs in experimental and control groups. Additionally, tabletop exercises targeting nurses should be expanded, and a training program should be developed to target various healthcare workers to improve overall emerging infectious disease responses.


Approximately one in ten children globally live with kinship caregivers—relatives and family friends who step in to care for a child when parents are unable to do so. When families take on the role of informal kinship care—care of a child outside of the child welfare system—they often do so without financial assistance and advice in navigating the systems of support available to them. This is the unique role of kinship navigator programs in the U.S: to provide kinship caregivers a single point of entry for connecting to needed resources such as financial, health, housing, and legal assistance.To the best of our knowledge, our team conducted one of the only participatory evaluations in which kinship caregivers were involved in all stages of evaluating a kinship navigator program—from designing the questions, to collecting and analyzing the data, to reporting the results. Black kinship caregivers took on decision-making power leading this formative evaluation of a kinship navigator program within one nonprofit organization in a Southeastern state.In this paper we reflect on our process and offer lessons learned from engaging in participatory evaluation that may apply to the field of kinship care and across social service delivery more broadly. We focus on (1) ensuring the nonprofit’s commitment to the study, (2) maintaining engagement through building relationships and facilitating a culture of learning within the study team, (3) sharing decision-making power so that people with lived experience have the authority and ownership to lead the evaluation, (4) developing team members’ skills, confidence, and sense of belonging, and (5) increasing the likelihood the nonprofit will act on the study findings.Through this process, we learned that participatory evaluation is a feasible and useful approach both to understanding the experiences of kinship families and to improving the supports in their lives. We hope this paper will inspire others to draw on the strengths and capacity of people with lived experience to engage in participatory evaluation. Greater recognition of the value of this approach in social change and increased funding to carry out the process are both needed. Involving people with lived experience in all stages of an evaluation can strengthen the credibility of the findings. This paper provides an example of this involvement. It focuses on a program designed to support kinship caregivers—those who step in to take care of a child when the parents are unable to do so. To the best of our knowledge, our team conducted one of the only participatory evaluations in which kinship caregivers were involved in all stages of evaluating a kinship navigator program in the U.S.—from designing the questions, to collecting and analyzing the data, to reporting the results. Through this process, we learned that participatory evaluation is a feasible and useful approach both to understanding the experiences of kinship families and to improving the supports in their lives. In this paper we share our collaborative journey through each stage of the evaluation, offering lessons learned about the process that may apply to the field of kinship care as well as to other areas of social service delivery. Participatory evaluation is rooted in the belief that involving people with lived experience in all stages of a program evaluation can promote a more inclusive approach to organizational learning and lead to more credible and useful findings , 2. ThisApproximately one in ten children globally live with kinship caregivers—relatives and family friends who step in to care for a child when parents are unable to do so . ChildreTo build effective programs that support kinship families, caregiver voice is essential . There aPublic involvement in research recognizes the importance of studies being carried out “with” or “by” the public rather than “about” or “for” them . There aThis participatory approach advances culturally responsive evaluation, deeply engaging historically marginalized groups in the evaluation process . The morindividuals directly impacted by a social issue or combination of issues who share similar experiences or backgrounds and can bring the insights of their experience to inform and enhance systems, research, policies, practices, and programs that aim to address the issue or issues .There is a need to involve people with lived experience in determining the supports that would make a difference in their lives. “People with lived experience” refers to:Diversity of our teams Cultural appropriateness and validity of our methodsAbility of our designs to reveal structural and systems-level drivers of inequityDegree to which those affected by what is being evaluated have the power to shape and own how evaluation happens.Aligning our practices with an equity approach—and even more powerfully, using evaluation as a tool for advancing equity. It means considering these four aspects, all at once:By involving people with lived experience, evaluations can then benefit from a deeper understanding of the conditions in which families live and the supports that are most needed . MoreoveChukwudozie et al.  conducteThis participatory evaluation was designed to gather data within a nonprofit organization in a Southeastern state in the U.S. serving kinship families, to strengthen the program model and service delivery. Formative assessments are critical in providing immediate and concrete suggestions to improve the program and ensure services are tailored to the unique needs of the population before engaging in a summative evaluation . FurtherTo offer important context, the following section briefly describes kinship care and the kinship navigator program that is the focus of this evaluation.Kinship care is the “full-time care, nurturing, and protection of a child by relatives, members of their tribe or clan, godparents, stepparents, or other adults who have a family relationship to a child” . We use Kinship care has historical roots in Black, Latinx, Asian, and Native American communities, keeping cultural identity and traditions alive, while protecting children from the trauma of entering the child welfare system –29. CompThis evaluation took place early in the COVID-19 pandemic when restrictions were high, with many kin caregivers experiencing increased economic hardship, parenting stress, and psychological distress , 34. COVIn response to caregivers’ needs for connecting to financial, health, housing, and legal systems, kinship navigator programs have emerged across the U.S. Today, approximately 70 kinship navigator programs exist to support kinship families across 26 states in the U.S. [HALOS started as a nonprofit in 1997 and launched the first kinship navigator program in South Carolina in 2007. Program staff assist kin caregivers with understanding, navigating, and accessing the system of out-of-home care supports and services over a 3-month period. Most kin caregivers are informal caregivers, meaning that they are not formally connected to the child welfare system. During COVID, HALOS remained open, though some in-person supports became virtual and other resources were temporarily suspended.During the focus of our study, March 2020 through February 2021, the program reached 146 kin caregivers and 479 kin children. The majority of kin caregivers were female (95%), grandparents (68%) and Black (59%). Of the 146, 30% reported having a disability and 77% had an annual income under $30,000, which is now the federal poverty level for a family of four .In the summer of 2020, HALOS Executive Director, Kim Clifton, and Kinship Navigation Director, Elizabeth McGuan, reached out to a community-based researcher on this team to initiate a formative evaluation to explore caregivers’ experience in the program and identify areas of program improvement. The community-based researcher was experienced in participatory evaluation, but new to the field of kinship care. We understood from the start that kin caregivers are experts who would be critical in designing the evaluation. In August 2020, HALOS invited a representative group of eight kin caregivers who had completed the 3-month kinship navigation program to join as advisors. The group included both new and experienced caregivers, diverse across gender, race, and age. We knew it was important to involve caregivers who were diverse not just in their personal characteristics, but also in the ways by which kin children came into their care, in their relationships to the kin children, and in their connections to the child welfare system. Cousins and Chouinard  find thaDuring the initial 2-month period, kin caregivers in our advisory group shared their own kinship stories and their experiences as participants in the program and at the same time learned about all aspects of the program model. We realized from these conversations that caregivers had strong qualitative research skills—strong listening skills, ease in building rapport, comfort in handling ambiguity, and patience in analyzing data. The next step was to move beyond an advisory role and ask kin caregivers of their interest in becoming core members of the evaluation team. After these first two months, we invited the five caregivers who remained in the group to join; ultimately, four signed on. Three of the caregivers were female, one was male, all were Black, and all were between the ages of 40 and 74. The four kin caregivers were helping to raise nine children ranging from 4 to 14 years of age.By October 2020, the evaluation team of seven was complete : Four kin caregivers , two staff members—the Kinship Navigation Director (EMcG) and Support Group Facilitator (DX)—and one outside evaluator (the first author) to guide the process . Two academic advisors also supported our team, one a scholar in participatory action research (APS) and the other a statistician and scholar in kinship care (YX).How are kin caregivers adjusting to life with a kin child? What supports and resources are most needed, especially during the pandemic?What are kin caregiver perceptions of the program? What are the program strengths and areas of improvement?What changes do kin caregivers experience during their time in the program, particularly in their well-being and ability to meet financial needs?Together, we determined the evaluation questions:A participatory evaluation approach was used throughout the evaluation process, bringing together both qualitative and quantitative data. Our evaluation relied on primary data collected through caregiver interviews at program completion. Interviews were supplemented by secondary data collected by program staff at intake and at program completion. Measures are described below and can be found in detail in the full evaluation report . The ColOur study team selected individual interviews as the primary methodology to learn more about caregiver experiences in the program. We decided that the kin caregivers on our team  would design the interview guide, conduct interviews, and interpret the data. To prepare for this new role, we met for nine hours to go through an interactive virtual training session led by the outside evaluator on our team. The training focused on the key concepts within the Collaborative Institutional Training Institute (CITI Program) Social and Behavioral Research Course . This inCaregivers on our team generated interview questions using a process developed by the Right Question Institute . Using tAll kin caregivers who started the kinship navigator program between June and September of 2020 were invited to take part in interviews after they finished the program. We first sent an introductory letter and consent form by mail and email. After a few days, we called to obtain informed consent by explaining the purpose of the evaluation, the voluntary nature of participation, and the risks and benefits of participation. We discussed the types of questions to expect during the interview and how their responses would be kept confidential. Caregivers received a $15 gift card for their participation. From the start we believed that caregivers were more likely to be at home and available for interviews during COVID. Of the 53 caregivers we contacted, 58% (N = 31) consented and were interviewed. Of the 53, 17% did not respond to our requests, 13% were not available during the scheduled interview time, 6% had non-working phone numbers, and 6% refused. We found that the population interviewed were similar to the population of HALOS caregivers across demographic characteristics. The Support Group Facilitator on our team (DX) oversaw all recruitment and scheduling.Between November 2020 and February 2021, the caregivers on our evaluation team conducted 31 interviews with kin caregivers once they finished the 3-month program. All interviews were semi-structured and conducted one-on-one, via phone. Most interviews were 15–25 min in length, with an average of 22 min. All were audio-recorded and transcribed by a professional transcription service. Caregivers on the team set the tone of each interview from the beginning by sharing a few personal details of their lives, immediately putting interviewees at ease and creating a safe, sharing environment. Once an interviewee discovered they had something in common, we heard a variety of positive reactions such as, “Wow, you’re a caregiver, too?” and (in response to the grandmother on our team raising five grandchildren), “Oh my, five children?!” The interview then benefited from being “a conversation between equals than as a distinctly hierarchical, question-and-answer exchange” .During interviews, we discovered that kin caregivers were open to talking with their peers—sharing their lives, their vulnerabilities, and their ongoing needs. The caregivers on the team all had first-hand experiences with the child welfare system and knew by asking non-scripted follow-up questions they would build trust and give interviewees the sense they were connecting with someone who understood their journey. This approach recognizes the long history of distrust among families with the child welfare system, particularly among communities of color , 41.After each interview, kin caregivers recorded their reflections as analytic memos . In someOver 8 months, we met for more than 50 h. Kin caregivers rarely missed a meeting, even attending from doctor’s offices, between work shifts, and while still recovering from COVID. This consistency helped us keep our momentum during analysis. Themes emerged inductively after careful review of every interview . A data To guide our analysis and strengthen the validity of our findings, we followed guidance from Maxwell . We lookTo supplement caregiver interviews, we examined data collected by program staff at intake and at the end of the 3-month program. The program measured change in the adequacy of family resources, from basic necessities such as housing and food, to funds for bills, to time for family, through the Family Resource Scale . In addiOur full evaluation team examined the survey results. Once our statistician completed analyses, she brought aggregate data, without names or identifiers, to three separate group meetings to ask kin caregivers on our team to reflect on the findings together. She asked kin caregivers for their interpretation of the data and how the data gathered through the program were consistent or different from the data gathered through interviews. When survey data showed that access to certain benefits was lower or higher than expected, kin caregivers on the team offered important context that might explain each finding, including information gathered from interviews and from their lived experience.Our team shared findings with HALOS program staff iteratively to avoid surprises and develop a better understanding of the types of challenges staff were facing. At the end of the evaluation, the team met with all HALOS staff to discuss the findings and brainstorm ideas to address the barriers to accessing needed services. Our team posed a series of questions for reflection, both in writing and in conversation, rather than a fixed set of recommendations. We did this realizing that we did not know all the dynamics involved in decision-making at the program level. The main themes and transcripts from our months of analysis meetings served as the text for our final evaluation report. Our evaluation team and advisors all co-authored the final report. In the end, a graphic designer, Oliver Moldow, joined several virtual calls to ask our team for advice in the report design. Figure process of conducting a participatory evaluation and offer lessons learned that may apply to the field of kinship care and across social service delivery more broadly.Through our evaluation, we learned about the varied ways the kinship navigator program made a difference in the lives of kinship families. We also uncovered vital new information to help the program improve. Full findings can be found in our evaluation report . In thisOverall, we learned that participatory evaluation is a feasible and useful approach both to understanding the experiences of kinship families and to improving the supports in their lives. Three aspects of our process are rare in participatory evaluation. First, Black kin caregivers, including Black grandparents, led all stages of a formative evaluation in a field in which they are the experts—an underutilized strategy in kinship care and in the wider field of child welfare . Second,In this section we share key lessons learned that fostered these successes. As there is no single method of conducting a participatory evaluation, we offer lessons from our experience. We audio-recorded and transcribed our meetings to reflect on our journey as a team, consistent with the longstanding practice of reflexivity in qualitative research  and consBelief in the importance of a participatory approach started at the top of the organization, with the HALOS Executive Director. There was an organizational commitment to the idea that those with lived experience have a critical role in producing knowledge for program improvement. Participatory evaluation as a method often requires a greater investment of time, program resources and organizational commitment . In our HALOS prioritized the evaluation process, dedicating the necessary funds and appointing program staff to see the process through. From the outset, the Kinship Navigation Director invested time responding to any questions kin caregivers had about the program model and delivery. She voiced from the beginning and throughout the process that honest critiques are essential in improving the program. The Support Group Facilitator had years of experience running kinship support groups and was able to oversee the interviewee recruitment and consent process.Participatory studies have lost local researchers due to lack of compensation . No costCousins and Chouinard  reveal tMeetings and interviews were held around caregiver schedules, changing week to week to accommodate family sickness, work and family commitments, and life changes. We established a general timeline from the beginning but worked at a pace that the team could accommodate. Several strategies fostered kin caregivers’ ongoing commitment: Time for relationship building as a study team The caregivers on our team bonded over their shared journey as well as their shared strength as deeply religious individuals. Historically, Black families turn to the church in distress for strength [strength . In the  Culture of learning as a study team For kin caregivers, the process of conducting an evaluation was novel. From the start, we built a culture of learning, knowing that we would make mistakes and learn along the way. Cousins and Earl [and Earl  find thaand Earl . Time for reflection as a study team One of the challenges for caregivers on our team was in listening to caregiver stories that often mirrored their own life experiences. After a difficult first interview, one member of our team reflected: “You already went through some of the same things they are going through. That was hard for me. Afterwards, oh, my goodness, the burden was still on me. I said, ‘Oh, it was so hard. I have to talk to someone.’” After each interview, kin caregivers reached out to another team member, often the outside evaluator, to process their experience.As a study team, we also made the decision early on to listen to each interview and read the interview text together at the same time, knowing we all learn differently. However, the act of listening to the interviews together became important in yet another way: it allowed all of us to share in the experience of the interviewer, to better understand how the interview felt, and to be a source of support to the interviewer. In one example, a caregiver praised another member of the team after listening to her emotional interview: “I feel a weight from this one. I don't know if you felt it when you did it, but you did really good. I'm really proud of you. It's just like a conversation between two friends, a friend you haven't talked to in a while, you're checking on her, and I love that.” Maintaining time and space for reflection enabled team members to examine their own experiences in a group setting to engage in deeper analysis of the data .Bringing together program recipients, staff, and an outside evaluator requires attention to inherent power imbalances , 52. Acc(1) Authority to make decisions Consistent with a central element of participatory work, our team had the “authority” to make decisions [Stakeholders perceive a process as authentic when their input is valued and they believe that the collaborative process itself, rather than some external agency, has the authority to generate plans and have those plans guide decision making (p. 470).ecisions . We valuecisions . Hicks eecisions  explain:After the initial 2-month period, when we asked caregivers if they were interested in becoming core members of the evaluation team, we heard comments such as: “It’s an honor to feel that my input matters and makes a difference,” and “It’s great to be heard and included.”(2) Ownership at each stage of the evaluation Following advice from Cousins and Earl [and Earl , we seleThrough each stage of the evaluation, we could see changes in all of us. Participatory evaluations are linked to positive changes among stakeholders, including increased learning/capacity, awareness of issues, and confidence . In partShe was an angel sent just for me. … She was so excited that HALOS sent someone to call her that would understand, and she could talk to… I felt so good after I left the call because I helped her in a little way by just listening to her. … This is easy. I was so nervous from the beginning. … I thank you all for the confidence for building me up.Each meeting and interaction presented an opportunity to build the team’s skills, confidence, and sense of belonging to the group. By listening to each interview as a group, we could all acknowledge the strengths in the way kin caregivers on the team asked questions and responded during often emotional interviews. One caregiver reflected on her growing confidence after a first pilot interview:Another caregiver declared at the end of an interview: “I leave this interview, I feel changed.” At the end of the evaluation, one caregiver reflected, “I feel like you’ve given us wings.”Since our evaluation, our team has remained active in HALOS. One kin caregiver on our team serves as a mentor to new kin caregivers. Another caregiver has been involved in training new staff to work effectively with kin caregivers and was invited to speak to the Governor of South Carolina about the needs of kinship families.Ultimately, the purpose of the evaluation was to provide valid, culturally-responsive data to improve the services HALOS provides to kinship families. By involving stakeholders—both staff and program participants—in the evaluation process, who have a deep understanding of local context and culture, there is an expectation the findings will be more relevant, and thus, more useful , 15. EveMany aspects of our process were intended to enhance HALOS’s appreciation for a participatory approach to learning. HALOS staff and kin caregivers participated in the evaluation training and planning together, so they were invested in the process and the outcome from the start. Throughout the analysis phase, we shared interview findings with the full HALOS team iteratively to ensure the information we were collecting would be valuable to them and to avoid any surprises. Finally, when we shared the final evaluation report with the full HALOS team, we shared questions for consideration rather than specific program recommendations. That way, we could meet with program staff to talk through potential program changes together and position ourselves as colleagues all working toward the same goal of improving services for kinship families.Together, these steps enhanced HALOS’s orientation to act on the evaluation findings. Following our evaluation, HALOS instituted several program changes suggested through the evaluation process. For example, HALOS created an introductory packet to ensure caregivers understand all program offerings to which they would have access and developed a peer mentoring program to ensure new kin caregivers have the benefit of connecting with more experienced caregivers. Also, HALOS now includes our full evaluation report in staff training materials.After completing the evaluation, HALOS began investing more effort in gathering data from caregivers to inform programming. Specifically, HALOS began taking greater steps to boost the caregiver response rate to the data collection that occurs at the end of the 3-month program by adding a $10 incentive and increasing outreach. In the 6 months leading up to this evaluation the response rate to the 3-month follow-up was 32%; In the 6 months following the evaluation, the response rate rose to 73%. Strategies to increase the response rate are particularly important to the kinship care field, as studies have discussed the difficulty of reaching informal kin caregivers and understanding their needs .As a formative evaluation effort there are some limitations to note. We recognize the limits of understanding the lives of kinship caregivers through a single, retrospective interview and a pre-post needs assessment of a single group. In an observational study, it is not possible to isolate the impact of the program. Rather, this formative evaluation effort was designed to strengthen the HALOS program model and improve service delivery. Of note, of the 146 kin caregivers who entered the program, 46 completed both the pre- and post-Family Resource Scale. Further, our team interviewed just 31 of the 146 kin caregivers who entered the program in the study year. While interviewees and survey respondents had similar demographic characteristics to the full population of HALOS caregivers, their lives and experiences may be different in important ways.One of the strengths of this evaluation was that data collection and analysis was led by kin caregivers who intimately understood kinship care. We recognize, too, that this approach had potential limitations. It may be that some interviewees were hesitant to share experiences that would put them in a negative light with their peers due to a social desirability bias. Interviewees may have been fearful of sharing negative feedback with our evaluation team, worried it may limit future program benefits. To help allay this fear, both during our consent process and at the beginning of each interview, we assured interviewees that their responses would be kept confidential and that we would share feedback only in summary form, without caregiver names.We recognize, too, that there are both risks and benefits of including program staff on the evaluation team. We understood that kin caregivers may be less critical in their analysis of the program and in their program recommendations in an effort to avoid offending program staff. While it may be that excluding program staff would eliminate these concerns, we believed that the potential benefits would outweigh the potential risks. By having program staff involved, we had a better understanding of the nuances of the program model, could communicate our progress and emerging findings easily throughout the study, and were motivated to continue seeing how the findings would be of value to the program. To lessen the potential risks, the Kinship Director stepped away from all meetings involving analysis of interview data to ensure kin caregivers could listen and reflect on interviews freely. Program staff repeatedly conveyed to kin caregivers on the evaluation team that their critiques were necessary to further improve the program.This article demonstrates the value of involving people with lived experience at every stage of a program evaluation. Ensuring that a team of Black kinship caregivers had decision-making power at every stage of the evaluation was an important move towards racially equitable evaluation methods. Through this process, we learned that participatory evaluation is a feasible and useful approach both to understanding the experiences of kinship families and to improving the supports in their lives.We hope this paper will inspire others to draw on the strengths and capacity of people with lived experience to engage in participatory evaluation. Greater recognition of the value of this approach in social change and increased funding to carry out the process are both needed , 46. The