Community-academic partnerships (CAPs) can improve the relevance, sustainability, and uptake of new innovations within the community. However, little is known about what topics CAPs focus on and how their discussions and decisions impact implementation at ground level. The objectives of this study were to better understand the activities and learnings from implementation of a complex health intervention by a CAP at the planner/decision-maker level, and how that compared to experiences implementing the program at local sites.The intervention, Health TAPESTRY, was implemented by a nine-partner CAP including academic, charitable organizations, and primary care practices. Meeting minutes were analyzed using qualitative description, latent content analysis, and a member check with key implementors. An open-answer survey about the best and worst elements of the program was completed by clients and health care providers and analyzed using thematic analysis.In total, 128 meeting minutes were analyzed, 278 providers and clients completed the survey, and six people participated in the member check. Prominent topics of discussion categories from the meeting minutes were: primary care sites, volunteer coordination, volunteer experience, internal and external connections, and sustainability and scalability. Clients liked that they learned new things and gained awareness of community programs, but did not like the volunteer visit length. Clinicians liked the regular interprofessional team meetings but found the program time-consuming.An important learning was about who had “voice” at the planner/decision-maker level: many of the topics discussed in meeting minutes were not identified as issues or lasting impacts by clients or providers; this may be due to differing roles and needs, but may also identify a gap. Overall, we identified three phases that could serve as a guide for other CAPs: Phase (1) recruitment, financial support, and data ownership; Phase (2) considerations for modifications and adaptations; Phase (3) active input and reflection. Collaborative relationships between community organizations and academic institutions, often known as community-academic partnerships (CAPs), are an effective way to improve the relevance and sustainability of research studies and the uptake of evidence into practice . Having Health TAPESTRY is a community program centered in primary care which helps older adults stay healthier for longer where they live; the Health TAPESTRY model has four key elements: volunteers, interprofessional health care teams, technology, and community engagement and connections , 11. In To implement Health TAPESTRY, a CAP was created between an academic institution, two non-profit community organizations, and six primary care clinics. Collaborating between nine partners while implementing a multi-site RCT was a large undertaking. While we used the RE-AIM framework  combinedWe used a qualitative descriptive approach  to descrThe CAP in this study had four different types of partners: an academic family medicine department of a university (McMaster University Department of Family Medicine) a national humanitarian charitable organization (Canadian Red Cross) a local community coalition (Windsor-Essex Compassion Care Community) and six primary care practices  across Ontario, Canada . The natMultiple formal collaborative membership agreements were created for the CAP. The agreements were either three-party or four-party agreements made between the academic partner, the volunteer organization, and a primary care site . The agreements outlined: benefits to the primary care practice; commitment and service requirements of each partner; financial arrangements; confidentiality; data management and sharing; and branding guidelines. The three-year study was funded through a provincial government grant and a private donor.There were two groups that actively met to provide oversight for the program and research study. A general governance group met a few times a year and was responsible for the overarching governance of Health TAPESTRY. This group maintained the integrity of the four main elements of the program and explored the potential scale-up of Health TAPESTRY. It consisted of individuals from the academic partner , and key leaders from the national charitable organization. The second group met monthly and was responsible for determining regular needs in the operational implementation of the program . Group membership was very similar to the governance group with the governance group being slightly larger including some additional members from the academic partner. Additionally, there were quarterly telephone meetings which we termed ‘community of practice’ meetings, where all partners were invited to attend, including select volunteers and health care providers from each site. The meetings were opportunities for the sites to connect with each other, and discuss progress and successes, as well as tod problem solve any barriers partners were experiencing. Lastly, there were bi-weekly meetings between the academic partner and the Volunteer Coordinators from the two volunteer organizations where all partners provided updates, solved any issues that arose, and helped the communities maintain fidelity to the program.Data included meeting minutes and a survey of clinician and client stakeholders. A specific framework was not used to guide the analysis; however, our intention was to try and understand the sensemaking work (or coherence) that the CAP partners, implementors, and clients engaged in when implementing Health TAPESTRY . After wWe conducted an a priori analysis of meeting minutes using qualitative descriptive analysis and latent content analysis (as described in “Study Design and Setting”) to make sense of the implementation of Health TAPESTRY. Meeting minutes were collected from all meeting groups listed above as well as meetings between the academic partner and primary care teams or volunteer organizations. All meeting attendees provided informed written consent to the meeting minutes being analyzed.intended to say [We used Bengtsson 2016)’s four stages of qualitative content analysis to analyze the meeting minutes . We bega16’s fourd to say .To further make sense of the meeting minute data, after initial analysis we completed a member check with selected participants who had been active members of the meetings and in the implementation of Health TAPESTRY. Member checking is commonly used in qualitative research to validate and assess the credibility of the data –24. The Clinical team members and clients were asked about the five best and five worst elements of Health TAPESTRY in an open-ended survey. These were not ranked responses. Clinical team members were asked this when they participated in focus groups, and almost all responded on paper, though some sent virtual responses outside of the focus group time. Clients were asked this in their six-month volunteer visit, and volunteers entered their responses into the TAP-App. The volunteers were not asked to complete this survey.The best/worst survey data were qualitatively analyzed using Braun and Clarke’s six steps of thematic analysis  but at aWe analyzed 128 documents for the meeting minute analysis. We invited six people from three stakeholder groups to complete the member check and all six agreed .The survey was completed by 49 health care team members and 228 clients. See Table During meetings, we found that the CAP partners focused on the following categories: primary care sites; volunteer coordination; volunteer experiences; internal and external connections; and sustainability and scalability. These key categories mirror the four key elements of Health TAPESTRY , other than the technology element, which was not a prominent topic of discussion in the meetings included in the dataset. The sub-topics discussed in each category included a focus on barriers or challenges, as well as general updates on study progress, feedback, and positive impacts of Health TAPESTRY. Further detail for each category is below. In Table When discussing the recruitment and onboarding of potential new Health TAPESTRY sites , conversations included reviewing the questions and concerns potential sites had about the program, and how to respond. The questions spread over several topics including recruitment, financial support and ownership of data. Further, the CAP identified facilitators to the site recruitment sessions, including the importance of having face-to-face conversations with the primary care practices. These topics  did not prominently appear in later data after this initial clarification, suggesting CAPs’ functions go through phases related to the time course of project implementation. This phase was also largely carried out by the academic and volunteer organization partners of the CAP, rather than the primary care teams which entered into the partnership later .Once the onboarding phase was complete, the meetings discussed adaptations of Health TAPESTRY in each of the six communities , with the individuality of the distinct sites influencing the conversation needs of the CAP planners/decision-makers. Through the member checks we found that meeting minutes alone were not able to describe the individuality of each of the six primary care sites, though it was clear that differences between sites was a major area of discussion. Our CAP’s function through this next period moved to explicit consideration and problem solving for modifications and adaptations for different contexts and sites as outlined in detail below.Barriers to the implementation of Health TAPESTRY relating to the primary care context included the difficulty of engaging health care providers (particularly family physicians) in the program, the frustrating process of client recruitment, logistical issues with the interprofessional primary care team that meets to review client information (the “huddle”), and providers not getting all the information they would like . It is important to note that these barriers were not necessarily present in every site. Some of the specific client recruitment challenges discussed included clients having privacy concerns, volunteer coordinators being unable to reach clients who had been referred to the program, a lack of physician buy-in to refer clients, confusing recruitment materials, and clients not feeling that they needed the program.The planner/decision-maker partners represented in the meeting minutes suggested some strategies which impacted implementation at the local sites. To facilitate recruitment they suggested: patient-facing materials to accompany the invite letter, physicians recommending clients, and large mailout efforts by the primary care practices. To improve implementation, they suggested: better huddle coordination , and having engaged and active leadership for Health TAPESTRY at each clinical site (including a practice model champion and strong huddle lead).A key area of discussion related to volunteer coordination was volunteer recruitment. The volunteer organizations did a variety of advertisement campaigns to recruit volunteers for Health TAPESTRY . Different strategies were considered successful in different communities, and these conversations held at the decision-maker/planner level had to be altered for each context by the local volunteer coordinators. Also pertaining to volunteer coordination on the ground in local sites, there were discussions around strategies to help facilitate the volunteer program such as assistance with booking client appointments during peak recruitment.Barriers identified through our meeting minute analysis pertaining to the implementation of the volunteer program within Health TAPESTRY included difficulties recruiting volunteers, limited volunteer availability, and the long period of time between volunteers receiving training and being able to have their first meeting with clients. Through member checking, difficulties scheduling client visits due to competing schedules was identified as an additional barrier not captured through the meeting minute analysis. These barriers were identified both directly by volunteer coordinators situated in local implementation sites, and the organizations they were a part of at the decision-maker/planner level.From a partnership-specific standpoint, there were also discussions relating to navigating collaboration and partnerships between the academic partner and volunteer organizations. These discussions included: clarifying how best to collect data for research purposes; clarifying which partner the data in the study “belongs” to  and whether the volunteer organizations could use it for their purposes ; branding and communications, including the decision to co-brand all products with both the academic partner and volunteer organizations’ logos; and preparation of an agreement to formalize the partnership between the legal arms of the academic partner and the volunteer organizations.Based on perspectives by volunteer coordinators and the volunteer organization partners, volunteers faced challenges during their participation with Health TAPESTRY such as gaps in knowledge , issues with surveys, technology difficulties, and transportation issues when attending client visits. There were also discussions about reasons why volunteers withdrew from the program and potential strategies that could facilitate the volunteer program.During the program there were issues regarding the connection, or lack thereof, between the clinical teams and volunteers. This information filtered up from the partners implementing the program at local sites and was dealt with quite extensively at all levels of meetings. Clinical teams were not utilizing the volunteers to their full capacity and volunteers wanted to be more fully engaged in follow-up appointments with clients and feel more connected to the clinic. Prior to the program though, the clinical teams had been enthusiastic about the volunteers and their potential.Other aspects of this internal and external connections theme included discussion about additional surveys or related projects  that were explored, but ultimately were not implemented within Health TAPESTRY. These additional surveys and projects had varied sources from local project implementers considering adding a theme to the surveys, to investigators from the academic team considering adding an element to evaluate; ultimately, wherever their source, these additional ideas tended to be considered and abandoned due to feasibility, interest by other implementers, or appropriateness for connecting to this program or its client population.A noticeable later function of our CAP was a focus on active input and reflection. This is seen in the discussion and its focus on sustainability and scalability. There were discussions about the possibility and feasibility of Health TAPESTRY being implemented in each site as a program with no research element present; in some sites these started from the implementer level, as they wanted to be able to continue the program. In other sites, however, this was a top-down discussion only. Specific topics of discussion around sustainability and scalability at the decision-maker/planner level included the interest of, and capacity at, each site to continue, possible client recruitment strategies (including improved messaging for clients to understand the benefits of the program), and the necessity of multiple partners to align. Knowledge translation strategies during the research study included community of practice meetings and events, videos, and sharing patient stories.There was general positive feedback from the clinical sites about the value of Health TAPESTRY and the ease of implementation. Health care providers felt that the program had value, was beneficial for clients and their team, and was easy to implement. They also felt that engaging volunteers to do home visits was appreciated, and they learned new things about the clients as well as about community resources and services.The survey makes visible some key areas of significance and concern to the clinic site members of the CAP, as well as the clients of the program. Table Clients and interprofessional team members both found strengths in the program in the aspects they specifically interacted with . They both also spoke of connection, collaboration, and engagement with other people and other services as a positive. There were also mirrors in what they found were the “worst” parts in terms of time involved in the intervention and confusion over the elements they were involved in. They also both spoke of efficacy in these top five worst areas, but clients who mentioned this just said the program did not work for them, while providers asid that clients did not follow through We noted that when these primary care teams and clients “on the ground” during the implementation of Health TAPESTRY were asked the best and worst aspects of the program, most of the ideas raised about the best aspects of the program and many of the issues were not discussed at the planner/decision-maker level. There were differences between what partners at the planner/decision-maker level focused on when implementing an intervention, which were largely around ensuring the implementation happened, and what elements or issues that the people implementing the program day to day and the clients receiving the program noted. For example, most of the ideas raised around the best aspects of the program were not discussed at the CAP level.The nine partners from the planner/decision-maker level of the CAP discussed a breadth of topics in meetings throughout the preparation for and implementation of Health TAPESTRY. Given the complexity of the intervention and number of partners, these findings are unsurprising. Downstream from the overarching CAP, in the survey, clients and health care team members also identified several areas of significant positive impact, as well as areas of concern that relate to a wide variety of program areas. It is useful to note that some of these overlapped with discussions by the planner/decision-maker CAP partners, however some did not. Through our analysis, it was evident that the CAP in this study did experience some of the challenges that have been reported by other scholars on research on CAPs. These included unclear roles, time commitment, and unequal power dynamics; , 2, 6 hoAn important area of learning is around the areas and stakeholders that had “voice” at the CAP level. When primary care teams and clients – the individuals who were “on the ground” during the implementation of Health TAPESTRY – were asked the best aspects of the program, most of the ideas raised were not discussed at the CAP level. For example, clients identified socializing with the volunteers in their homes to be a highlight of the program, which was not discussed by CAP partners. Clients also noted how helpful and respectful the individuals involved in delivering the program were. It could be surmised that the CAP partners did not discuss these topics for two reasons. First, meetings at the partner level were focused on the program’s high-level protocol, issues that arose, or major problems to be solved whereas both the home visits and the recruitment of strong volunteer teams were already accepted hallmarks of the program from the first implementation. Second, the CAP partners did not heavily, or at all, discuss more minor issues, problems that arose for the on-the-ground implementors, or things unrelated to the protocol. For example, individuals being respectful and helpful was more a product of the people involved and less about the program’s protocol, which would have made it difficult to discuss at the CAP partner level. This may also explain the fact that of the four key elements of the Health TAPESTRY model , technology was not heavily discussed at the planning/decision-maker level of the CAP.adaptability, a key domain seen in our data) and Characteristics of Individuals. At one level this makes sense as individuals from the planner/decision-maker level of the CAP were the source of the intervention, so would not need to discuss these details of the intervention that had been established in previous research; characteristics of individuals is an attribute that would be more relevant with the people implementing the project at local sites. The lack of representation of some of the clinician and client perspectives we identified in the survey in the higher-level CAP discussions indicate a value for considering a process for these “voices” to feed in to the CAP discussions during implementation. For example, while the survey results of the best and worst elements of Health TAPESTRY may have indicated that clients and providers involved in local site implementation may have been considering how the program became “normalized” into normal practice, such as is seen in Normalization Process Theory [When we compared the topics discussed in the meeting minutes with an established framework for implementation, CFIR , we founs Theory , the meeWe identified three phases in the process of this CAP that could serve as a guide for explicit planning for CAP process and agenda setting. In Phase 1, recruitment, financial support, and ownership of data were issues that required discussion and clarification. It might also be useful to identify at this phase what each organization hopes to gain from the project, as this was an area that the surveys suggested was unclear at times. Phase 2 is where explicit consideration and problem solving for modifications and adaptations for different contexts and sites should occur. Phase 3 is about active input and reflection on process experiences of planning/decision-making partners and integration of experiences of those involved in direct implementation at local sites . We also noted that three of the four key elements of the Health TAPESTRY intervention were mirrored in the key category areas in our analysis. This suggests that key intervention elements could be included in CAP processes explicitly, particularly in Phases 2 and 3 but also in the preparation phase.Based on the implementation of Health TAPESTRY using a CAP, we have three key learnings and propose suggestions for future work. First, we did not actively engage clients (a stakeholder group) in the design or implementation of Health TAPESTRY. Instead implementors relied on qualitative feedback (from the previous and current implementation) to address client concerns. Considering the number of barriers clients communicated, the discussions that were directly related to the clients, and that stakeholders can engage in knowledge exchange from each other, among other advantages , 3–5, weSecond, due to the objectives  and design of the study , the acquisition of funding, and the performance requirement not all partners had equal influence regarding study decisions. The academic partner in the CAP did retain a large amount of control in certain aspects of the study , despite good intentions. Other authors have also reported unequal power dynamics , 26. TheLastly, we encourage future CAPs to assess discussions and feedback from multiple levels of program implementation to determine where discrepancies and agreement exist on different elements of a program. For example, in this project, community involvement and engagement are important aspects of Health TAPESTRY as they were a topic of discussion by the CAP as an area for improvement, as well as identified by both clients and health care providers as a positive element of the study. Therefore, a further exploration and development of important program elements such as community involvement should be considered prior to implementation.This study had strengths and weaknesses. A prominent strength of this study was the large and varied sample as well as using the member check process with key implementation stakeholders, which allowed us to validate the implementation story and gain final reflections on the program’s progress. Second, the structure and organization of this large CAP seemed to contribute to a streamlined decision-making process that avoided unresolved issues and other common barriers such as financial pressures and differing timelines even with many CAP members. A limitation of the study was that the meeting minute analysis and member check were not planned a priori, which meant that some meeting minutes lack detail that would have benefitted this analysis. A second limitation, due to the nature of the study and number of partners, is that many decisions were made by email or through informal conversations, which were not captured. Therefore, our analysis likely missed topics and decisions. Lastly, the member check discussion was not anonymous and was facilitated by the academic partner. Therefore, participant responses may have been biased as to not offend other partners in the CAP.Using the CFIR was useful in identifying general areas for guiding optimization of CAP process. The specific focus in this study on the experience of implementing a CAP provided insights and reinforced the need for learning not just from the outcomes of a program but from different aspects of implementation. The results of this study provided interesting insights into the differences between what CAP partners focus on when implementing an intervention and what elements or issues that the people implementing the program day to day and the clients receiving the program recall. Even though the CAP was essential to the implementation of Health TAPESTRY, some of the lasting impacts of the program identified by clients and health care providers were not discussed at the partner level, potentially because they were already accepted elements of the program or more on-the-ground protocol issues that did not require CAP level discussion. Our findings identified several distinct phases as well as broad context areas in the process of CAP implementation and these could be used as a guide to optimize CAP process. Our findings also highlight the importance of gaining feedback and insights from implementors at multiple levels, and a process for integrating these voices, as there will likely be differing opinions on the elements of implementation as well as the significant value of equitable partnerships within a CAP to facilitate implementation.


Oral malignant melanoma (OMM) has a prevalence less than 1% of all melanomas and it commonly develops on the oral mucosa following a slow and unspecific transformation of unstable melanocytic lesions, often resulting in a diagnostic delay. The marker PReferentially Expressed Antigen in MElanoma (PRAME) seems to be a valid tool to investigate the biological and histological nature of cutaneous melanocytic lesions, but to date its use to characterize pigmented lesions in the oral cavity is largely unexplored. The aim of this study was to create preliminary knowledge on the PRAME expression in OMM, and to compare its expression respect to other dysplastic pigmented lesions of the oral cavity. Interestingly, PRAME has been demonstrated to be reliable in the clinical conditions investigated in our pilot study; in fact, it has clearly differentiated the cases of Melanoma, which showed diffuse and intense positivity (score 6+/7+) to PRAME, from the other melanocytic nevi, which resulted to be mainly negative to PRAME. This means a better differential diagnosis, a reliable early diagnosis and a proper clinical/surgical management of the oncological lesions. In conclusion, PRAME can be a valid qualitative marker for differential diagnosis, not only in cutaneous melanomas, but also in malignant melanoma of the entire head and neck area. Oral Malignant Melanoma (OMM) is a rare variant of mucosal melanoma Although progresses have been made in understanding the etiology of mucosal melanoma, the pathogenesis is not yet fully understood; on the other hand, it is clear that the molecular signature of OMM is different from melanomas originating from surfaces not exposed to ultraviolet (UV) In recent years, a growing interest has involved the marker called PReferentially expressed Antigen in MElanoma (PRAME): it is a member of the cancer testis antigen (CTA) family, which has been studied in the skin melanocytic lesions Despite its strategic role in several pathogenesis, and although it has been studied in the onset of different neoplasms, there is scarce knowledge about the use of PRAME in the differential diagnosis between OMM and other oral lesions, including the mucosal nevi of the oral cavity.In this paper, we highlight the most impacting preliminary results of a pilot study carried out on 9 cases of OMM, analyzed with immunostaining for PRAME, and compared with a control group (9 control patients) with benignant nevi developed on the oral mucosa. The main and most important outcome of our comparative study is the preliminary validation of PRAME in the differential diagnosis of similar lesions detected on the intraoral mucosa.Finally, we critically discuss our results, highlighting the potential clinical perspectives deriving from the early detection of PRAME in OMM, also comparing this approach to the current state of the art.The cases reported in this pivotal study have been obtained from the archives of the Laboratory of Pathological Anatomy (University of Bari "Aldo Moro"). The authors have used the following searching strategy: the patients were preliminarily clustered by searching the keywords "Melanoma" OR "Malignant Melanoma" AND "Oral". Then, the searching was further narrowed to find “second level” keywords, namely “Nevi” OR “Mucosal Nevi” AND “Oral”. Inclusion criteria were the followings: i. the primary onset of melanoma on the oral mucosa; ii. the absence of other neoplasms in the last 2 years; iii. the absence of a clinical history reporting cutaneous melanoma. Nine patients with the diagnosis of OMM and nine cases reporting intraoral nevi were selected, between the years 2005 and 2019. The clinical and histologic features were investigated, and critically compared, by the same two resident oral pathologists (E.M. and E.C.), and one dermatopathologist (G.C.). All the samples selected and included in this investigation were stored in the same labs and analyzed (5-micron thick sections) with the immunostaining targeted against the recombinant Anti-PRAME antibody [EPR20330] (ab219650). We also decided to compare the PRAME expression in the selected samples with the clinical behavior  of the patients: more in detail, we investigate the overall percentage of PRAME-positive cells compared to the overall intensity of the immunostaining, by using a cumulative scale considering the amount of tumor cells  and the related PRAME expression intensity in such tumor cells . Sebaceous glands were used as a positive control. Finally, all the reported cases have been further assessed by two independent pathologists, not involved in this study.We recruited nine patients diagnosed for OMM ; these patients were 65 to 87 years old, with similar baseline clinical conditions.in situ, just 1 year following treatment.The OMM onset was diagnosed on hard palate (8 cases) and on gingiva (1 case). All the reported nine cases showed invasive melanomas, three of which were subjected to clinical relapses as melanoma Nine (9) patients affected by oral mucosal nevi (MN), 4 males and 5 females, were from 21 to 48 years old. The oral nevi onset was tissue-specific, as follows: on the gingiva (4 cases), on mucous membrane of the cheek (2 cases), on labial mucosa (1), alveolar ridge (1) and vermilion (1). All the reported lesions were safely removed without any relapse after 5 years.Tables 1 and 2.Clinical and pathological features of these patients were summarized in To better correlate the PRAME expression with its nature , we categorized PRAME tumor cells' percentage positivity and intensity of immunostaining in a cumulative score, obtained by adding the quartile of positive tumor cells  to the PRAME expression intensity in tumor cells . More specifically, we used the following scores for the percentage positivity of tumor cells: 0% (score 0), 1% to 25% (score 1+), 26% to 50% (score 2+), 51% to 75% (score 3+), and 76% to 100% (score 4+). Furthermore, we used a score for intensity by measuring the nuclear immunostaining for PRAME as weak, moderate, or strong . Sebaceous glands were used as an internal control to confirm the functioning of the PRAME antibody stain. These characteristics of immunoexpression were estimated by both pathologists during the review of the cases.Eight out of nine cases of OMM were diffusely and intensely positive for PRAME with a total score of 6 + / 7 +; only 1 case was positive for PRAME with a total score of 4+/7+  The immunohistochemical characterization of OMMs is well described by the scientific literature: it can be focally positive to the pan-cytokeratin markers (AE1AE3 or MNF116), and the positivity is usually dot-like, as a diffuse and homogeneous staining is not evident in these lesions. On the other hand, OMMs can be strongly characterized by the HMB45, Melan-A and S100 protein staining. Generally, both the clinical and histological differential diagnosis are not always easy to do, as specific forms of OMMs can be negative (or mildly positive) to the classic melanocytic markers, such as Melan-A (MART1) and Human Black Melanoma Antigen-45 (HMB-45) et al.In this landscape, the scientific community has highlighted the need to find a novel diagnostic approach that may be useful to the clinicians, to the oral pathologists and to the dermatopathologist, especially for those cases where a complex differential diagnosis is required. In the recent years, several studies have found the reliability of the marker called PReferentially expressed Antigen in MElanoma (PRAME), as a strategic advisor about the potential malignancy of a dermatological pigmented lesion. More in details, after the first paper by Lezcano et al.in situ and in invasive melanomas. These preliminary data have been firstly confirmed by our paper: here, in fact, we have reported the same intensity and distribution of PRAME immunostaining applied to the OMMs; similarly, we have also highlighted the negativity to PRAME in all the mucosal nevi involved in our pilot investigation.Interestingly, to our knowledge, only 1 paper has been published, by Hovander et al.in situ, further demonstrating the usefulness of PRAME in such lesions.In a recent paper, Vos It is worthy of discussion also the immunomodulatory effects of mesenchymal stem cells (MSCs) in oncological and inflammatory lesions in situ OMM, but also because it first suggests the use of PRAME as a reliable marker, and a kind of surgical guide during the surgical resection of the tumoral margins, improving the chance to successfully achieve the tumor eradication In conclusion, our paper can be considered of particular interest not only because it reports the use of PRAME in the differential diagnosis between intraoral nevi and It is also remarkable that the OMMs are not so common lesions, thus easily subject to wrong diagnosis and clinical misinterpretations; on the other side, there is a concrete limit related to the current difficulty to improve the workflow of all the Pathology laboratories Differential diagnosis among several benign pigmented lesions of the oral cavity and the OMM represent a challenging topic still little investigated, basically because of the relatively scarce incidence of these lesions. Nevertheless, a late diagnosis could lead to a life-threatening condition for the patient; in fact, both these lesions are usually asymptomatic in the early stages, even showing misleading morphological characteristics. To get a final diagnosis, the physician should require accurate clinical evaluations, as well as specific histological examinations based on the use of different markers that may not have a clear and diriment expression. The use of novel and more specific immunohistochemical markers, able to discriminate among benign pigmented lesions and OMM is undoubtedly a strategic support for a reliable differential diagnosis. Our pilot study has retrospectively and critically assessed the use of PRAME as a valid support in this regard. The authors need to underline that to definitely understand pros and cons of PRAME in OMM diagnosis, further studies are desirable. According to the current scientific literature, this is one of the first studies addressing this clinical challenge: the use of PRAME-supported diagnosis can be a breakthrough in this field, ensuring a more accurate and quick diagnosis. Of course, several trials need to be carried out to strengthen this protocol in the daily clinical practice; our scientific contribute is the first milestone in this direction, and we hope it will be the driving force for future investigations.


Bronchoscopy with radial-probe endobronchial ultrasound, a guide sheath, and electromagnetic navigation can improve the diagnostic yield of peripheral lung nodules. However, the suitability of specimens for genetic analysis remains unsatisfactory. We hypothesized that a transbronchial biopsy performed after closely approaching the bronchoscope tip to the lesion might provide more suitable specimens for genetic analysis. We enrolled 155 patients with peripheral pulmonary lesions who underwent bronchoscopy with a thin or ultrathin bronchoscope. Bronchoscopy was performed using virtual bronchoscopic navigation and radial-probe endobronchial ultrasound with a guide sheath. The bronchoscope tip was placed closer to the lesion during bronchoscopy to collect larger specimens with higher malignant cell content. The patients who underwent a close-to-lesion biopsy had higher rates of overall diagnostic yield, histopathological diagnostic yield, and specimen quality for genetic testing than those who did not. The significant determinants of the specimen’s suitability were the close-to-lesion approach, within-the-lesion image, the use of standard 1.9-mm-forceps, and the number of cancer-cell-positive specimens. The significant predictors of the specimen’s suitability for genetic analysis were close-to-lesion biopsy and the number of malignant cell-positive tissue samples. This study demonstrates that the close-to-lesion transbronchial biopsy significantly improves the suitability of bronchoscopic specimens for genetic analysis. The diagnostic yield of subsegmental pulmonary lesions ≤ 30 mm is 80.4% when VBN and RP-EBUS are used in combination4. The American College of Chest Physicians (ACCP) guideline recommends using RP-EBUS or electromagnetic navigation (EMN) because these techniques substantially improve the diagnostic yield5. The RP-EBUS image of the lesion during bronchoscopy has a diagnostic significance. The diagnostic yield is 83% when the image is within the lesion, 61% when adjacent to the lesion, and only 4% when outside the lesion6. Transbronchial biopsy performed with an RP-EBUS image within the lesion provides the highest diagnostic yield; however, there are cases where the diagnosis cannot be confirmed due to low quality and inappropriate volume of biopsied tissues. Improper tissue sampling is also a major obstacle to achieving a successful genetic analysis of lung cancer using next-generation sequencing (NGS). NGS has become a necessary test for expanding therapeutic options in patients with non-small cell lung cancer (NSCLC), so it is essential to ensure that an appropriate volume and high-quality tissue samples are collected during transbronchial biopsy7. Large 1.9-mm forceps and cryo-probes are recommended for this purpose8. However, tissue sampling of peripheral pulmonary lesions with 1.9-mm forceps and cryoprobes is generally not performed because bronchoscopes with small diameters are routinely used in clinical practice.In recent years, the accuracy of computed tomography (CT) has led to increased detection of small peripheral pulmonary lesions. The diagnostic yield of pulmonary lesions less than 2 cm is low using conventional bronchoscopic methods. However, the introduction of virtual bronchoscopic navigation (VBN) and radial-probe endobronchial ultrasound (RP-EBUS) with guide sheath has significantly improved the diagnostic yield of peripheral pulmonary lesions7. We hypothesize that bringing the bronchoscope tip closer to the lesion will allow us to collect larger specimens with higher malignant cell content and improve the suitability for genetic testing of specimens collected from non-small cell lung cancer (NSCLC) patients with peripheral pulmonary lesions. To test this hypothesis, we propose a modified bronchoscopic technique that differs from the conventional procedure as follows: after confirming the lesion position with the RP-EBUS, we use the guide sheath as a rail to insert the bronchoscope tip closer to the lesion under RP-EBUS and fluoroscopy guidance. Then, we remove the RP-EBUS and the guide sheath after re-confirming the lesion position and introduce a 1.9-mm forceps for tissue sampling. Using this novel approach, we assessed the diagnostic yield of peripheral pulmonary lesions and the suitability rate for NGS analysis of the collected tissue samples.The suitability for NGS of specimens collected by bronchoscopy in lung cancer patients with peripheral pulmonary lesions is unclear. Previous studies have reported the outcomes of NGS analysis using tissue samples obtained by combining different methods, including transbronchial biopsy, computed tomography-guided needle biopsy, or surgical resection in patients with lung cancer. However, there is no data on the suitability and success of NGS analysis using tissue samples collected by bronchoscopy in lung cancer patients with peripheral pulmonary lesions. The volume and the malignant cell content of the collected specimen are crucial for the success of NGS analysisThis study enrolled 155 patients with peripheral pulmonary lesions that underwent bronchoscopy using a thin bronchoscope with RP-EBUS and guide sheath or an ultrathin bronchoscope with RP-EBUS from September 2021 through November 2022  bronchus sign, overall and histopathological diagnoses, bronchoscope tip close-to-lesion biopsy, and lesion size, RP-EBUS image, chest X-ray finding, fluoroscopy visualization, and lung distribution (2.5 cm from the hilum/intermediate/2.5 cm from the pleura)3.All patients underwent CT  prior to surgery, and VBN images were generated using CT-Digital Imaging and Communications in Medicine (CT-DICOM) data . Bronchoscopy was performed using the thin flexible bronchoscope type P290  or the ultrathin bronchoscope type MP290F . Topical lidocaine was used for pharyngeal anesthesia, while intravenous midazolam and fentanyl were administered for patient sedation. The airways were nebulized with lidocaine using a 1.9-mm spray catheter , and additional intravenous fentanyl was administered as needed. Bronchoscopy with a thin or ultrathin bronchoscope was conducted with VBN, fluoroscopy, and 1.4-mm RP-EBUS  with or without a 1.95-mm guide sheath . Biopsy was performed using a 1.5-mm  or a 1.9-mm forceps . Bronchial brushing and washing with 20 mL of physiological saline were performed after a close-to-lesion transbronchial biopsy (CL-TBB). As previously described, RP-EBUS images were categorized as within or adjacent to the lesion or invisible10.A tissue sampling was defined as a CL-TBB when the bronchoscope tip was within or adjacent to the lesion by RP-EBUS and ≤ 10 mm from the lesion under fluoroscopy Fig. . We asseThe final diagnosis in all cases was made based on histopathological, cytological, microbiological, culture, and clinical findings. Samples collected by bronchoscopy , CT-guided lung biopsy, or surgery were used for pathological diagnosis. Lung cancer was diagnosed when any of these samples confirmed malignancy. Organizing pneumonia was diagnosed when the bronchoscopy showed no significant findings, CT-guided lung biopsy or surgery was not performed, or a subsequent CT study confirmed the disappearance or shrinkage of the lesion.The thin bronchoscope was advanced through the bronchus segment and positioned as close to the lesion as possible under the airway lumen and VBN image guidance. Then, the RP-EBUS with the guide sheath was inserted through the 2-mm working channel of the thin bronchoscope until the RP-EBUS-guide sheath tip reached the lesion, and this can be visualized and confirmed by RP-EBUS. When the RP-EBUS image showed within or adjacent to the lesion, the bronchoscope tip was further advanced as close to the lesion as possible using the RP-EBUS-guide sheath as a rail under RP-EBUS and fluoroscopy guidance Fig. . During The thin bronchoscope was substituted by the ultrathin bronchoscope when the close-to-lesion approach was impossible because the lesion was invisible by RP-EBUS or because the area of contact with the lesion was less than 180° when the RP-EBUS image was adjacent to the lesion. Neither the guide sheath (1.95-mm) nor the standard (1.9-mm) forceps can be used with the ultrathin flexible bronchoscope because its working channel is only 1.7-mm in diameter. Therefore, the biopsy was performed after bronchoscope substitution using the 1.5-mm forceps under RP-EBUS (1.4-mm) and fluoroscopy guidance. The technique for approaching close to the lesion was similar to that used with the thin bronchoscope. The ultrathin bronchoscope was advanced through the bronchial segments toward the lesion under RP-EBUS guidance until the bronchoscope tip was as close to the lesion as possible. The RP-EBUS was then inserted through the bronchoscope working channel until the RP-EBUS tip reached the lesion area, and the lesion could be visualized and confirmed by fluoroscopy and RP-EBUS. After confirming the RP-EBUS image was within or adjacent to the lesion, the RP-EBUS was withdrawn to introduce a 1.5-mm forceps and collect ten tissue samples.Medical records were examined for procedure-related complications within one month after the bronchoscopy. After the transbronchial biopsy, the forceps was withdrawn, but the bronchoscope tip or guide sheath was kept securely in the bronchial branch for 3 min for compressive hemostasis to prevent bleeding.P < 0.05 was considered significant.SPSS Statistics version 23 software  was used for statistical analysis. The chi-square test was used to compare proportions between groups, and the Mann–Whitney U test was used to analyze continuous variables. Logistic regression analysis was used to identify factors that can predict the suitability of collected specimens for NGS analysis. A two-sided During the investigation period, 352 patients underwent bronchoscopy for lung parenchymal lesions. However, only 155 were eligible for the study after excluding patients with central (n = 28) or intermediate (n = 85) lung segmental lesions, ground glass nodules (n = 37), cavitary lesions (n = 28), or patients undergoing examination with thick bronchoscope (n = 12). We also excluded a case undergoing a second biopsy, a patient who did not consent to substitute the bronchoscope, and those lost to follow-up (n = 5)  mm. Among all 155 patients, 81.3% had lesions confirmed by fluoroscopy, and 80.0% showed a CT bronchus sign. Thirty-eight 24.5%) required the substitution of the thin bronchoscope with an ultrathin bronchoscope because the lesion was invisible by RP-EBUS (n = 21), sample collection was not possible by forceps bending (n = 2) in patients with RP-EBUS image within the lesion, or the area of contact with the lesion was less than 180° in patients (n = 15) with RP-EBUS image adjacent to the lesion  patients diagnosed with a malignant lung tumor , and 39 (25.2%) with benign lung disease. A final diagnosis was not achieved in 8 cases (Table P = 0.023) in the CL-TBB group than in the non-CL-TBB group. However, the lesion size was not significantly different between both groups (Table Close-to-lesion biopsy was possible in 82 patients (52.9%), as shown in Fig. P = 0.019) than specimens collected from patients who did not undergo CL-TBB. In addition, the lesion visibility under fluoroscopy  was also significantly increased in the CL-TBB group compared to the non-CL-TBB group (Table Among patients with NSCLC (n = 94), adenocarcinoma was more frequent than squamous cell carcinoma. Moreover, specimens collected from patients who underwent CL-TBB showed a significantly higher NGS suitability rate . Only two patients in the CL-TBB group required hospitalization. One had grade 3 anorexia, and the other had grade 3 pneumonia. In contrast, three patients in the non-CL-TBB group were hospitalized. Two had grade 3 hypoxemia, and one had grade 3 pneumonia. No pneumothorax, bleeding, or bronchial tears that needed therapeutic intervention occurred in either group. These findings suggest that CL-TBB is a safe and feasible technique for patients with peripheral pulmonary lesions.Complications were graded according to the Common Terminology Criteria for Adverse Events  13. However, there is still room for improvement. For instance, a biopsy performed with an RP-EBUS image within the lesion means that a tissue sample was collected from inside the lesion, and thus, theoretically, the pathology should be diagnosed in all cases. However, in some cases, the pathology of the lesion is not diagnosed due to insufficient tissue volume, uneven distribution of pathological areas, or poor technical approach14. The thin bronchoscope with RP-EBUS-guide sheath is most commonly used to approach peripheral lung lesions and collect tissue samples by transbronchial biopsy using a 1.5-mm forceps. The main problem with this technique is that the 1.5-mm forceps can only collect a small volume of tissue7. The standard 1.9-mm forceps can also be used with the thin bronchoscope but without the guide sheath. However, a previous prospective study has shown that the diagnostic yield is better using a guide sheath, which only allows using a 1.5-mm forceps3. The guide sheath enables highly reproducible transbronchial biopsy of peripheral lung lesions. Another prospective study has demonstrated that patients undergoing bronchoscopy with the ultrathin bronchoscope had significantly higher accessibility to peripheral lung lesions and improved diagnostic yield than those undergoing bronchoscopy with the thin bronchoscope12. However, the large 1.9-mm forceps cannot be used with the ultrathin bronchoscope. Therefore, techniques are urgently needed to allow reproducible transbronchial biopsy using the thin or ultrathin bronchoscope or a combination of both bronchoscopes.The recent widespread use of CT scans and other imaging technologies has increased the number of patients diagnosed with small peripheral pulmonary lesions. This trend has resulted in a compelling need for bronchoscopists to collect an appropriate volume of samples for accurate pathological diagnosis of these lesions. Several new bronchoscopic techniques have been developed recently to address this need, resulting in a substantially increased diagnostic yield of peripheral pulmonary lesions. For example, since the introduction of RP-EBUS-guide sheath, virtual bronchial navigation, ultrathin bronchoscope, robot bronchoscope, and cone-beam CT, the diagnostic yield of peripheral pulmonary lesions has increased by 60–80%15. Therefore, collecting an adequate volume of high-quality tissue samples during the transbronchial biopsy is essential. Many reports have shown that NGS analysis failed due to insufficient amount or poor quality of tissues collected by transbronchial biopsy using small forceps7. Murakami et al. performed NGS analysis on 69 of 184 patients diagnosed with NSCLC and found that the tissue surface area and the success rate of DNA (RNA) sequencing were significantly higher when samples were collected using large forceps7. Thick bronchoscopes used for therapeutic purposes are inappropriate for transbronchial biopsy of peripheral lung lesions due to the small diameter of peripheral bronchial branches. We generally use a thin bronchoscope with an RP-EBUS-guide sheath in routine bronchoscopic studies14. When we confirm that the RP-EBUS image is within or adjacent to the lesion, we advance the bronchoscope tip as close to the lesion as possible before performing transbronchial biopsy because this increases the diagnostic yield16. In cases where the RP-EBUS image is within the lesion, but CL-TBB is impossible with the thin bronchoscope, we perform transbronchial biopsy through the guide sheath using 1.5-mm forceps. However, in cases where the RP-EBUS image is adjacent to the lesion or invisible, and CL-TBB is possible, we remove the thin bronchoscope and introduce the ultrathin bronchoscope to collect tissue samples by the CL-TBB technique. When the transbronchial biopsy is performed through the guide sheath, tissue samples can only be collected with forceps in one direction. As a result, the biopsy samples may contain excessive blood components with less lesional tissue because they are only from one portion of the lesion. However, when CL-TBB is performed after removing the guide sheath and inserting a 1.9-mm forceps, tissue samples can be collected from different portions of the lesion by changing the angle of the bronchoscope tip.Genetic analysis using NGS is becoming necessary for expanding therapeutic options in patients with NSCLC3. Therefore, the 1.5-mm forceps should be replaced by standard (1.9-mm) forceps only when the bronchoscope tip can be confirmed with the RP-EBUS image. Moreover, in cases where the RP-EBUS is adjacent to the lesion, and the CL-TBB technique is possible, the RP-EBUS image can be changed to within the lesion by adjusting the angle of the bronchoscope tip. The CL-TBB technique improved the histopathological diagnostic yield among eligible patients in the present study. Furthermore, in 94 NSCLC cases where CL-TBB was possible, the suitability of the specimens for NGS analysis was significantly increased, and the overall and histopathological diagnostic yields were high compared to controls.A prospective study has shown that using a thin bronchoscope with a guide sheath under RP-EBUS guidance improves the histopathological diagnostic yieldThis study showed that the CL-TBB technique, the RP-EBUS image within the lesion, the use of 1.9-mm forceps, and the number of samples with malignant cells were determinant factors for the suitability of collected specimens for NGS analysis. According to logistic regression analysis, the main predictors of suitable NGS specimens were the CL-TBB technique and the number of samples with cancerous tissues. Large lesions and RP-EBUS images within the lesion are generally known to improve the diagnostic yield of peripheral pulmonary lesions. However, in this study, the NGS suitability rate of collected specimens did not depend on the lesion size. Notably, there were cases where the bronchoscope lens directly touched the lesions, and the lesions could not be directly observed due to clot formation.17. Intrabronchial hemorrhage may occur after a biopsy. Five samples are usually collected for diagnosis of peripheral pulmonary lesions. In this study, we collected 10 biopsy samples and observed no increased complications compared to previous studies. After tissue sampling, we kept the bronchoscope tip, and the guide sheath wedged at the biopsy site for 3 min to prevent local bleeding after the biopsy. The current bronchoscopic procedures are safer than previous conventional procedures because selective wedge of the peripheral bronchial branch is much easier with small-diameter bronchoscopes, and compressive hemostasis can be done using the guide sheath.Complications previously reported during bronchoscopy include bleeding in the airways, pneumothorax, and pneumoniaThe limitations of this study are its retrospective nature, single-institution setting, and small sample size. Therefore, future studies with larger populations are needed to confirm these findings.This study demonstrates that the CL-TBB significantly improves the suitability of specimens collected by bronchoscopy for NGS analysis.Supplementary Legends.Supplementary Video 1.


The VR_AICENICU program comprised three scenarios: high-risk medication with lipid solution, dressing and management for peripheral inserted central line, and aspiration prevention and skincare management during ventilator use for premature infants. Data were collected between February and July 2022. The experimental group showed significantly greater improvements in infection control knowledge and performance confidence compared to the pre-test. The average scores of presence, empathy, and program satisfaction of the experimental group were 4.39 ± 0.36, 4.33 ± 0.75, and 4.90 ± 0.31, respectively. The VR_AICENICU program has implications for the education needs of nurses working in NICUs and enhances their knowledge and performance of infection control. Future studies should apply the VR_AICENICU program to different severity grades of NICU patients and develop additional VR programs.This study examined the effects of a virtual reality simulation for advanced infection control education in neonatal intensive care units (VR_AICENICU) on nurses’ infection control knowledge and performance confidence. We examined nurses’ presence, empathy, and program satisfaction using a non-equivalent control group pre-post design. Nurses were divided into an experimental group ( Healthcare-associated infections (HAIs) or nosocomial infections are among the major causes of mortality and morbidity in newborn infants admitted to the neonatal intensive care unit (NICU). Considering the incident of newborn deaths from NICU infections in South Korea in 2018, operating a NICU infection control system has been made mandatory nationwide since 2019 . In the Effective infection control education is critical in preventing HAIs and reducing their incidence , especiaConfidence in nursing was increased among nurses through a VR-based learning intervention involving patients on ventilators  and amonHowever, most VR simulation studies have been conducted with nursing students, and few studies involve practicing nurses. Although some studies on neonatal care exist ,27, theyThis study aimed to develop and implement a VR simulation for advanced infection control education in NICUs (VR_AICENICU); assess its effectiveness in enhancing NICU nurses’ knowledge and confidence in infection control; and evaluate nurses’ levels of presence, empathy, and satisfaction associated with the program.Hypothesis 1. The experimental group exposed to the VR_AICENICU program will score higher on “knowledge of infection control” in the post-intervention test than the control group, who have not been exposed to the VR_AICENICU program.Hypothesis 2. The experimental group will score higher on “confidence in infection control” in the post-intervention test than the control group.This study used a non-equivalent control group pretest-post-test design to identify the VR_AICENICU program’s effects on infection control knowledge, confidence, presence, empathy, and program satisfaction in NICU nurses.This study included NICU nurses working in two general hospitals. Both hospitals have similar numbers of beds, nurse staffing, and patient characteristics, but they are located in different regions. Participants were recruited at two geographically distant sites to prevent contamination between the groups. The experimental group was from G University Hospital in J City and the control group was from CG University Hospital in C City. The inclusion criteria were nurses working in patient care in the NICU. The minimum required sample size was calculated to be 37 using the G * power 3.1.9 program, assuming an effect size (f) set of 0.55, a significance level (α) of 0.05, and a power (1 − β) of 0.90. A large effect size was selected based on comparable previous studies, which reported effect sizes ranging from 0.40 to 0.68 ,28. ConsParticipants’ general characteristics included age, sex, education level, marital status, clinical career, NICU work experience, position, prior experience with NICU infection control education, and prior experience with VR-related education.Infection control knowledge was measured using the knowledge-related items excerpted from the ‘’high-risk neonatal infection control competency scale’’ developed by Yu et al.  for NICUConfidence in infection control was measured using the items of the “performance” domain of the high-risk neonatal infection control competency scale developed by Yu et al.  for NICUPresence was measured using the ‘’presence scale”, with 19 items, developed by Chung and Yang  for 3D iEmpathy was measured using the “VR-related empathy scale” developed by Oh , with itProgram satisfaction was measured using the three-item VR program satisfaction scale developed by Yu et al. . ParticiThe VR_AICENICU program was developed as follows.Three infection control scenarios were selected for VR simulation: catheter-related bloodstream infection (CRBSI), infection-prone intralipid infusion, and ventilator-associated infection, which are healthcare quality indicators for infection control. These scenarios involve procedures for preventing CRBSI at the catheter insertion site for drug injection in high-risk newborns hospitalized for an extended period, managing intralipid infusion vulnerable to contamination when left at room temperature for a prolonged period or infused in portions, managing the respiratory circuit for a noninvasive NCPAP ventilator, performing gastric decompression to prevent gastric juice reflux, and preventing skin damage from nose mask compression. NICU nurses and doctors must be familiar with these measures, which are infection control techniques and nursing procedures.The content validity of the statements and procedures of the scenarios was evaluated by a seven-member expert panel comprising a nursing professor, a NICU nurse manager, three experienced nurses with at least 10 years of clinical experience, a teaching nurse with a master’s degree in child nursing and VR simulation development experience, and a nurse specializing in nosocomial infection. The Delphi technique was used to gather experts’ opinions on each procedure or statement using a four-point Likert scale .The panel members were also requested to provide their opinions on related topics through open-ended questions. An item-level content validity index (I-CVI) was used to determine those who rated 3 or 4 on each item, and the items with an I-CVI of 0.80 or higher were used without modifications, while those lower than 0.80 were revised three times. Expert opinions were documented in an open-ended format. The final versions of the three scenarios included 18 steps related to CRBSI, 15 related to intralipid infusion, and 13 related to noninvasive ventilator infection control.To implement the three scenario modules as a VR simulation program, a VR environment mimicking the NICU setting was created using equipment and tools, including an automatic faucet, a radiant warmer, an incubator with a cover, a treatment table, an infusion pump, a ventilator, a biohazard waste bin, a preterm infant mannequin, and various medicines and disinfectants. A research assistant and a neonatal education nurse conducted demonstrations following the scenario steps. Based on the demonstration recording, the VR program was completed through the 3D modeling and animation production process. Hand-tracking technology enables the user to select objects through hand movement. Audio sources replicating sounds such as a baby’s cry and alarms commonly heard in a NICU setting were added to provide a more realistic, engaging, and immersive experience .TM  hand-tracking device with a VR Developer Mount. Additionally, one large TV monitor was employed to allow multiple users to view the program concurrently. The VR simulation program software was commissioned by a VR manufacturer  based on prerecorded images and video recordings of the three simulation scenarios.The equipment used to produce the VR simulation program included a VR kit containing an EliteDesk 800 G4 laptop computer, a Vive Pro Full-Kit Head Mounted Display (HMD) and sensor , and a LEAP Motion ControllerThe experimental group underwent pre-survey, experimental treatment, and post-survey from 1–20 February 2022. The control group underwent pre-survey and post-survey from 7–30 July 2022, in accordance with the approval date from the Institutional Review Board. The details of the procedure are described below.The pre-survey, which collected information on general characteristics, infection control knowledge, and confidence in infection control, was conducted in early February for the experimental group and in early July for the control group. The questionnaire was distributed and collected by the researcher in person.The 20 participants in the experimental group were divided into small groups of three to five members considering their work schedule, and the program was conducted from 10–15 February 2022. Each program session comprised three stages: pre-briefing, VR simulation, and debriefing. The pre-briefing stage, which lasted 20 min, was divided into three parts: infection control principles’ overview and scenarios presentation, VR simulation practice environment presentation, and HMD and LEAP motion controller use demonstration along with precautions. The VR simulation lasted 10–15 min per scenario. During a 20-min debriefing stage, users discussed their VR simulation experience and its applicability to education. The program lasted 80 min in total . As for The experimental group was instructed to complete the post-survey immediately after the program ended. The control group was given the post-survey two weeks after the pre-survey to prevent the testing effect and maintain internal validity.t-test, and non-parametric Mann–Whitney test. The study variables’ normality was examined by performing the Shapiro–Wilk test. Intergroup differences in infection control knowledge and performance confidence were analyzed by performing Wilcoxon’s signed-rank and Mann–Whitney tests. The presence, empathy, and program satisfaction of the experimental group were analyzed using mean and standard deviation.Data analysis was performed using the SPSS/WIN 25.0 program . General characteristics were analyzed using frequency, percentage, mean, and standard deviation. The homogeneity between the experimental and control groups was tested by performing the chi-square test, independent Before data collection, approval was obtained from the Institutional Review Board of the two hospitals where the experimental and control groups were recruited (GNUH 2021-09-024-002 and GNUCH 2022-05-023-001). The researcher explained the study purpose and procedure to the participants, the anonymity and confidentiality in handling the data, and their right to withdraw from participation at any time and obtained their written consent. Each completed questionnaire was placed in an envelope and collected directly by the researcher. Each participant was compensated with a gift card worth approximately 30 USD, along with snacks. The control group was informed of the possibility of participating in the VR simulation program and receiving infection control education materials. 2 = 9.23, p = 0.002). Regarding study variables, homogeneity was confirmed only in infection control knowledge , with the control group showing no significant change . However, as no intergroup differences were observed in the post-hoc test score , hypothesis 1 was rejected. Regarding individual items of infection control knowledge, while no significant intergroup differences were observed in invasive procedures, instrument management, and medication management, the mean score for basic infection control was significantly higher in the experimental group than in the control group , with no significant change observed in the control group . The experimental group scored significantly higher for confidence in infection control than the control group , supporting Hypothesis 2. Considering individual items of confidence in infection control, the experimental group scored significantly higher than the control group in basic infection control , and significant intergroup differences were observed in invasive procedures, instrument management, and medication management  line, and newborn care on NCPAP. In this process, NICU nurses can apply aseptic techniques when performing high-risk medication insertion and infusion, assessing PICC site infection, preparing items necessary for dressing changes, dressing the PICC insertion site, handling the NCPAP machine, assessing the skin around a nasal mask, and performing respiratory nursing. These procedures can help NICU nurses learn techniques for preventing nosocomial infections related to CRBSI, ventilator-related pneumonia, and skin infections frequent in premature infants. Additionally, to help users practice proper hand hygiene and personal protective equipment use during sterile dressing changes, the execution time for each procedure is set , and the program algorithm is arranged to allow users to progress to the next step only after appropriate execution of the current step. Therefore, users can accurately learn the basic infection control procedures and sequences by repeatedly practicing on a neonatal mannequin in a safe virtual environment even if they make mistakes. The scenarios developed in this study correspond to the medications and treatments that nurses use when treating premature infants in a NICU setting. However, to ensure patient safety, novice nurses are not allowed to care for severely ill, high-risk newborns or premature infants without relevant treatment experience. Therefore, this study’s educational course provides an appropriate training program for novice nurses to care for premature infants. The satisfaction score obtained in this study was higher than scores achieved by nursing students  exposed Based on these opinions, the VR_AICENICU program could benefit both introductory education for new nurses and relearning education for experienced nurses. Additionally, the high level of program satisfaction and intention to recommend the program to others suggests that it can become a part of nurse education, providing the empirical foundation for the integration of the VR SBE program. Comparing infection control knowledge and performance confidence between NICU nurses who received VR-based simulation education  and those who received conventional lecture-centered oriented education (control group) showed that the experimental group scored significantly higher in infection control knowledge. This finding aligned with those of Yoon’s  study, wSBE can induce learning motivation by improving learners’ confidence, the main advantage of applying simulation to education . In thisThe VR_AICENICU program was developed using actual demonstrations of the nursing procedures by a NICU training nurse following the planned scenarios, modeling of the demonstrations, and several revisions for program refinement. The final program accurately replicated NICUs’ materials and equipment, such as an incubator and an infusion pump, and included background noises such as monitor alarms, phone ringing, and nurses talking. However, there were technical issues such as functional errors in recognizing nurses’ movements, which sometimes required repeating the exact movement, and loading-related disconnections. These limitations could be addressed by enhancing the LEAP motion sensor sensitivity, an important VR device, and reducing program errors to increase user immersion and presence.The experimental group had a mean post-test empathy score of 4.33 out of 5 points, showing a higher level of empathy for patients in the VR simulation video than in previous studies using the same empathy scale of 3.80 in a study using VR 360-degree intravenous infusion training for nursing students  and 3.16This study is significant because it is the first VR simulation-based advanced infection care training program for NICU nurses that includes implementation and assessment. The results provide empirical data for introducing and utilizing VR simulation-based training programs for nurse education and practical training. The VR_AICENICU program can help nurses learn and practice nursing skills, which are challenging to perform on high-risk newborns, by providing realistic experiential learning opportunities in a safe VR environment. It can also help nurses who have temporal constraints due to shiftwork. Since it enables nurses to learn through repeated practice on a virtual patient in a safe virtual environment, it can strengthen nursing competency, improving patient safety.This study has some limitations. First, since convenience sampling was used to recruit NICU nurses from two general hospitals in two different regions, caution is required when generalizing the results to all NICU nurses. Second, homogeneity in education level between the experimental and control groups could not be established because the NICU nurses from the experimental and control groups were from two different general hospitals. Additionally, owing to the limited number of NICU nurses, all nurses except managers were enrolled in this study. Therefore, differences in participants’ education level and nursing experience based on the severity level of high-risk newborns could not be controlled for in the baseline homogeneity test.Participants who received the VR_AICENICU program showed increased infection control knowledge and performance confidence, and they attained high scores for presence in the VR environment and empathy for patients. These results verify the feasibility of the VR_AICENICU program as an effective infection control training tool for new and experienced nurses. Future studies should apply the VR_AICENICU program to different severity grades of NICU patients and develop additional VR programs, given the positive outcomes, to enhance nursing competencies in other areas and test their effectiveness.


The World Trade Center Health Program (Program) provides limited health care to those directly affected by the 9/11 terrorist attacks. Because of physical/mental trauma arising from the 9/11 attacks, Program members might be at high risk of opioid use. To prevent prescription opioid overuse, in 2018 the Program implemented various measures to improve opioid prescribing and expand access to non-opioid pain management among Program members. However, the characteristics of opioid prescriptions dispensed among this population has never been described.Administrative and claims data from 07/01/2011 to 09/30/2022 were used to describe opioid prescriptions dispensed during 2013–2021.n = 70,721), the proportion with opioid prescriptions for cancer/hospice/end of life care increased from 0.5% in 2013 to 1.6% in 2018 (p = 0.010), then decreased to 1.1% in 2021 (p = 0.070), and the proportion for non-cancer surgery/acute/chronic pain decreased from 0.6% in 2013 to 0.23% in 2021 (p = 0.0005). Among members prescribed opioids without cancer/hospice/sickle cell disease, the proportion who started with long-acting opioids or had opioid prescriptions from ≥ 4 prescribers were below 6.5% annually; the proportion receiving a high-dose , or with concurrent opioids and benzodiazepines use, or who started opioids with MED ≥ 50 or with long duration (≥ 7 days’ supply) were above 10% annually, but decreased since 2017.From 2013–2021, 108,285 members were Program-enrolled for ≥ 10 months, 4,053 (3.7%) had 22,938 outpatient opioid prescriptions, of which, 62.1% were for cancer-related pain, 11.1% for hospice/end of life care, 4.8% for surgery pain, and 9.8% for acute/chronic pain. Among members with Program-paid diagnostic/treatment claims (Prevalence of outpatient opioid prescriptions paid by the Program was very low and prescriptions were primarily dispensed for cancer/hospice/end of life care. Although Program efforts to improve opioid prescribing coincided with improvements in outcomes, ongoing surveillance is needed. The opioid overdose epidemic is one of the most pressing public health challenges in the United States. Opioid-involved overdose deaths increased from 21,089 in 2010 to 80,411 in 2021, and overdose deaths involving natural and semisynthetic opioids , a category inclusive of prescription opioids, rose from 3,479 in 2001 to a peak of 14,495 in 2017, with 13,618 in 2021  §§ 300 mm – 300 mm-61) to provide health care to those directly affected by the 9/11 attacks. The Program is a limited health benefit program, which covers monitoring and treatment for specified health conditions resulting from exposures related to the 9/11 terrorist attacks in New York, the Pentagon, and in Shanksville, Pennsylvania. To become a Program member and receive Program health benefits, one must meet certain exposure, latency, and symptom-onset requirements for listed conditions.Individuals potentially eligible to enroll in the Program include emergency responders and other workers or volunteers who were directly involved in the rescue, recovery, clean-up, and related support activities following the attack (collectively referred to as “responders”), and those who were present in the New York City disaster area on 9/11 or its aftermath because of their work, residence, or attendance at school or day care (referred to as “survivors”). It is estimated that potentially eligible individuals include more than 17,000 responders from the Fire Department of New York City (FDNY), more than 80,000 non-FDNY general responders, and more than 360,000 survivors [Program members are a unique population because of their environmental exposures and physical and mental trauma arising from the terrorist attacks. As such, this population has an increased risk for certain covered health conditions, including, but not limited to, aerodigestive, musculoskeletal, mental health, and cancer conditions , 10. A mTo improve opioid prescribing and expand access to non-opioid pain management among members, the Program implemented various measures beginning in September 2018, such as: I) first opioid prescription cannot exceed a 7-day supply and implemented a point-of-sale limit on opioid prescriptions with > 90 morphine milligram equivalents (MMEs) per day; II) added coverage for non-opioid pain management , and removed coverage of opioids with high misuse or overdose potential, specifically methadone and all oral fentanyl products; III) semi-annually reviewed controlled substance prescription data to identify potential fraud, waste, and abuse; and IV) periodically identified members with opioids prescriptions that were considered potentially high-risk: those who receive opioid prescriptions from two or more prescribers or two or more pharmacies, and members with average daily opioid utilization > 90 MMEs, and/or received concurrent treatment with opioids and benzodiazepines. For those members who were prescribed opioids identified as potentially high-risk, the Program follows up with their providers to review their medical records and help to ensure their opioid prescriptions are clinically indicated.To align with recommendations in the 2016 CDC Guideline, the Pharmacy Quality Alliance , a natioThe intent of our study was to use the Program’s administrative  and medical and pharmaceutical claims data to characterize opioid prescriptions dispensed among Program members, and to estimate the PQA measures to explore areas that may benefit from improvements in opioid prescribing for members without a cancer diagnosis, sickle cell disease, or hospice care. The Program’s experience may help provide insights to other health programs in managing opioid prescribing.Administrative and claims data routinely collected by the Program were used for this analysis on Program members’ opioid prescriptions during 2013–2021. Member enrollment data include demographic information . Certification data include members’ Program-certified health conditions and the date of certification, which is generally when the member satisfies 9/11 exposure criteria and a Program-affiliated physician attests that those exposures were substantially likely to have been a significant factor in aggravating, contributing to, or causing the health condition . MedicalMembers enrolled for at least 10 months with the Program during 2013–2021 were included. This 10-month requirement was adopted because the PQA manual recommends individuals to be enrolled in a health care program for at least 10 months in a measurement year to be included in estimates of measures related to opioid prescription. This 10-month requirement also accounts for the time lag between Program enrollment and getting certified for a WTC condition (certification is required to receive Program covered condition-related treatment). For this population, opioid prescription prevalence from 2013 through 2021 was estimated for all members and different subgroups defined by age, sex, race/ethnicity, member type (responders or survivors), Program-certified conditions, with or without diagnostic/treatment-related claims, and with or without cancer , hospice care, surgery pain, acute pain, or chronic pain.ICD-9-CM codes 140–239 except 173), or ICD-10-CM codes C00-D50 (except C44) were used to identify members with WTC certifications for cancer and to identify medical claims with a cancer diagnosis. Members with surgical procedures, acute pain, chronic pain or sickle cell disease were identified as those with any claims that had a surgery CPT code, or an ICD-10-CM code for corresponding conditions listed by Mikosz, et al. 3, or ICD. MembersOpioid NDCs and their oral MME conventional factors provided by CDC  and PQA For each outpatient opioid prescription, it was assumed that use began on the dispense date and continued daily until the last day of supply. If a member had any day covered by more than one opioid prescription based on this assumption, those days were counted only once when calculating the length of opioid use, and the MED was the sum of MEDs of all prescriptions on that date.To understand the purpose of an opioid prescription, the written/dispense date was linked to the member’s medical claims and WTC condition certifications, and an indication was assigned to each prescription in the hierarchy of hospice/end of life care, cancer, surgery pain, acute pain, or chronic pain. The algorithms are shown in Fig. p value of 0.05 was used to determine whether a trend was statistically significant or not.Simple linear regression analyses were conducted to test the trend of the proportion of members with diagnostic or treatment-related claims who had opioid prescriptions for different purposes, with year as the independent variable and proportion as the dependent variable. A PQA measures on opioid prescriptions by year from 2013 to 2021 were estimated following the manual provided by PQA through licensing. Descriptions of the measures and methods are shown in Table A total of 108,285 members were enrolled for at least 10 months in the Program during 2013–2021 and included in the analysis. In this sample, 78.3% were males, 91.6% were 18–65 years old on 01/01/2013, 52.0% were white, 13.0% Hispanic and 8.9% black, and 73.5% were responders and 26.5% survivors  of the prescriptions dispensed had 1–7 days’ supply, 23%  had 8–29 days’ supply, 52%  had 30 days’ supply, and only 0.1% (n = 27) had > 30 days’ supply (data not shown). About 66%  of the prescriptions were dispensed on the same day when they were written, and another 24%  were dispensed within one week (Table n = 737) of members who were prescribed opioids had at least one high-dose prescription (≥ 90 MMEs/day), and those high-dose prescriptions accounted for 29.8%  of the total opioid prescriptions. Three quarters  of the prescriptions were short-acting opioids, and the rest were long-acting opioids. More than 20%  of members receiving opioid had “new” opioid prescriptions dispensed after more than 90 days of no known opioid prescriptions. Most opioid prescriptions were for cancer-related pain ; more than 10%  were for hospice or end-of-life care, 5%  for surgery pain, and 9%  for acute or chronic pain , or 21 opioid prescriptions dispensed per 100 Program members. About 25% (p = 0.024), then it decreased to 1.5% in 2021 (p = 0.003). A similar trend was observed in the proportion of members with outpatient opioid prescriptions for hospice/end of life care or cancer, with 0.5% in 2013, increasing to 1.6% in 2018 (p = 0.010), then decreased to 1.1% in 2021 (p = 0.070). The proportion of members with opioid prescription for surgery, acute or chronic pain decreased from 0.6% in 2013 to 0.23% in 2021 (p = 0.0005).A total of 70,721 members had diagnostic/treatment-related claims paid by the Program. Among these members, there was an increasing trend of the proportion of members with any opioid prescription, from 1.5% in 2013 to 2.5% in 2017 . The population of evaluation interest is defined as persons with ≥ 2 opioid prescriptions dispensed, ≥ 15 days' supply, and opioid episode (from the first date of supply of the first opioid prescription dispensed to the last date of supply of the last opioid prescription dispensed in a measurement year) ≥ 90 days in the year and ranged from 56 individuals in 2013 to 83 in 2018. OHD, the proportion who had opioid use at high dosage (≥ 90 MME/day), increased from 18% in 2013 to 27% in 2017, then decreased to 14% in 2020, and increased to 20% in 2021. OMP, the proportion who received opioid from multiple providers within 180 days were zero in all years, except 1.5% in 2014 and 2.4% in 2018. OHDMP, the proportion with opioid use at high dosage and from multiple providers, were zero in all years.Measure COB (concurrent use of opioids and benzodiazepines), percent of individuals with ≥ 2 opioid prescription dispenses and ≥ 15 days' supply who had COB for ≥ 30 cumulative days in the year, was approximately 30% from 2014 to 2018, then decreased to 16% in 2020 and increased to 24% in 2021. Measure AMO, the proportion of persons on long-term opioid therapy  who had no annual monitoring (drug test), was close to 100% in all years, indicating that drug testing among persons on LTOT was rare.A total of 3.7% of the Program members received Program-paid outpatient opioid prescriptions from 2013 to 2021, with an annual prescription rate from 0.6% to 1.1%. This is much lower than the reported annual United States population opioid prescription rate of about 20% each year from 2014 to 2016 . Also, tMembers with hospice care or surgery procedures had the highest opioid prescription rates, followed by members with cancer, and acute or chronic pain claims. The majority of the opioid prescriptions were dispensed for cancer, hospice, or end-of-life care. This is not surprising as opioids are widely used for treatment of severe pain from cancer or those in hospice care, and nearly 20% of the Program population had a cancer certification. This study showed that members with PTSD but no cancer certifications had a higher prevalence of opioid prescriptions, compared to those with other non-cancer WTC Health Program conditions. This is consistent with the findings of a previous study , which uAmong those with Program-paid diagnostic or treatment claims, opioid prescribing rates increased monotonically from 2013 to 2017, then decreased monotonically over the subsequent years. This trend was different from the national trend, which decreased from 21% in 2014 to 15% in 2018 , probablThe PQA opioid measures were developed so that claims data could be used to evaluate opioid prescription safety using the 2016 CDC Guideline recommendations, and excluded individuals with hospice care, cancer, or sickle cell diseases. The 2016 CDC Guideline recommended that when starting opioid therapy, clinicians should prescribe immediate-release opioids, the lowest effective dosage, and no more than the quantity needed . In thisThe 2016 CDC Guideline also recommended avoiding increasing the dosage to ≥ 90 MME/day without justification and avoiding COB prescribing whenever possible . CMS alsThe Program semiannually reviews concurrent prescriptions of high dose opioids (> 90 MMEs/day) and benzodiazepines. When reviewing the medical records of 31 members from 2017 and 2018 with opioid > 90 MMEs/day or with benzodiazepine concurrent use for non-cancer treatment, the Program found that such use was medically necessary in 61% of instances. Although opioid use by some non-cancer members appears inconsistent with CDC 2016 recommendations based on claims data, the related benefits might outweigh the risks.The 2016 CDC Guideline recommended urine drug testing  before starting opioid therapy and consider testing at least annually for persons who were prescribed opioids for chronic pain to assess for the prescribed opioid as well as for other controlled prescription drugs and illicit drugs. The PQA measure AMO, annual monitoring for persons on long-term opioid therapy, in this study showed that annual drug monitoring had been rarely conducted during 2013–2021. It is possible that drug tests were conducted for the members but not paid by the Program and thus not captured by this analysis. Furthermore, the updated 2022 CDC Guideline recommended that clinicians consider the benefits and risks of toxicology testing to assess for prescribed medications as well as other prescribed and nonprescribed controlled substances when prescribing opioids for subacute or chronic pain [There are several limitations with this study. First, electronic health records were not available for this analysis, making it challenging to identify the actual medical conditions related to opioid prescriptions. As a result, this study developed algorithms to estimate opioid prescription indicators, and 12% of opioid prescriptions during 2013–2022 had an unknown indicator. Second, only claims paid by the WTC Health Program were included. As such, data on the scope of opioid use and related services, such as annual urine drug tests, may be incomplete since claims paid by other insurance providers or by self-pay were not available. In addition, opioid prescriptions for conditions unrelated to WTC exposure (and thus not covered by the Program) were also not available. Third, chronic pain- and acute pain-related ICD-10-CM codes were abstracted from Mikosz et al. , and if Program-paid opioid prescription rates for all members and for those with non-cancer pains were low, and opioid prescriptions paid by the Program were primarily dispensed to members with cancer. Based on the trends of opioid prescription rates and practice, Program efforts to improve opioid prescribing coincided with improvements in outcomes. Nevertheless, potential areas for improvement were identified and continued surveillance is needed, such as urine drug tests during LTOT and prescribing opioids at high dose or concurrently with benzodiazepines to members without cancer.