Probe-based confocal laser endomicroscopy (pCLE), also known as optical biopsy, is a new endoscopic technique that provides real-time magnification of 1000 × microscopic tissue information to diagnose indeterminate biliary strictures. Tissue sampling by endoscopic retrograde cholangiopancreatography (ERCP) is routinely performed to evaluate indeterminate biliary strictures. To evaluate the accuracy of pCLE and tissue sampling by ERCP in the diagnosis of indeterminate biliary strictures, 18 articles were included from 2008 to 2021 through Embase, PubMed, Web of Science, and Cochrane library databases. The summary estimates for the pCLE diagnosis of indeterminate biliary strictures were: sensitivity 0.88 , 0.84–0.91); specificity 0.79 (95% CI 0.74–0.83); and Diagnostic Odds Ratio (DOR) 24.63 (95% CI 15.76–38.48). The summary estimates for tissue sampling by ERCP diagnosis for indeterminate biliary strictures were: sensitivity 0.54 (95% CI 0.49–0.59); specificity 0.96 (95% CI 0.94–0.98); and DOR 11.31 (95% CI 3.90–32.82). The area under the sROC curve of pCLE diagnosis of indeterminate biliary strictures is 0.90 higher than 0.65 of tissue sampling by ERCP. The pCLE is a better approach than tissue sampling by ERCP for the diagnosis of indeterminate biliary strictures by providing real-time microscopic images of the bile ducts. Cholangiocarcinoma is the most common malignant tumor causing malignant strictures of the bile duct, and most tumors grow along the bile duct wall instead of radially to form masses2. Therefore, cholangiocarcinoma is usually at an advanced stage when diagnosed, and the survival rate of patients is low. At the same time, as many as 15% of the suspected hilar cholangiocarcinoma were confirmed to be benign by surgical histopathology3. Thus, an urgent clinical need exists to improve the diagnostic accuracy of indeterminate biliary strictures so that malignant tumors can be detected as early as possible, improving the prognosis for patients and reducing unnecessary surgeries.Biliary strictures are a common disease of the biliary system. The treatment and prognosis of benign and malignant biliary strictures vary greatly, so it is especially important to distinguish between benign and malignant biliary strictures. Although there are various advanced imaging techniques such as peroral choledochoscopy (POCS) and biopsy techniques such as endoscopic ultrasonography guided fine needle aspiration (EUS-FNA) and endoscopic retrograde cholangiopancreatography (ERCP) biopsy and cytology brush, accurate diagnosis of indeterminate biliary strictures still has challenge4. The pCLE has been used to assess the histopathology of the gastrointestinal tract in vivo6. The accuracy of pCLE in the diagnosis of gastrointestinal tumors has been demonstrated in previous studies8. To date, pCLE has been studied in a variety of digestive diseases, including ulcerative colitis, colorectal polyps, pancreatic disease, and Barrett's esophagus12. In the same situation, pCLE can also enter the intrahepatic and extrahepatic bile ducts to provide real-time enlarged tissue information for bile duct lesions. The pCLE thus improves the accuracy of diagnosing indeterminate biliary strictures. With the development of pCLE technology and the improvement of the diagnostic classification of indeterminate biliary strictures, a series of studies have been published on the diagnosis of indeterminate biliary strictures in pCLE30. The present study will perform a meta-analysis to assess the accuracy of pCLE in diagnosing indeterminate biliary strictures and to compare the accuracy of pCLE and ERCP with brush cytology and intraductal biopsy in diagnosing indeterminate biliary strictures.Probe-based confocal laser endomicroscopy (pCLE) is a new diagnostic endoscopy approach that provides real-time microscopic tissue information. The pCLE enables low-energy laser light to be focused and illuminated through its inner small hole to produce a tissue image, which improves the accuracy of targeted biopsy and realizes instant optical biopsy30. In total, there were 750 lesions reported by 18 studies, 12 prospective and 6 retrospectives in Table Searching using the preset search strategy resulted in 1286 records in Fig. 2 = 0.0%, I2 = 33.2%). An indicator of test accuracy, the Diagnostic Odds Ratio (DOR), combines sensitivity and specificity data into one. DOR equals the value of positive likelihood ratio (PLR) divided by the value of negative likelihood ratio (NLR). A higher value indicates that the test has better diagnostic capabilities. The combined PLR, NLR, and DOR were 3.67 (95% CI 3.02–4.45), 0.18 . The combined PLR, NLR, and DOR were 5.84 (95% CI 2.21–15.43), 0.57  in Fig. A total of 16 studies including 784 lesions evaluated the accuracy of tissue sampling by ERCP in the diagnosis of indeterminate biliary strictures, among which intraductal biopsy was 9 articles, brush cytology was 4 articles, brush cytology and intraductal biopsy were 3 articles. Figure We conducted subgroups based on the characteristics of pCLE to diagnose indeterminate biliary strictures. To understand the pCLE diagnosis of indeterminate biliary strictures between different subgroups, we conducted a subgroup analysis and compared whether there are significant statistical differences. The subgroup analysis is shown in Table 15. The results of the sensitivity analysis showed that the diagnostic efficacy of pCLE for indeterminate biliary strictures remained stable at high levels.Deek’s funnel plot was used to analyze the potential publication bias of the meta-analysis. Deek’s test showed a value of 0.47 (95% CI − 4.95 to 10.32), and these suggested no possibility of publication bias in Fig. The identification of benign and malignant biliary strictures is critical to patient treatment and prognosis. Early diagnosis of benign biliary strictures can reduce unnecessary surgery and costs, and early detection of bile duct malignancies allows for early surgical intervention to improve surgical cure rates and improve patient prognosis. However, due to the particularity of the anatomical structure of the bile duct and the way of tumor growth, the current diagnostic methods still have insufficient sensitivity and accuracy. Therefore, accurate diagnosis of indeterminate biliary strictures is still a major clinical problem.31. However, the shortcoming of this method is that it is limited to intrahepatic biliary strictures or proximal extrahepatic biliary strictures. Single-person operation of POCS allows insertion of the extrahepatic bile duct for direct visualization of the lesion and obtaining a targeted biopsy. In a meta-analysis of single-operator POCS including 335 patients, the sensitivity and specificity of visual representation for diagnosing indeterminate biliary strictures were 90% and 87%, respectively, whereas the sensitivity and specificity of single-operator POCS biopsy for diagnosing indeterminate biliary strictures were 69% and 98%, respectively32. However, differences in the interobserver agreement were found in various studies, which may be due to the lack of a standard consensus for the diagnosis of indeterminate biliary strictures in POCS34. In addition, POCS may have difficulty accessing the intrahepatic bile duct and obtaining sufficient tissue samples to confirm a malignancy, but pCLE allows adequate visualization of intrahepatic bile duct strictures because of its small diameter14. pCLE has a high accuracy rate in both intrahepatic and extrahepatic biliary strictures due to its fine outer diameter and access to the intrahepatic bile ducts, and there is no statistical difference between intrahepatic (100%) and extrahepatic biliary strictures (86%)17. In addition, for indeterminate biliary strictures where the mass is not visible on the image, it is difficult to obtain the specimen by biopsy, but pCLE can obtain a high diagnostic accuracy of 79% and an negative predictive value (NPV) of 100%30. pCLE is a new imaging technique capable of providing real-time microscopic tissue information in vivo to obtain images of the GI epithelium and sub epithelium at 1000 magnification during endoscopy. Real-time high-resolution histological diagnosis of mucosa and submucosa tissue structure can achieve the purpose of real-time optical biopsy35. As an emerging endoscopic technique, pCLE is gradually spreading worldwide, and the recent American Society for Gastrointestinal Endoscopy guidelines on the management of biliary tract tumors mention pCLE as a useful alternative to existing diagnostic workup36. In many studies, pCLE has high sensitivity and accuracy for the diagnosis of indeterminate biliary strictures.The diagnosis of biliary strictures is a complex and multiple approach. EUS-FNA is a method for obtaining biopsy tissue from biliary lesions and has 87% sensitivity and 87% accuracy for the diagnosis of indeterminate biliary strictures in a prospective single-center trial37. Our study reported a pooled DOR of 24.63, which also illustrates the high overall diagnostic accuracy. Because PLR and NLR are more easily applied in the clinical setting, they were used as an indicator to assess diagnostic accuracy in the current meta-analysis. A PLR value of 3.67 indicated that patients with malignant biliary strictures were approximately three times more likely to be positive compared to patients without malignant biliary strictures. In contrast, the NLR is found to be 0.18, which means that if the pCLE test is negative, the patient has about a 18% chance of having malignant biliary strictures. Therefore, based on current data, PLR and NLR may be used as valuable tools for diagnosing uncertain biliary strictures in pCLE in the future.The meta-analysis showed that pCLE as a diagnostic tool for indeterminate biliary strictures had a pooled sensitivity of 0.88 and a pooled specificity of 0.79. We reported that the index Q value of sROC was 0.84 and AUC was 0.90, indicating that the overall diagnostic accuracy is very high. The range of DOR is from 0 to infinity, which can be used as an overall assessment of the accuracy of diagnosis. When the DOR value is 1.0, the test is an invalid diagnostic method38. The image features of Miami classification are shown in Table 28. The Paris classification is based on the Miami classification with the addition of 4 image features of benign inflammatory stenosis as shown in Table 29. Similar results were obtained by using the Paris classification in the subgroup of the present study, improving the specificity of the diagnosis from 77.6 to 85.0%. In future studies, each criterion of the Paris classification scheme for diagnosis of indeterminate biliary strictures needs to be evaluated and appropriate thresholds need to be set. In addition, the thickness of the reticular structure in the Paris classification also needs to be quantified.We noted differences in diagnostic accuracy between subgroups. The diagnosis of pCLE for indeterminate biliary strictures has a Miami classification and a Paris classification, and we compared the diagnostic accuracy of the Miami and Paris classifications in a subgroup analysis. The Miami classification of pCLE diagnostic indeterminate biliary strictures was developed in 2012 and tested by a multicenter study17. There are similar results in the present study, where the sensitivity and specificity of pCLE for the diagnosis of indeterminate biliary strictures in the cholangioscopy subgroup were higher at 92.3% and 93.5%, respectively. Additionally, pCLE operators' skills should be improved through training, as highly skilled operators are able to achieve high accuracy39.Because pCLE can reach the biliary strictures by a catheter or a cholangioscopy, we compared the diagnostic accuracy of the two approaches in a subgroup analysis. The pCLE may reach the biliary strictures more accurately by direct visualization with a cholangioscopy, while the pCLE can only reach the biliary strictures by catheter under fluoroscopic guidance. The cholangioscopy can adjust the angle in the bile duct to make the mini probe to the site of interest diagnose indeterminate biliary strictures in pCLE and obtain a high accuracy of 93.3%23. Good image quality achieved 100% accuracy while poor image accuracy was only 79%, indicating that good image quality can significantly improve accuracy28. In addition, the Gastroflex probe had a larger outer diameter but no reduction in cannulation rate19. Usually, the larger the outer diameter of the probe, the lower the rate of cannulation. In Shieh's study, the high rate of cannulation with the Gastroflex probe may be due to the operator's being a skilled endoscopist19. More accurate diagnoses were obtained in some studies using the Gastroflex probe, but the small number of Gastroflex probes used in these studies will require further confirmation in studies with large sample sizes. In the subgroup analysis of this study, the sensitivity and specificity of pCLE in the subgroup of using the GastroFlex mini probe for the diagnosis of indeterminate biliary strictures reached 93.3% and 80.6%. However, the diagnostic performance of the GastroFlex mini-probe will need to be further clarified by multi-center prospective studies in the future.In most pCLE diagnostic studies on indeterminate biliary strictures, the CholangioFlex mini probe was used, which has an outer diameter of less than 1 mm and a resolution of 3.5 mm. However, some studies have used the GastroFlex mini probe, which increases the number of optical fibers and has better resolution and image quality20. Based on the pattern of tumor growth, cholangiocarcinoma is classified into three subtypes: mass-forming, periductal infiltration, and intraductal growth, with the mass-forming type being the most frequent and the intraductal growth type being the least common. There is a lack of study of pCLE diagnosing indeterminate biliary strictures based on tumour growth patterns, and further study is needed to determine if pCLE is accurate in diagnosing indeterminate biliary strictures based on different tumour growth patterns. In sum, although the data on the accuracy of CLE was promising, a high level of operator expertise and standardized training is required. There was no data on operator learning curves, although one study showed that inexperienced operators achieved 83% accuracy after three weeks of professional training36. Therefore, the training of endoscopists in pCLE has been strengthened to improve the reliability of pCLE imaging and the consistency of image interpretation in order to better apply this new technology.A subgroup analysis was performed to determine the diagnostic ability of pCLE for different stricture locations. The sensitivity and specificity of pCLE in diagnosing indeterminate biliary strictures were high for both extrahepatic and common bile duct strictures. The diagnosis of indeterminate biliary strictures in the intrahepatic bile ducts was of concern. However, due to the lack of data on intrahepatic biliary strictures, meta-analysis was not performed, and more studies are needed in the future to determine the accuracy of pCLE in the diagnosis of intrahepatic biliary strictures. In addition, if some objective diagnostic methods such as artificial intelligence and signal-to-noise ratio are used, it may improve the sensitivity and specificity of pCLE to diagnose indeterminate biliary strictures40. We performed a meta-analysis of brush cytology and intraductal biopsies by ERCP. The meta-analysis showed that ERCP with brush cytology and intraductal biopsy as a diagnostic tool for indeterminate biliary strictures has a pooled sensitivity of 0.54 and a good specificity of 0.96. We reported that the index Q value of sROC is 0.61 and AUC is 0.65, indicating that the overall diagnostic accuracy is poor. There is no overlap between the 95% AUC confidence intervals of pCLE and brush cytology and intraductal biopsy in the diagnosis of indeterminate biliary strictures, indicating that there are significant differences between the two groups. The sensitivity of 0.54 for ERCP with brush cytology and intraductal biopsy is indeed disappointing. The reasons for the low sensitivity could be tumor-associated fibrosis, submucosal spread, or bile duct compression by external lesions. Desmoplastic tumors have a relatively small number of cells and are very firm, making sampling very difficult. Another reason for the poor diagnosis of ERCP with brush cytology and intraductal biopsy is the random sampling of tissues. The pCLE also allows for a more accurate diagnosis by locating different biliary strictures in real time. ERCP with brush cytology and intraductal biopsy combined with pCLE can obtain a more accurate diagnosis30.Brush cytology and intraductal biopsy by ERCP are routinely performed to evaluate indeterminate biliary stricturesThis study has several limitations. First, as most of the included studies were retrospective studies, and there was potential selective bias needing further prospective studies to confirm. Secondly, the poor quality of some of the included studies may affect the results of the meta-analysis. Third, some patients in some of the included studies were followed up rather than pathology as the gold standard.In conclusion, pCLE is a reliable and accurate method for diagnosing indeterminate biliary strictures, especially when reaching the biliary strictures by cholangioscopy. However, pCLE is expensive compared to common ERCP, and future studies are needed to confirm the cost-effectiveness. With the development of diagnostic classifications and advances in technology, pCLE will improve the accuracy of diagnosing indeterminate biliary strictures. The pCLE has the potential to overcome the limitations inherent in tissue sampling by ERCP by providing real-time microscopic images of the bile ducts to make an accurate diagnosis of indeterminate biliary strictures.The study was conducted in accordance with the PRISMA  guidelines. The PRISMA checklist for this study was in the The studies included in the meta-analysis met the following criteria: 1) The included studies were all published proposing the diagnosis of pCLE on indeterminate biliary strictures; (2) All studies included for the diagnosis of indeterminate biliary strictures had pathology and follow-up as the gold standard for final diagnosis; (3) The number of true positives (TP), false positives (FP), true negatives (TN) and false negatives (FN) can be obtained directly or indirectly in the study. Exclusion criteria: (1) Conference abstracts, letters to editors, reviews, case reports, comments, editorials; (2) pCLE diagnosis of pancreaticobiliary strictures; (3) animal or pediatric studies; (4) pCLE diagnosis of ampullary strictures; (5) artificial intelligence-assisted pCLE (Supplementary Information  The inclData extraction for each selected study was performed independently by two reviewers (Mi and Wang), and disagreements were resolved by consensus. The extracted content included the general information of the study, such as year of publication, first author, age, sex, country, diagnostic classification, mode of entry into the bile duct, follow-up time, study design, number of research centers, type of microprobe, and presence of primary sclerosing cholangitis. To assess the quality and potential bias of the studies, QUADAS was applied to evaluate the quality of the included studies. Each eligible study was independently evaluated by two authors (Ma and Zhao), and differences were resolved through discussions with the third author (Mi).Q test was used to assess heterogeneity between studies, with heterogeneity between studies rather than chance expressed as a percentage (I2). More than 50% of the I2 were considered to be significantly heterogeneous. The sROC using the method of Moses and colleagues is a comprehensive indicator to evaluate the accuracy of diagnosis. sROC is a graph of the functional relationship between the true positive rate and the false positive rate based on a linear regression model. Overlapping 95% CI for the given values were determined as qualitative methods indicating no significant differences. To explore the accuracy of pCLE diagnostic indeterminate biliary strictures in different subgroups, a subgroup analysis was performed according to the following: year of publication, first author, country, diagnostic classification, mode of entry into the bile duct, follow-up time, study design, number of research centers, type of microprobe, stricture locations and presence of primary sclerosing cholangitis. The present meta-analysis used Deek’s test and funnel plot analysis for publication bias.Estimates of diagnostic accuracy for pooled studies, including sensitivity, specificity, PLR, NLR, and DOR were calculated. The primary outcome of the present meta-analysis was to assess the pooled results of the accuracy of the pCLE diagnostic indeterminate biliary strictures, with the secondary aim of assessing the pooled results of the accuracy of the tissue sampling by ERCP diagnostic indeterminate biliary strictures. The pooled results used a random effects model, as the random effects model (DerSimonian–Laird method) provides more conservative results. In the present meta-analysis, the Cochrane Meta-DiSc software version 1.4 was used to test the diagnostic accuracy, including sensitivity, specificity, PLR, NLR, DOR, sROC, and subgroups. The Spearman’s correlation coefficient was used to assess the threshold effect using Meta-DiSc software version 1.4. STATA software version 16.0 was used to analyze publication bias. The quality of the included studies was assessed using Review Manager5.3 software. To assess the robustness of the synthesized results, sensitivity analyses will be conducted. The significance level was measured at P < 0.05.Supplementary Information 1.Supplementary Information 2.Supplementary Information 3.Supplementary Information 4.


The name of the measure  was inadvertently deleted. It should read as follows: “At 3 months, participants in the commercial program group had significantly greater improvements than those in the DIY group in quality of life .” This article has been corrected.1In the Original Investigation titled “Efficacy of a Commercial Weight Management Program Compared With a Do-It-Yourself Approach: A Randomized Clinical Trial,”


The transdisciplinary Innovations in Aging Graduate Fellowship program brought together graduate students from medicine, nursing, pharmacy, and social work. As a result, fellows entered the program with a range of exposure as to how aging impacts the body, social interactions, health needs, and society. Therefore, onboarding modules were utilized to provide an integrated perspective on aging. Topics for these onboarding modules included ageism, dimensions of wellness, and health and well-being in later life. Through these modules, fellows were able to customize their learning to meet their unique educational needs and goals related to their involvement in the fellowship program. In this session, we will share development of the fellowship program and highlight factors, such as the onboarding modules, that led to the program’s success.


Introduction: The objective of this study was to review internal medicine residency program websites in the United States based on their published support for wellness, diversity, equity, and inclusion concepts. Inclusion of wellness, diversity, equity, and inclusion on program websites can serve as critical student benchmarks, and it may be paramount to optimize residency program websites accordingly.Methods: This is a cross-sectional study of the websites of 597 internal medicine residency programs accredited by the Accreditation Council for Graduate Medical Education between March 25 and April 25, 2022. The websites were assessed based on 22 characteristics consisting of wellness verbiage, gender and underrepresented in medicine evaluation of faculty and residents, and diversity, equity, and inclusion-related semantics. Website photos were used to assess ethnic/sex representation. These attributes were devised by two sequentially set up focus groups consisting of 49 racially, ethnically, and gender-diverse medical students.Results: A total of 579 internal medicine programs were reviewed. Only 239 (41%) had a dedicated page for resident wellness activities and efforts, while 134 (19%) had no mention of the concept throughout their web pages. Similarly, only 136 (23%) had a dedicated wellness officer, whether faculty or resident, who was focused on departmental interests. Gender diversity could be determined in 445 (77%) and 459 (79%) websites for faculty and residents, respectively. Underrepresented in medicine faculty and residents was noted in 293 (51%) and 393 (68%) of websites, respectively. A diversity, equity, and inclusion section was present in 172 (30%) of programs, with 93 (16%) having an assigned faculty or resident. Chairpersons or program directors stressed diversity, equity, and inclusion in up to 456 (79%) of the websites, with 181 (31%) having program mission statements or goals that include diversity, equity, and inclusion verbiage.Conclusion: A deficit of various essential wellness, diversity, equity, and inclusion attributes persists across internal medicine residency websites. Residency programs would benefit from optimizing their websites to attract more diverse applicants. Residency selection among medical students applying to internal medicine (IM) programs is a daunting task, as students must gauge the alignment of their values and interests against those of the programs they choose to apply to. There are currently 579 IM programs across the United States for rising medical students to select from, and the predominant method is obtaining information from program-specific websites . This meCurrent literature describes many studies that target wellness and other diversity, equity, and inclusion (DEI) in residency programs and across various specialties -13. HowePerceived goodness of fit and work/life balance rank among the most critical factors applicants consider when choosing a residency program -16. FurtThe objective of this study was to describe how IM programs across the United States focus on DEI and wellness based on a structured approach devised and implemented by prospective residency applicants.This is an IM-focused secondary analysis of a cross-sectional qualitative study of ACGME-approved residency program websites across the United States between March 25 and April 25, 2022. The website assessment was based on a compilation of 22 attributes devised by two focus groups. The first focus group consisted of nine medical students who developed the criteria based on research and relevant online commentary. The second group comprised 40 voluntarily recruited students from a large Midwestern medical school. The second focus group piloted and refined the questions. The questions formed all focused on the presence of these attributes on program websites. The attributes assessed fit into three categories.The first category is related to the concept of wellness. Wellness concepts assessed on program sites include mention of wellness in the department chairperson/program director's welcome letter or video, presence of a wellness page, presence of a wellness champion, and presence of group wellness activities among the residents. A wellness champion was defined as an appointed or elected individual (either faculty/resident) who was assigned to advocate for resident wellness. Group wellness activities were determined via descriptions of events or photos of events found on program websites.The second category explored on program websites related to identity-related characteristics and concepts. This included any mention/emphasis of lesbian, gay, bisexual, transgender, queer/questioning, intersex, and asexual (LGBTQIA), use of inclusive pronouns, and gender diversity among faculty and residents. Mention of LGBTQIA was defined as any written or verbal reference to LGBTQIA initiatives, while emphasis was determined if an entire website section was dedicated to LGBTQIA causes. Gender diversity was determined by the medical student researcher's photo-based self-perceived identification of at least one member of a differing gender compared to the majority  within the faculty members and resident class.Lastly, the concept of general equity and diversity-related characteristics was explored on program websites. Concepts assessed here include the presence of a dedicated DEI section, use of DEI verbiage in the chairperson/program director's letter or video, dedicated DEI officer or resident, racial and ethnic diversity in both residents and faculty, residents sharing of perspectives on DEI aspects of the program, resident involvement with underserved/lower socioeconomic communities, explicit incorporation of DEI in curriculum/curriculum objectives, DEI-focused research initiatives/projects displayed on websites, DEI verbiage in the mission statement or program goals, explicit mention of the holistic application review process, explicit mention of microaggression/implicit training bias for faculty/staff and residents, and explicit recognition of religious/cultural holidays with allocated resident time off. Racial and ethnic diversity was defined as the medical student researcher's self-perceived identification of at least one African American/Black, Hispanic/Latino, American Indian/Alaska Native, or Native Hawaiian/Pacific Islander faculty member and resident among their colleagues based on publicly accessible website photographic representation. Resident sharing of DEI perspectives was defined as direct quotes or videos of residents using DEI verbiage.Thereafter, a cohort of racially, ethnically, and gender-diverse medical student researchers performed the website assessment based on the above-mentioned attributes to ensure representative benchmarking. Data were collected from the assessment and tabulated according to the respective DEI or wellness category. Analysis of the trends in diversity and wellness were computed using descriptive statistics.A comprehensive list of accredited IM programs was obtained from the ACGME's website. To include programs in the assessment, each program had to be listed in the active ACGME listings, have a program identifiable online under the same name as in the list and utilizing the referenced ACGME program number, and have an accessible website with online content at the time of the study. Websites were visited three times at one-week intervals if not accessible during the initial search. Websites were excluded if their name correlated with more than one program, did not have full matriculation of all classes in each residency, and were not accessible upon searching.A total of 579 residency programs  were analyzed based on the content of their websites. Up to 470 81%) of the websites mentioned the concept of wellness, with 239 (41%) having a dedicated page. Only 136 (23%) had a wellness champion (resident or faculty), and 36 (6%) listed cultural competency as a component of their resident professional development events (Table % of the Identity initiatives were limited to 125 (22%) mentioning the support of LGBTQIA personnel and 12 (2%) utilizing inclusive pronouns on their websites. Gender diversity was clearly noted in 445 (77%) of websites in relation to faculty and 459 (79%) in residents. A total of 105 (18%) and 113 (20%) program websites did not clearly determine faculty and resident gender diversity respectfully (Table Residency-related DEI focus was noted in 172 30%) of websites through dedicated web pages and 456 (79%) through leadership verbiage, whether in video or written format. DEI verbiage was present in 181 (31%) of mission statements or program goals. Additionally, 99 (17%) of programs explicitly listed the incorporation of DEI in their curriculum or its objectives, and 93 (16%) appointed dedicated DEI champions to bolster departmental efforts  when assessing the websites and the reliance on website content that may be outdated and/or limited by standardized institutional website criteria. Similarly, the students' perceptions of racial and ethnic representation are subjective as they rely on photographs, and this may not reflect the actual program's diversity. Additionally, considering one person per group as sufficiently representative of gender or racial/ethnic diversity is an understatement, as representation needs to be aligned with national or community-based percentages. Strengths of the study include the comprehensive analysis platform used that was devised by a medical student. An additional strength is the data collection being performed by students interested in the specialty and intending to apply within the year.Residency program websites for IM need to emphasize wellness and DEI further. Websites can improve by having dedicated wellness and DEI pages. Support for LGBTQIA healthcare professionals could be better represented on websites. Additional studies are needed to explore how each attribute affects a medical student's decision-making. Improvements may help attract a diverse applicant pool while ensuring the resilience of the trainees.


Despite the widespread adoption of interprofessional simulation-based education (IPSE) in healthcare as a means to optimize interprofessional teamwork, data suggest that IPSE may not achieve these intended goals due to a gap between the ideals and the realities of implementation. We conducted a qualitative case study that used the framework method to understand what and how core principles from guidelines for interprofessional education (IPE) and simulation-based education (SBE) were implemented in existing in situ IPSE programs. We observed simulation sessions and interviewed facilitators and directors at seven programs. We found considerable variability in how IPSE programs apply and implement core principles derived from IPE and SBE guidelines with some principles applied by most programs  and others rarely applied . Through interviews we identified that buy-in, resources, lack of outcome measures, and power discrepancies influenced the extent to which principles were applied.To achieve IPSE’s intended goals of optimizing interprofessional teamwork, programs should transition from designing for the ideal of IPSE to realities of IPSE implementation.The online version contains supplementary material available at 10.1186/s12909-022-03370-2. Interprofessional Simulation Based education (IPSE), is a form of interprofessional education (IPE), where students from two or more professions learn with, from and about each other, during simulated patient care scenarios . IPSE isStudies of IPE and simulation-based education (SBE) offer some insight into potential shortcomings of IPSE, largely related to implementation. A rich literature describes the challenges associated with IPE, including professional silos, power differentials and hierarchy –8. SBE cTaken together, IPSE is a highly complex modality requiring significant resources, yet little is known about the actual implementation of IPSE programs. As a first step to creating more effective IPSE, we conducted the current study to examine features of in situ IPSE programs across multiple disciplines and institutions. We created a framework based on existing guidelines for IPE and SBE to explore the extent to which programs abided by such guidelines and what challenges exist. Our findings can provide insight into factors that may contribute to the implementation success of IPSE, and inform strategies to improve IPSE in ways that can help achieve the goals of improved teamwork and patient care.We used case study methodology  to examiWe included seven IPSE programs as cases, using the following purposive sampling strategy. First, through the University of California San Francisco (UCSF) Kanbar Center for Simulation contact list, we identified eight in situ IPSE programs in Northern California and reached out to leaders from these programs to inquire whether they defined their programs as “interprofessional”. We then observed one session in each program to verify the interprofessional nature, after which we excluded one program because only one profession participated. The final seven programs represented different contexts. They included five hospitals  across five specialties . These programs varied in number of professions involved, participants’ level of training, acuity of scenarios, and profession(s) within which the IPSE program resides Table . We assi Three investigators  identified frequently cited evidence-based guidelines. These included guidelines for IPE developed by the Centre for the Advancement of Interprofessional Education in the UK  and the Four investigators  performed direct observations of IPSE sessions. At least two investigators observed each program to ensure diversity of viewpoints and to account for different degrees of familiarity with the setting. Three investigators  conducted and audio-recorded semi-structured interviews with IPSE session facilitators and program developers. The UCSF and UC Davis Institutional Review Boards reviewed the study and determined it to be exempt. All methods were performed in accordance with the relevant institutional ethical guidelines and regulations for exempt research, including the requirement for verbal versus written consent. Per UCSF and UC Davis Institutional Review Boards’ policy on exempt research, researchers obtained verbal consent from all participants at the beginning of each interview.We created an observation guide based on the 12 IPSE principles we synthesized from the literature. We anticipated that three of the 12 principles would not be observable during the simulation sessions: “program evaluation”, “train facilitators”, and “institutional support”. We therefore omitted these from the observation guide and relied on interview data to obtain information about these three principles. One investigator (MJ) pilot tested during the first 3 observations and met with one other investigator (B’OB) to review the experience and the notes that were taken, leading to modifications to increase ease of use and include space for reflexivity on the observation guide.After we completed all observations, we developed a semi-structured interview guide  to analyze the data. Transcribed data from all sources were de-identified. Three investigators  coded the data in an iterative manner. They coded each transcript independently and regularly met to reconcile any differences in opinion. We then used the framework to organize the data by cases (IPSE programs) and codes (IPSE principles) in a matrix. We collated excerpts from each source of data (observations and interviews) for a given program and given principle. Two authors (NB and MJ) separately reviewed these collated excerpts and assigned a designation of fully present, partially present and absent. We then met to reconcile differences in our designation. While we primarily used a deductive approach utilizing the predetermined list of codes (the 12 principles), we remained open to other insights gathered from our observations and interviews to ensure we captured all elements and perceptions of IPSE. In particular, we developed more codes to capture affordances and barriers to applying the principles during simulation sessions or as part of the programs.The investigator team consisted of two medical education qualitative researchers  and four physician educational researchers from critical care clinical backgrounds . The four physician investigators also functioned as facilitators in two of the IPSE programs selected as cases for this study. We ensured that members of the investigator team who were not directly involved with these IPSE programs conducted the observations and interviews. As part of the observation guide, investigators reflected on their presence as an observer, including any interactions with participants and feelings experienced while observing.Between August 2016 and August 2017, we observed three simulation sessions for each program, for a total of 21 sessions (30 h). Between March 2019 and September 2019, we interviewed two facilitators and/or program developers from each program. At Program E, we interviewed one program developer because this program shut down in the period between observations and interviews. Altogether, we interviewed 13 program developers and facilitators. All programs were held in situ and structured the simulations around patient care scenarios that constituted clinical emergencies with an aim to allow team members to practice patient management and interprofessional teamwork in the context of the simulation setting.Table Interviewees often emphasized the distinction between what they considered to be an ideal for IPSE, with the full application of the principles, and what they were able to do in their programs. An interviewee compared how facilitators in the program would implement “equitable distribution” in an ideal situation with what tended to happen during simulation sessions:“So, in the ideal world, there’s co-facilitation between the physician and the nurse. I think every single session that I’ve been at, the physician typically takes, opens up the conversation. But, ideally, the nurse facilitators actually take on a big piece.” .For programs B and F, we identified a lack of congruence between observation and interview data. In our observations, some principles were not fully applied . However, interviewees reported aiming to apply these principles in their programs.Data from interviews with program facilitators and developers helped us understand some of the affordances that supported sustainable IPSE programs as well as some of the barriers encountered. In addition, they helped us understand why some principles were more easily applicable than others in some programs. We describe these facilitators and barriers below.Interviewees emphasized that getting people at all levels of training and from all professional backgrounds to believe in the value of the IPSE sessions was an important factor in the success of programs. This buy-in needed to come from participants, facilitators, and institutions. Buy-in had the potential to grow over time, as people’s experiences and interactions with the IPSE programs increased. Institutions’ buy-in led to the allotment of more resources, including money, space and time, thus increasing “institutional support”.“Once [participants are] there…they’re very engaged…that wasn’t always the case. When we first started this out they would all sort of stand against the wall and be like, ‘I’m not doing anything, this is scary’ … But, that has changed. People have really started to realize, ‘this is important and I can learn something here and I want to participate.’” .Interviewees discussed how choosing the right type and level of fidelity, or realism, was important to achieve participant buy-in. In particular, they noted that “sociological fidelity” – the extent to which the simulation mimics how people in real life interact  – leads “If the monitor doesn’t look realistic people really lose their ability to understand what’s going on. So we work very hard to make sure that our monitors are in place of the actual patient monitor that would be there, that they look, in terms of color and sound, as realistic as possible, that the equipment that they use is all real equipment, kept in the right location…that adds to the learner’s perspective of realism in ways that are more meaningful.” .Interviewees cited resources such as money, time and physical space as important to the success of IPSE programs. In addition to increasing buy-in, these resources enabled the application of principles such as “program evaluation”, “interprofessional competency-based assessment,” and “repeated and distributed practice.”“I think to do it more frequently would probably require additional support. For now, I think we are able to maintain what we have… I think there is interest from everyone, and most people who come say that we should do it more frequently. I think the challenges are to try to schedule a time that’s convenient for everyone.” .“That funding kind of waxes and wanes, so right now we don’t have that much funding… So right now we’re in a little bit of a coasting phase where we’re just keeping the sims going, but we’re not really trying to improve the program or make any curricular changes. But we were able to do quite a bit of curriculum development and quality improvement within the simulation program. Previously over the last like two to three years where we had a half time patient safety coordinator who was really instrumental in that.” .As these quotes show, more resources potentially enabled facilitators to run more sessions, thus allowing participants to attend more simulation sessions over time (“repeated and distributed practice”). More resources also enabled programs such as Program C to hire someone to evaluate teams during simulated sessions  and to analyze the impact of the program  with the goal of improving the usefulness of the program. Yet, we found that most programs operated with limited resources and sometimes solely depended on the commitment of facilitators.“It’s one of the things people find a lot of value in, at least by word of mouth. It was something that we definitely wanted to continue. The residents […], I think they find usefulness in it and we did too so we kept it on the schedule because it’s pretty important. […] If we weren’t to do it [run the program], I don’t know that anyone would put up much of a fuss but it’s something that people find a lot of value in. But we do have a lot of ownership of it to make it actually happen.” .In addition to limited resources, interviewees mentioned that their limited knowledge of instruments to assess team performance in simulation acted as a barrier to applying the principle of “interprofessional competency-based assessment.”“I don’t think that we’ve had a good tool. And, then it’s also simply bandwidth; who’s going to do it, how do you record it, what do you do with the information, what’s really the purpose of the assessment? … I haven’t come up with a non-labor-intensive way to do it and we’re already so crunched right, for time… So, how do we fit it all in and then, what’s the cost benefit analysis of doing an assessment.” .Furthermore, interviewees viewed the small-scale nature of their program as a barrier to undertaking “program evaluation.” All the programs involved in our study focused on a small number of professions at one department in a hospital, which limited the number of sessions and of participants per year.“I don’t think there is enough N for that [evaluating the program], and then, additionally, to attribute it to a single course versus other medical center initiatives, I think, would be also difficult. I suppose you could look at codes that happened prior to 2010 and codes that have happened subsequently, after that, but we don’t have any of that information currently.” .“[F]rom the standpoint of having the desired effect and again, our numbers aren’t big enough to measure impact but there’s other literature out there like from larger systems like in Massachusetts, they pulled three or four different birthing hospitals that started simulation programs and it did show an improvement in outcomes.” .The last major factor that we identified as influencing the application of principles in IPSE programs was power. Interviewees described how multiple forms of power discrepancies influenced the programs and sometimes prevented the full application of the principles “attention to differences and hierarchy,” “equitable distribution,” and “psychological safety”. “I’ve never heard a nurse participant speak up and give feedback about the sim when a trauma attending is there or when trauma nursing leadership is there. The nurses are very quiet. I don’t hear interns asking questions. It seems a lot more constrained when there are high level administrators there.” .Experience also created power discrepancies between simulation participants, preventing participants from sharing feedback to people whom they considered have more experience:“And so I think it’s definitely awkward to critique one of your colleagues that’s an experienced ICU nurse as well. And most of them have more ICU experience than I do, or a couple of them do.” . Interviewees also noted that power discrepancies between professional groups influenced interactions between facilitators and participants, as well as among participants. Some programs sought to mediate this by involving facilitators from multiple professions at each session , applying the principle of “equitable distribution” for both participants and facilitators. Other programs chose to only use facilitators from one professional group .“Physicians are not, in my experience, super well-suited to facilitate simulations because there’s an already imposed hierarchy that comes into play when a physician is running the code, and I feel like it does, to me, dampen that group participation when there’s a physician” .Through this case study research of seven interprofessional simulation programs, we found variable implementation of 12 principles for effective IPSE we synthesized from the literature. Some principles such as “active learning”, “psychological safety,” “sociological fidelity”, and “feedback during debriefing” are commonly applied, while others such as “interprofessional competency-based assessment” or “repeated and distributed practice” are rarely applied. Based on the interviews with program facilitators, we believe that the full application of all 12 principles represents the ideal of IPSE, but important barriers prevented the programs we studied from accomplishing this ideal. In particular, we identified that buy-in, resources, lack of outcome measures, and power discrepancies influenced the extent to which principles were applied. The framework and results of our study can be used by those who plan a new ISPE program or want to optimize an existing program to consider the realities of implementation along with ideal features of IPSE programs. Reviewing the degree of application of the different principles, we noted that those grounded in SBE guidelines, such as “active learning”, “feedback during debriefing,” and “sociological fidelity” were more frequently applied. In contrast, those grounded in IPE guidelines, such as “interprofessional competency-based assessment”, “attention to differences and hierarchy,” and “equitable distribution” were inconsistently applied across programs, highlighting the difficulty of implementing IPE. Challenges to implementing IPE, such as limitations in resources, scheduling and stereotypes are well-known , 23; thuAn additional challenge our study has uncovered is the limited use of evidenced-based outcome measures to evaluate IPSE programs. The limited application of “interprofessional competency-based assessment” and “program evaluation” in the programs we studied reflects the lack of clarity among program leadership as to what they are measuring, and how they should be measuring it. This sentiment is echoed in the literature, with multiple systematic reviews on IPSE assessments highlighting the lack of rigorously gathered validity evidence, including limited evidence that improvement in studied outcome measures leads to improved interprofessional collaboration and patient safety in practice , 25. It In alignment with IPE literature, we found that power discrepancies between professional groups played a role in the implementation of IPSE programs –29. AddiOur work highlights the gaps found between principles and practice in IPSE. In order to see the full effect of IPSE on interprofessional collaboration and patient safety, we must change our focus from the ideal of simulation as imagined to the realities of simulation as enacted. To this end we must not only consider the theory and design of IPSE programs, but also pay attention to implementation. Applying implementation science to in situ interprofessional simulation may decrease the gap between research and practice . ImplemeOur study has important limitations. There was a two-year gap between the initial observations and interviews. While we asked interview questions that checked for program changes in the interim and found few, one program ended. A benefit of the delay was the opportunity to understand sustainability of programs and to highlight the role that program facilitators play in maintaining the programs. In addition, we did not discuss data from our observations with interviewees as we did not want interviewees to feel as though we were evaluating or critiquing their program. This limits our ability to explain discrepancies between observation and interview data for a few principles for programs B and F. We believe that these discrepancies are due to facilitators’ inability to apply principles as fully as they intend due to the barriers we identified in this study. We found that IPSE programs varied in their implementation of IPSE principles derived from commonly used guidelines for IPE and SBE due to a number of important barriers. To truly change interprofessional teamwork, IPSE programs may benefit focusing on the how with implement programs to achieve the ideal. The gaps that exist between the ideals of IPSE and the realities of implementation may be narrowed by higher buy-in from simulation participants, facilitators, and institutions; by more resource allocation to IPSE programs; by development and sharing of instruments to assess learners and programs; and by acknowledgement power discrepancies and their impact on learning.Additional file 1.


Sustained engagement with Internet-based behavioural interventions is crucial to achieve successful behaviour change outcomes. As this has been problematic in many interventions, a lot of research has focused on participants with little or no engagement. However, few studies have attempted to understand users with continuous long-term engagement, the so called ‘super engaged users’, and why they keep on using programs when everybody else has long stopped. Therefore, the aim of this research was to qualitatively examine characteristics, usage profile and motivations of super engaged users in the 10,000 Steps program.Twenty 10,000 Steps users  participated in semi-structured interviews, that were transcribed and thematically analysed.Participants were aged 60 years on average, with more than half being overweight/obese and/or suffering from chronic disease despite logging high step counts (219 million steps per participant on average) on the 10,000 Steps platform. Participants indicated that the reasons for sustained use were that engaging the program had become a habit, that the program kept them motivated, and that it was easy to use. Few participants had suggestions for improvement or expressed there were program elements they did not like. Uptake of program innovations  was modest among the super engaged users.The findings from this study emphasise the need for digital health programs to incorporate features that will support the development of habits as soon as participants start to engage with the program. While a program’s usability, user-friendliness and acceptability are important to engage and retain new users, habit formation may be more important for sustained long-term engagement with the behaviour and the program. In 2005 Eysenbach formulated the so called ‘law of attrition’ following the observation that in any eHealth trial a substantial proportion of users drop out before study completion, don’t use the application as intended or stop using the application prematurely . This paDespite the many innovations in e- & mHealth that have occurred since Eysenbach’s paper in 2005  , high atwww.10000steps.org.au). In its current configuration the 10,000 Steps program allows steps to be manually logged , as well as automatically synced from advanced activity trackers  to encourage physical activity self-monitoring and goal setting. A detailed description of the 10,000 Steps program can be found elsewhere [The 10,000 Steps Australia program is no exception. The program was first developed in 2001 as a whole of community physical activity promotion intervention , 14, howlsewhere . Usage plsewhere .While the majority of registered 10,000 Steps members stop using the program after a period of time, the program also has some very loyal users, who have been with the program for many years and logged millions of steps. Usage statistics indicated that to date over 37,000 participants logged steps on more than 100 separate days, over 7,000 participants logged steps on at least 365 separate days , and over 170 participants logged steps on at least 3,650 days . The participant with the longest engagement has logged steps for more than 15 years. While non-usage attrition was well studied in many programs , few stuwww.rev.com, checked for quality (CH) and then thematically analysed . Those who completed the study were eligible to participate in a random draw for the chance to win one Garmin activity tracker . The Central Queensland University Human Ethics Research Committee provided ethical approval to conduct this study .The database that supports the 10,000 Steps platform, and that includes information about registered 10,000 Steps members was used to select participants for this study. To be eligible participants needed to be listed in the database and have logged steps via the program website or smartphone app on a minimum of 365 separate days since tracking of steps was recorded on the 10,000 Steps website. To ensure a diversity of participants, and not only interview the most engaged super users, participants were divided into two groups: in the first group 10 participants were recruited who logged steps on more than 365 days, but less than 3650 days; in the second group 10 participants were recruited who logged steps on more than 3650 days. An attempt was made to recruit equal numbers of participants across different age groups, genders and geographical locations . To achieve this, the spreadsheet with data of registered 10,000 Steps participants was sorted by number of days with steps logged, gender, age, location and the date they last used the program. When there were multiple potential participants we could invite to participate, the participant who had most recently used the 10,000 Steps program was invited to participate first. If they did not respond or declined, the next person on the list was approached. E-mail was used to invite eligible 10,000 Steps members; up to 3 reminder e-mails were sent before selecting another potential participant. An appointment to conduct a telephone-based semi-structured interview was organised when potential participants responded to the e-mail invitation and agreed to participate. The e-mail also included a Participant Information Sheet to fully inform participants about the aims and nature of the study, as well as an informed consent from and a brief demographics questionnaire, that needed to be returned to the research team before the interview was conducted. Participation was completely voluntary, and participants were informed that they could withdraw anytime throughout the interview. All interviews were recorded, professionally transcribed verbatim by Data obtained through the 10,000 Steps website was: date of registration , number of step log entries, total steps logged, age, gender, and use of advanced activity trackers. Data obtained through the demographics survey was: height and weight used to calculate Body Mass Index (BMI), educational attainment, work status, health status, presence of chronic disease and the behavioural automaticity index score. The four item self-report behavioural automaticity index (SRBAI) was used to determine to what extent the physical activity behaviour of participants was automatic , 19. ParPrior to conducting the interviews, the researchers had developed an interview guide see . InterviData collection was conducted from February 2020 to September 2020. Data analyses were conducted between January and February 2021. The interview recordings were analysed thematically . InitialThe original intent was to analyse participants who logged less than 3650 days separately from those who logged more than 3650 days. However, as there was no clear distinction in how they responded to the questions in the interviews, they were all analysed as one group. Other studies using semi-structured interviews indicated to have achieved data adequacy with less than 20 participants –26. TherIn order to recruit 20 participants, a total of 52 people were invited to participate. An equal number of men and women participated, equally distributed between those who logged steps on the 10,000 Steps platforms more or less than 3650 times see . The aveThe 20 interviews ranged in time from 6 to 33 minutes, with the average time being 16 minutes. Nine main themes were identified: 1) reasons for starting to use the 10,000 Steps program, 2) reasons for long-term use of the 10,000 Steps program, 3) what participants liked about the 10,000 Steps program, 4) how the 10,000 Steps program can be improved, 5) what participants did not like about the 10,000 Steps program, 6) encouraging others to use the 10,000 Steps program, 7) how participants changed their use of the 10,000 Steps program over time, 8) using the 10,000 Steps program while on holiday, and 9) tracking behaviours other than physical activity.The majority (n = 12) of super engaged users started using the 10,000 Steps program because their workplace organised a ‘Tournament’ , or because their workplace provided them with a pedometer (n = 3). A small number of people found out about the program through word of mouth (n = 3) and others started using the program because they were diagnosed with diabetes (n = 1), because they were able to use a pedometer through a library loan scheme (n = 1) or because they bought a Fitbit activity tracker (n = 1).“Well, I guess, it all started a number of years ago, it’s been quite a while. I couldn’t tell you how long now. My HR department gave out pedometers and links to the 10,000 Steps website. And that was where it all began.” “I don’t remember. Years ago, way back in about 2004 or something like that. I don’t know. I was walking a lot, and I think someone said, "Do this 10,000 Steps program where you can put all your steps online." And that’s how I looked it up and joined.” The two main reasons why the super engaged users keep using the 10,000 Steps program, is because over time using the program became a habit (n = 13) and because the program keeps them motivated to stay active (n = 13). Others indicate they keep using the program because they like how data driven it is (n = 6) or because it keeps them accountable to be active (n = 4). A couple of participants indicated they keep using the program because it helps them to keep their weight under control (n = 2), because they like the monthly Challenges (n = 1), because the program allows them to sync their steps with different devices (n = 1), because it is simple to use (n = 1) and because they are loyal to the program (n = 1).“Yeah, it’s got to be a bit of a habit. Almost an obsession now that I make sure I get my steps in. I’ve been doing it for so long that it’s part of me now, you know?” “If I don’t have my pedometer, the physical one, I get a bit, where is it, I need it. I don’t feel like I’m complete without it. So really quite a habit. Yes.” “So yeah. It’s just something that just keeps you, I suppose, motivated to keep logging the numbers I suppose.” When asked what the super engaged users liked about the program the most common response (n = 7) related to the simplicity of the program and its ease of use. Another common response (n = 5) was that it records all activities and provides lifetime statistics even for long-term users. Several participants (n = 4) indicated that they liked that the program motivates them to stay active. Other less common (n = 1) responses were: that it is an Australian program, that it allows to add other types of activity  that can be converted into steps, that it is designed for average people, that their cardiologist recommends they use the program, that they like the 10,000 Steps concept and the monthly Challenges, that the program allows to set daily aspirational goals, and that the program adopts new technology as it becomes available.“What I do like about is the app and the website are both easy to use and navigate. I find that useful.” “Oh, it gives you a goal to aspire to, for each day, and then, you can make your own targets.” “I think what is good about it is like it’s kept up-to-date with the changes in technology, like it’s not the same as it was in 2004 or 2010.” When asked if anything could be improved to the 10,000 Steps program, half of the participants indicated no improvements were needed. Those that did have ideas mostly suggested improvements to the monthly Challenges  and the 10,000 Steps smartphone app . Some participants also had suggestions to improve the overall engagement with the platform .“No, nothing that I can think of. I’m quite happy with it. I just put my steps on just yesterday and I can’t think of anything else that could improve.” Nearly all of the Super Engaged Users (n = 16) indicated there wasn’t anything they specifically disliked about the 10,000 Steps platform. However, when those that did mention something they didn’t like, it always referred to very specific technical aspects. For example, not being able to sync trackers directly with the platform, or not liking how the leader board or step challenges are organised.“No, I can’t think of anything that is occurring that I dislike. I think it’s amazing how it integrates with the Fitbit effortlessly.” “I guess I could say I don’t like the fact that the Apple integration isn’t there. But you’ve mentioned that that’s something that might coming down the pipeline in the future.” The majority of Super Engaged Users (n = 14) did indicate that they encouraged others to walk, though the remaining participants said they didn’t do this. Some indicated that they specifically encouraged others to use the 10,000 Steps program (N = 7), whereas others encouraged the habit of walking more broadly, without specifically referring to the 10,000 Steps program (n = 2). Some indicated they had tried to get others into walking, but that it didn’t work (n = 4). Several specifically referred to encouraging their partner to walk more (n = 3).“Well, I tried. Tried to get people to join the team at work, but I don’t know. I think I was too much of a hard-ass.” “I do encourage other people to walk. But I’m not sort of preaching it, but it’s probably actually more neighbours and stuff like that know I walk everywhere. And they always say something about it and they always say to me, "I need to get into” They don’t always do that but sometimes they do. Yeah.” The 10,000 Steps program launched an entirely new website  in 2017. Nearly all participants either didn’t notice/remember the change to the latest version of the 10,000 Steps website, or they liked it better. Though one participant mentioned that she struggled at first to get used to the latest website, but got used to it quickly thereafter.“Yeah, yeah. No, I have liked the changes. I do it all on my iPad, on the website, I don’t use a computer or laptop, but no, I have liked the changes. To me the changes have kept up-to-date.” “I found them a problem initially, but once I worked out… I can remember contacting someone once and saying, "how do I do something or other?" But once I worked it all out I was right. Because I’m not very technical anyway.” The 10,000 Steps program first launched a smartphone app in 2009. The usage of the app among super engaged users is mixed, with the majority indicating they don’t use it (n = 11), a couple who use both app and website (n = 4), and only a few who say they only use the app (n = 3). Of those that use the app, satisfaction is mixed, with some really liking it and others indicating the functionality of the app is too limited compared to the website.“No, you see I’m one of these prehistoric people that doesn’t own a mobile phone.” “I actually use both. I use the website when I’m at work, and I use the app when I’m at home.” “Yeah, the website because I found that the… There’s the app and for some reason the app and the website don’t sync. If you put into the app it doesn’t sync to the website. But this was a while ago.” The 10,000 Steps program first integrated the use of activity trackers including Fitbit (2017), Garmin (2019) and Google Fit (2021), to enable automatic syncing of steps, in addition to the option of manually updating step counts for those that are using a traditional pedometer or other step tracking device. While many of the super engaged users are still using pedometers (n = 8), a significant proportion are now using activity trackers (n = 9). The majority of these trackers are compatible with the 10,000 Steps website (n = 6), but some are also using trackers that don’t automatically sync , meaning they still have to enter their steps to the 10,000 Steps platform manually. Several (n = 3) also indicated that they use the step counting feature of their smartphone as a backup for when their pedometer or tracker is not available.“I have used a variety of things. I’m currently using an Apple Watch.” “I find a pedometer is actually a little bit more accurate and I’m not really interested in how many calories I’m burning and stuff like that. But I thought the pedometer is much more accurate than the other devices, for some reason. My backup is my phone. It just works off GPS, whatever algorithm they’re using is probably reasonably good, but the pedometer is my go-to.” Most participants (n = 11) reported that they tried to keep track of their steps during holidays. Some (n = 3) indicated it is easier to be active during holidays, as there is more time for it. Several (n = 5) participants indicated that the 10,000 Steps platform (website + app) makes it easy for them to keep recording steps, though many others (n = 7) indicated that they record their steps offline  for a while  before logging them on the website. One participant indicated that they always wear their Fitbit, but only sync their steps when they take their tablet with them during the holiday.“But on holidays when I’ve got a lot more time, I go out of my way to do it.” “Oh, the same. We are very conscious of when we go out for a walk, and all of our holidays are planned around walking. Walking is our life, in a way. We love it. And so we always, and my partner is much better at this than I am.” Many participants (n = 7) indicated that they are not tracking any other behaviours  other than their steps and physical activity. Some (n = 5) do use other apps to track their activities .“No. I’ve thought about the sleeping part and I’ve thought, I’m not sure it’ll tell me what I really want to know. I’m not sure how it works it out to be honest.” “Oh, actually I think it is an app now, but Mapometer. If you’ve ever heard of that. That actually measures a route so you can draw the route you take. So if you cut through a park, you can draw through that park and find out how many kilometers that particular route was.” This study aimed to gain a deeper insight into what drives super engaged users of the 10,000 Steps program to continue to use the program, well beyond the levels of engagement typically observed in this program and many other interventions. Both participants’ scores on the SRBAI, as well as participants’ responses in relation to why they keep using the program point at the development of deeply ingrained routines and habits. Habits are automatic and impulsive processes, activated by contextual or environmental prompts, which require minimal cognitive effort, awareness, control or intention . Habit sAs many of the super engaged users indicated that no further improvements to the program were needed, and that there was nothing they didn’t like about the program–despite significant changes to the program over time , it is l4,17 Likewise, there are distinct benefits to using traditional mechanical pedometers over more advanced activity trackers. Pedometers are significantly simpler in use: typically one function only  compared to a plethora of functions  [Other factors that point to the importance of habit formation in the super engaged users are the low usage of the 10,000 Steps smartphone app and the high usage of pedometers . This may be due to the program not having a smartphone app available prior to 2009, nor the option to log steps with advanced activity trackers prior to 2017, when they first started to use the program. It may be that these participants got used to the program without having these features and possibly didn’t feel a need to change how they interact with the program. However, it is also important to note that several participants did mention the limited functionality of the app  that may have prevented adoption of this feature. Despite the lower adoption of the 10,000 Steps app in super engaged users, our previous research nevertheless demonstrated the importance of the app in improving engagement with the program and offering users a range of options on how they use the program.racking) . Good quracking) . Pedometracking) . While pracking) .To a large extent the success of the 10,000 Steps program was driven by the very large adoption of the program by workplaces  through their participation in ‘Tournaments’  . The impFinally, while some of the super engaged users indicated that they specifically started using the program in order to manage a chronic disease, the high level of overweight and obesity among the participants, as well as the high prevalence of chronic disease among participants was a surprising observation. Given the sustained high engagement with this physical activity program , and given the demonstrated effectiveness of the program , 14, a lA strength of this study relates to the inclusion of participants with extremely high levels of engagement with a health promotion program over a long time period. The experiences of super engaged users are rarely reported, often due to the lack of access to such users given high non-usage attrition reported in many digital health programs , as wellIn conclusion, the super engaged users demonstrated very high levels of habit strength. This emphasises the need for digital health programs to incorporate features that will support the development of habits as soon as participants start to engage with the program. Over time, the power of habit appears to become more important than a program’s usability, user-friendliness and acceptability or latest new features. However, as all users are new users at some point in time, the importance of having a highly convenient and user-friendly program should not be underestimated or ignored. When a program goes through cycles of renewal it is equally important that it doesn’t disrupt the habits of its super engaged users, so they can keep using the program as they always have.S1 Data(CSV)Click here for additional data file.


This is a quasi-experimental and pioneering study in Portugal.(1) to provide assessment materials for symptoms of internalized homophobia, depression, and anxiety targeted at LGBT people; (2) offer support materials for psychotherapeutic work-oriented in the areas of internalized homonegativity, depression, and anxiety; and (3) offer monitoring measures throughout the program to demonstrate changes. It consists of three phases .38 LGBT + individuals participated, average age was 34.15 years, 30 self-identified as male. Measures used for the pre and post-intervention assessment were the sociodemographic questionnaire, the LGBT identity questionnaire, the Rosenberg self-esteem scale, and the BSI-18. Participants were invited to join the program online, through a platform created for this purpose, where ethical aspects were clarified, namely: confidentiality and commitment to adherence. Therapeutic tasks were sent by email or WhatsApp depending on the preference of each participant.Relevant differences in internalized homophobia, depressive, and anxious symptoms between the pre and post-intervention moments were observed, indicating that the program is effective in changing these symptoms.The importance of validating this type of program allows reaching “hidden” populations by offering online support that minimizes the effects of sexual stigma on LGBT + populations.


For seven years, the University of Rhode Island (URI) Engaging Generations Cyber-Seniors Program has utilized university student mentors through an internship format and integration within service learning courses. In 2019, community interest in the program significantly increased as did student interest in gerontology internships. This led us to develop a robust internship program focused on building age and digital inclusivity across campus and the state of RI. The program integrates a three-pronged approach where students complete field hours within the URI Cyber-Seniors program, focus on enhancing the Career Readiness Competencies employers seek in graduates today as determined by the National Association of Colleges & Employers, and complete the components necessary to earn the Geriatric Education Center Interprofessional Teamwork in Geriatrics and Gerontology Certificate. This process has enabled 350 students from different majors, including Pharmacy and Human Development & Family Science, to complete internship experiences ranging from 30 to 210 hours.


The intervention group reported significant improvements in resilience, acculturation stress, depression, and quality of life in comparison with the control group. The results indicated that the CAPRP, developed based on positive cognitive appraisal, was an efficient nursing intervention for mothers-in-law in multicultural families.Mothers-in-law in multicultural families tend to experience psychological burden. This study aimed to verify the effects of the cultural adaptation promotion resilience program (CAPRP) on resilience, acculturation stress, depression, and quality of life among mothers-in-law in multi-cultural families. Forty-two participants from multicultural family support centres in the metropolitan city A were assigned to either the intervention group or the control group. The CAPRP was performed for 60 minutes, twice a week for four weeks. The intervention group showed a significant decrease in the acculturation stress ( After initiating a couple of decades ago, the percentage of transnational marriages reached the highest percentage at 30.7% in 2005. Although there has been a decrease, it is still at about 10% of the total number of marriages in South Korea, where the current intervention study was conducted . Thus, tp < .001) [Different from USA, Australia, and European countries, the married couple, a Korean husband and a foreign wife, are more likely to live in together with the Korean male’s parents. Multicultural families with Korean males and foreign females are five times more likely to live with mothers-in-law at 75% than domestic families at 14.1% , 7. SimiExposure to negative stress and failure of coping strategies can lead to depression, loss of meaning of life, demotivation, and impairment of quality of life (QoL) . CompareAccording to Lazarus , stresseOn the other hand, a resilient response to stress can lead to flexible coping strategies in various situations , 18. ResResilience programs help in building psychological flexibility and relaxation  and in eA non-equivalent control group pre-posttest design was adopted for evaluating effects of the cultural adaptation promotion resilience program (CAPRP) for mothers-in-law in multicultural families. Both groups completed pre- and post-test questionnaires, but the intervention group participated in the CAPRP. After completion of the study, the control group was provided an alternative program, only one three-hour session as needed. The study design is presented in Participants were recruited from multicultural family support centres in the metropolitan city A. A list of nine multicultural centres in the metropolitan city A was found and two centres accepted to participate in the study after the first author either phoned or visited the centres to explain the study’s process. The intervention group was randomly chosen from either C or D centres by picking out one piece of paper from an opaque box. Twenty-four participants from the C centre who consented and were enrolled to the study considered to be in the intervention group, while 25 from the D centre to be in the control group. Inclusion criteria were participants who were 1) Korean mothers-in-law, living with her foreign daughter-in-law more than six months; 2) providing consent to participate in this study; 3) without mental or cognitive impairment; 4) able to understand and answer questions in Korean; and 5) without any hearing impairment.The sample size of the study was calculated in consultation with a health statistician based on the previous studies –33. The The questionnaires of the study consisted of socio-demographic characteristics , resilieThe socio-demographic characteristics of participants in this study included age, marital status, education level, and religion. Their perceived economic level, health state, and satisfaction of family support was answered on a five-point Likert scale. Information of whether they have any disease was also asked as well as disability due to disease on a three-point Likert scale. Questions about their daughters-in-law included the country which their daughters-in-law were from, duration of living with them, perceived daughters-in-law’s Korean level on a five-point Likert scale, satisfaction with the relationships with daughters-in-law on a five-point Likert scale, and ethnocentrism, asking if their daughters-in-law should fully adapt Korean culture regardless of their original culture, on a three-point Likert scale.The Resilience Scale-14 (RS-14), developed by Wagnild & Young , was useAcculturation stress of mothers-in-law in multicultural families was measured with a scale developed by Chung& Park . This inDepression of mothers-in-law in multicultural families was measured by the 30-item Geriatric Depression Scale Short Form-Korean Version (GDSSF-K), which Kee  standardThe Korean Version of WHO Quality of Life Scale Abbreviated Version (K-WHOQOL-BREF) by Min et al.  was usedThe CAPRP program was developed by the first author for mothers-in-law in multicultural families in this study and provided by the first author and an assistant researcher after appropriate training. For preparing for implementation of the program, the first author had completed several programs of art counselling psychology, positive psychology, resilience coaching, narratives, motivation counselling, and built experiences in developing and conducting a strength-focused resilience promotion program for older women. In addition, the first author tried to expand her expertise through attending various conferences and workshops related to psychology. Two assistant researchers were prepared for data collection by the first author for consistency.Goals, components, contents, activities, and the number of sessions of the program were formulated after relevant literature review , 39, 40.Previous literature revealed that major stressors of mothers-in-law in multicultural families were differences in culture and lifestyle from their foreign daughters-in-law, and that they have somatic symptoms, severe depression, impaired QoL, enhanced by passive and negative coping strategies. The final goals of the program thus are ‘improvement in cultural adaptation resilience’, ‘a decrease of cultural adaptation stress’, ‘a decrease of depression’, and ‘an increase of QoL’.Based on the seven components of resilience, a goal for each session was ‘self-opening’, ‘causal analysis’, ‘impulse control’, ‘emotion regulation’, ‘empathy’, ‘sense of optimism’, and ‘utilisation of strengths’. The contents and other details of the program are presented on The CAPRP in this study was implemented twice per week for four weeks, 60 minutes per session for a total of seven sessions. The program was provided to 21 participants in the intervention group by the first author of the present study with the help of a pre-trained assistant. An education room in the multicultural centres was selected, considering the participants might be vulnerable, so that the participants was able to feel comfortable and free to share their thoughts. The venue was equipped with internet and a beam projector, used for presenting various examples of multiculturalism.Data were collected from January 04, 2016 to March 04, 2016 by two trained research assistants to avoid researcher bias. On the first day of the program, three of the enrolled participants in the intervention group were no-shows, and 21 participants completed the baseline questionnaires and took part in the program. In the control group, 22 participants attended to answer the baseline questionnaires. One participant in the control group dropped out at the follow-up. A total of 42 participants were included in and completed the study. A week after completion of the program, both participants in the intervention group and the control group were asked to answer follow-up questionnaires.An ethics approval from the Institutional Review Board of the related university was obtained (2015_88_HR). Two research assistants explained the purposes of this study, its procedure, voluntary participation, confidentiality and privacy, then the written informed consent was obtained from all participants. Incentives for participation were provided with the equivalent of transport fares.The collected data were analysed using the SPSS/WIN 23.0 program. The homogeneity of the two groups was analysed with descriptive statistics, chi-square test, Fisher’s exact test, and t-test. The effects of the program were examined by t-test after a normality test of the dependent variable and the effect size of the CAPRP was calculated and presented with Cohen’s d  based on resilience, acculturation stress, depression, and QoL in both pre-and post- scores. Statistical significance was set at the level of p < .05 for a two-tailed test.Looking at p = .002), and depression , while resilience  and QoL  significantly increased compared to the control group. The intervention group showed a significant decrease in acculturation stress  scored resilience  as 4.4. This was similar to the study conducted by Kim & Na . The subThe 5-point converted score of acculturation stress was 2.9 among mothers-in-law in multicultural families in this study. This score was similar to the score 2.8 of Chinese international students in South Korea , while iAs a result of the CAPRP, the 5-point converted score of acculturation stress decreased by 2.3 among the mothers-in-law in multicultural families who used to restrain their emotions in a patriarchal society. Participating in the program, they learned how to express their emotions in a positive way throughout the activities such as drawing a person that everybody knows, drawing a mandala, and tearing a piece of paper. It was considered that the time when they appreciated each other’s drawings might help them understand various points of view. Park & Park  applied The 15-point converted score of depression was 5.3 (mild depression), showing a decrease to 3.0 after the CAPRP among mothers-in-law in multicultural families in this study. This score was higher than older adults living at home, who scored 1.8 . On the After the CAPRP, the 5-point converted score of QoL increased from 2.9 to 3.7 among the mothers-in-law in multicultural families. The results of the current study were similar to those of a study comparing a positive psychotherapy program with a cognitive behaviour therapy program among depressed older adults . The parSimilar to the previous studies , 46, theThere are several limitations to the present study as follows: Firstly, the mothers-in-law in multicultural families, who participated in the current study, were unable to take part in this study for a longer period because they had to spend much time with house chores and rearing their grandchildren. It is suggested to examine the long-term effectiveness of the program in the future studies. Secondly, comparison with previous studies was limited because there were no previous resilience programs for mothers-in-law in multicultural families. Further studies, applying the CAPRP program with larger sample sizes would be recommended to examine effects of the intervention. Thirdly, the control group did not receive any treatment, so it can be recommended to compare the CAPRP program with other treatments or regular programs for the seniors. In addition, the effect of a program may decrease over time , thus a In spite of these limitations, the current study has a few strengths. To our knowledge, this CAPRP is the first program for cultural adaptation of older females, that is, mothers-in-law. Previous programs for the elderly were mostly focusing on their depressive symptoms . RecentlThis study aimed at developing and applying the CAPRP to mothers-in-law who have foreign daughters-in-law, and to identify the effects of the program. The intervention group reported significant improvements in the resilience, acculturation stress, depression, and QoL in comparison with the control group. The results indicated that the CAPRP, developed on the basis of positive cognitive appraisal, was an effective nursing intervention for mothers-in-laws in multicultural families. The visual materials used in this program were efficient enough for the older adults to expand the eligibility of the program.


Educational practices are indicated to promote the health of people with fibromyalgia in primary health care. We aimed to develop an educative interdisciplinary program intended at the health promotion of individuals with fibromyalgia.It is a study protocol that was developed following three phases in the city of São Paulo city, Brazil. Qualitative research was carried out, through a focal group, with 12 individuals with fibromyalgia and 10 health professionals. A thematic content analysis was made according to the content proposed by Bardin.Fibro Friends is an interdisciplinary program with educational approaches that must be performed in 15 meetings, once a week for 1 h and 20 min. Participants were the following professionals: a Physiotherapist, a Doctor, a Psychologist, a Nutritionist, a Nurse, a Pharmacist/Druggist, a Speech Therapist, an Occupational Therapist, a Naturopath, and a Social Worker. A physical exercise program will also be carried out. The professionals must discuss in a lecture, conversation hearing, and/or group dynamic, about strategies to promote health and pain control in fibromyalgia.Fibro Friends is a program presenting educational interdisciplinary information to individuals with fibromyalgia, being considered a trend to future care. Fibro Friends is a practical guide, logical, and efficient to patients with fibromyalgia at the basic attention to health. Fibro Fet al., Sistema Único de Saúde), with complete elementary school grades to participate in qualitative research through group discussion. Since it is a qualitative approach, the sample is defined by the saturation of the question to be analyzed. There was not, therefore, a previous statistic estimate to define the number of subjects to compose the sample. Therefore, once we have deep and sufficient information for a scientific investigation, data collection might be ended and considered finished. The interview technique, due to the time involved and deepness of the search, limits the number of individuals to be researched. The most common and indicated group technique in qualitative researches is the focal group. The focal group is a type of interview which is generally carried out with a small and homogeneous group from 6 to 12 people  were invited to participate in the group discussions about the program development. The choice of the number of professionals was according to the interdisciplinarity and with the possibility of a positive contribution to the aspects related to the quality of life of an individual with fibromyalgia. The invited health professionals were a Physiotherapist, a Doctor, a Psychologist, a Nutritionist, a Nurse, a Pharmacist/Druggist, a Speech Therapist, an Occupational Therapist, a Naturopath, and a Social Worker. This meeting aimed at questioning the professionals about what interdisciplinary matters are important, practical, and necessary to be approached by the program in primary health care, based on the professionals’ experience and on the current scientific evidence to fibromyalgia treatment, as well as the number, the time length, and the frequency of meetings. The project proposes to do group work rather than individual treatments. The program’s proposal is an interdisciplinary approach and all professionals that are in primary health care could participate.et al., The qualitative research was carried out through spontaneous oral speeches, in which the individuals with fibromyalgia and the healthcare professionals have spontaneously spoken about the investigated theme. Qualitative research considers exclusively meanings and processes, and not measurements. The results are presented descriptively and not numerically Material exploration (interview transcription)Data treatment (inference and Interpretation)The different phases of content analysis have been organized around three poles:et al.  were carried out on different days and hours, and the posterior meeting of patients and professionals was not carried out. This validation step sought to allow individuals with fibromyalgia and professionals, getting to know the program structure and the possibility of adjusting before starting the viability study phase.The results will be presented in the thematic category description defined by the professionals and individuals with fibromyalgia. The interdisciplinary program structure of promotion to fibromyalgia patients was conducted using a combination of activities that have followed the series of themes addressed which are based on the National Policy of Health Promotion. As a primary outcome, the study participants have chosen the ‘quality of life’, the secondary ones were: ‘level of pain’, ‘sleep quality’, and ‘self-care management’. A physical exercise program will also be carried out. To an educational program in health, directed to people with fibromyalgia, 15 weekly meetings lasting 1 h and 20 min are suggested on Tuesdays afternoon  under reference number: (RBR-3rh759).The next step is employing the ‘Fibro Friends Program’ through an important random clinical essay to assess the efficiency of the interdisciplinary program of health promotion in patients with fibromyalgia, and, soon, the researchers must provide new information about this subject. The protocol was registered at the Brazilian Registry of Clinical Essays (