Background and objectiveBoth family medicine applicants and programs dedicate significant resources to the interview process, a time for both parties to make an impression on the other in an attempt to find their best match. Despite the importance of this process, little research has been completed to ensure the process efficiently addresses applicant preferences on interview day and the surrounding process. This study aimed to determine the factors influencing the family medicine applicant preferences regarding the pre-interview, interview, and post-interview ranking process. MethodsThe study method was a cross-sectional electronic survey utilizing anonymous questionnaires that assessed demographics, pre-interview, interview, post-interview ranking preference, and applicants' experiences applying to a community-based family medicine residency program after their interview for the 2020 application cycle.ResultsOut of the 106 family medicine applicants, 48 responded; 52.08% were males, 52.5% were married, 58.33% applicants were from the osteopathic medical school, 33.33% were from the allopathic Liaison Committee on Medical Education (LCME) non accredited medical school/international medical graduates (IMG's), and 8.33% were from the allopathic LCME accredited medical schools. Free hotel accommodation was not offered from half of the programs to 27.8% of the applicants in the 2020 match cycle (pre-pandemic). Respondents favored electronic means of scheduling interviews with a positive experience with the online self-scheduling Electronic Residency Application Service (ERAS) calendar. A significantly higher proportion of IMGs applied to a higher number of family medicine programs followed by the osteopathic applicants. There was no statistical difference found between osteopathic and allopathic applicants for the number of programs they got invited to; however, the difference was significant for osteopathic and allopathic LCME accredited applicants who interviewed and ranked programs in the range of 11-20 . The respondents' most important experiences were interviewing the program director, faculty members, and tour the hospital facility. When ranking programs, these family medicine applicants considered the strength of program training, the quality of current residents, and the program's geographic location as the top three most significant factors, with mean importance ratings of 5.08, 5.02, and 4.35, respectively. Applicants also considered how the current residents perceive the program director, prior teaching experience, and program diversity with mean importance ratings of 3.42, 2.89, and 2.09, respectively.ConclusionAlthough applicants' preferences for family medicine residency programs are similar to generally reported by The National Resident Matching Program (NRMP) surveys, some key differences do exist. The program leadership should consider these preferences from the candidates’ perspective for a successful match in family medicine residency on both sides.  The National Residency Matching Program (NRMP) can be an arduous process for potential residents. Residency programs evaluate and rank candidates based on a number of factors such as results of standardized exams like the United States Medical Licensing Examination (USMLE) or Comprehensive Osteopathic Medical Licensing Examination (COMLEX), letters of recommendations (LoR), research, and interviews. However, the NRMP ranking goes both ways, as the applicants also create rank lists for the programs they apply to. According to the 2019 NRMP Applicant Survey, all applicants considered the top four factors when ranking programs, including overall goodness of fit, desired geographic location, quality of the residents in the program, and their interview day experience [Both applicants and residency programs devote significant time, resources, and expenses to the interview process, as this is an opportunity for each party to make an impression on the other.On average, family medicine residency programs were found to spend $17,079 on interviewing annually and $234 per residency interview [In 2019, 4,128 residency positions were available for family medicine, with 3,848 spots filled, with a fill rate of 93.2%. In 2020, the fill rate went down to 92.5% - the lowest rate since 2010. Additionally, fill rates for allopathic seniors reached historic lows in 2019 (39.2%) and 2020 (33.2%), resulting in more positions being taken up by osteopathic students and international medical graduates (IMGs) [Currently, there are few studies concerning the family medicine residency interview process about the applicants' perspective and needs. To our knowledge, there have only been two papers published on the subject. Schneeweiss et al. were published in 1982; the match process has evolved significantly since then . WoloskiThere are currently no studies based at community hospitals or comparisons between center data and national NRMP data. Six-hundred and ninety-five family residency programs are registered with the NRMP, and out of those, 564 are community programs , 8. ConsThis study is part of a quality assessment and improvement initiative for applicant experience during the residency interview season. It surveyed candidates applying to our community hospital, family medicine residency program on pre-interview, interview day experiences, and post-interview ranking preferences. This will serve the Family Medicine Match stakeholders to identify significant factors that influence the ranking process, specifically in community hospital-based family medicine residency programs.The Family Medicine residency candidates applying to community hospital-based WellSpan Good Samaritan Hospital’s residency program in the 2020 application cycle were surveyed via an anonymous electronic survey questionnaire about their pre-interview and interview experiences. The survey required approximately five minutes to complete. A group of physician investigators knowledgeable in survey development adopted the survey used by Woloski et al. . We estiThe questionnaire elicited information regarding demographics (four questions), pre-interview experience (eight questions), interview experience (nine questions), and post-interview/ranking questions (six questions). A multiple-choice response format was used for most questions. For most pre-interview experience questions, the response options were: all programs, nearly all programs, approximately three-fourths of the programs, half of the programs, and less than half of the programs. For interview experience questions, the options were rated on a scale from one to three: important, neutral/either way and not important/would rather keep it out. For the post-interview ranking questions, the options were rated on a scale from one to five about the importance of the ranking process for the applicant. A small number of the questions were open-ended . Descriptive statistics  were collected. The statistical software SAS 9.4  was used for the categorical variables. The Chi-square statistics were calculated to analyze significant differences between different variables. A p-value of <0.05 was considered statistically significant. Statistical significance was also determined between different medical schools, applicant’s preferences, and application process. A total of 48 questionnaires were collected and analyzed from 106 family medicine residency applicants (response rate 45.2%) applying for the 2020 match. Table Figure Many applicants were not offered free hotel accommodation from half of the programs they applied to 27.08% n=13). A majority of applicants were offered dinner with current residents at the program before interview day from all  or nearly all  of the programs; though 87.50% (n=42) of applicants preferred to have dinner with current residents the evening before the interview , continuity clinic tour (66.67%), and interview with a resident or chief resident (62.50%) during the interview day as shown in Figure Out of the allopathic LCME-non-accredited/IMGs applicants, 37.5% applied to 101-150 programs; however, 53.5 % of the osteopathic applicants and 75% of the applicants from allopathic LCME-accredited medical schools applied to the 1-50 programs as shown in Figure The results on the number of programs applied to by the applicants from different medical school categories were statistically significant, with a p-value of 0.0302. The number of applied programs was statistically significantly different (p=0.018) between applicants from allopathic medical schools and applicants from osteopathic medical schools. A significantly higher proportion of IMGs applied to a higher number of programs followed by the osteopathic applicants (p=0.0302). We did not find any significant difference between applicants from allopathic LCME-non-accredited/IMG’s 75%), LCME-accredited (93.75%), and osteopathic schools (62.96%) in the range of 1-20 programs that they got invited to for interview (p=0.25). However, we did find significant statistical significance (p=0.0013) between applicants from allopathic LCME-non-accredited/IMG’S (25%), LCME-accredited (93.75%), and osteopathic schools (29.63%) who were interviewed for 1-10 programs. Many osteopathic applicants (62.96%) were interviewed for 11-20 programs. Ranking of the programs in the range of 1-10 was statistically significantly different (p=0.0018) among applicants from allopathic LCME non-accredited/IMG’s (33.33%), LCME-accredited (93.75%), and osteopathic schools (29.63%). A considerable percentage (66.67%) of applicants from osteopathic and allopathic LCME-accredited schools ranked programs in the range of 11-20, as shown in Figure %, LCME-aWhile ranking the programs, applicants considered the strength of program training, current residents' diversity and background, and geographic location of the program as the top three most significant factors, with mean importance ratings of 5.08, 5.02, and 4.35, respectively Figure .The training program's strength was considered an important factor while ranking by 58.33% of applicants from osteopathic medical school, 33.33% of applicants from the allopathic LCME-non-accredited/IMG’s and 8.33% of the applicants from allopathic LCME-accredited medical schools.It was also interesting to find out that the strength of the training program was nearly equally highly ranked by all ethnicities, then the geographic location of the program was given the highest score by the White applicants with a mean score of 5.53 as compared to applicants with other ethnic backgrounds (p<0.0001). Current residents with their diversity and happiness in the program were highly evaluated by the Black applicant with a mean score of 6, followed by applicants who preferred not to disclose their ethnicity with a mean score of 5.67 and Asians with a mean score of 5.63 (p=0.0001). The diverse background and experience of the current faculty were ranked highest by Hispanic/Latino applicants (mean score of 4.20) followed by Asians and Black respondents with mean scores of 4.18 and 4, respectively (p=0.0003). The program director's teaching experience and diversity factor were scored highest by applicants who marked their ethnicity as "others'' and Asians with mean scores of 4.67 and 3.82, respectively (p=0.0013).Considering the program's leadership, applicants took into account how the current residents perceive the program director, prior teaching experience, and diversity factor with mean importance ratings of 3.42, 2.89, and 2.09, respectively Figure .Interestingly, when these results were analyzed according to various ethnicities of the applicants, Black applicant evaluated highly how the current residents perceive the program directors with a mean score of 4, followed by applicants who marked their ethnicity as "other" or preferred not to disclose their ethnicity with mean scores of 3.71 and 3.67, respectively, compared to their White applicants with a mean score of 3.60 (p<0.0001). Similarly, the diversity factor  of the leadership was evaluated higher by Hispanic/Latino applicants with a mean score of 3.40, followed by applicants who marked their ethnicity as "other" and Asians with mean scores of 2.50 and 2.06, respectively (p<0.0001). The program director's prior teaching experience was ranked highest by those applicants who preferred not to disclose their ethnicity, followed by White applicants with mean scores of 3.33 and 3.20, respectively (p<0.0001).This study examined the residency applicants' preferences and experiences during the family medicine residency match application, specifically the experiences during the pre-interview process and at the interview day, interview scheduling, and post-interview ranking.For the past eleven years, the number of family medicine residency positions offered and filled has increased steadily. However, it has been postulated that this is only a consequence of an increase in graduates' number, not an increase in interest . In the Despite minimal numbers for allopathic seniors, there are increases for every other applicant type. In 2020, the number of osteopathic US seniors matching to family medicine reached a historical high, 23.4% (versus 19.5 % in the 2019 match), with 104 more applicants matching in NRMP than 2019 combined NRMP and American Osteopathic Association's (AOA) match. It could be to the fact that most osteopathic institutions have a strong orientation towards primary care, while a number of allopathic schools simply see it as another specialty . Both thThis study shows that the osteopathic and IMG applicants apply to a higher number of programs than LCME-accredited allopathic medical students. There is not much published data on this, given that 2020 was the first match where students who graduated from osteopathic schools competed directly with all other applicants in NRMP. Before this, they had a separate National Match Program (NMP). With participation in regular matches and integration of osteopathic programs into the Accreditation Council for Graduate Medical Education (ACGME), many osteopathic schools have anecdotally been reported to advise their students to apply and interview "widely" to increase their chances. Weissbart and colleagues discussed such behaviors in the light of game theory and prisoner's dilemma for the Urology Residency Match specifically, but it is widely applicable to all NRMP .International medical graduates in residency recruitment are primarily stigmatized due to their limited clinical experience in the US as well as the impact on a program's reputation if applicants deemed less competitive are accepted . AnecdotThe sociodemographic characteristics of our sample closely represent the target population similar to the results of the biennial 2018 NRMP Program Directors survey, especially when compared with those family medicine programs which typically interview and rank each applicant group . Percentage of programs that interview and rank each applicant type was 99% US seniors, 94% osteopathic physicians, 77% US IMGs, 44 % non-US IMGs, and 42% Canadian applicants. Our survey had 58.33% representation from osteopathic medical school, 33.33% in combined allopathic, LCME-non-accredited, or international medical school, and 8.33% from allopathic LCME-accredited programs [The Accreditation Council for Graduate Medical Education (ACGME) requires all residency programs to have diversity. They state: "CPR 1.C. The program, in partnership with its Sponsoring Institution, must engage in practices that focus on mission-driven ongoing, systematic recruitment and retention of a diverse workforce inclusive of residents, fellows (if present), faculty members, senior administrative staff members, and other relevant members of its academic community" . Jarman Figure Data are scarce regarding the preferences of applicants to family medicine residency programs. An old study exploring the characteristics of the interview day that were preferred by the medical students applying to family practice residency programs showed that interviews and meetings with residents informally were considered as most helpful for applicants; this was followed by interviews with the program director or faculty. Kikano et al. surveyed the program directors to explore the markers of successful recruitment of students to family practice residency programs and demonstrated high-quality current residents and faculty as well as the current residents' positive feeling about the program to be statistically significantly associated with a successful match (p<0.01) . WoloskiAlthough the strength of the training program was nearly equally highly ranked by all ethnicities, the program's geographic location was given the highest score by the white applicants with a mean score of 5.53 compared to applicants with other ethnic backgrounds . Current residents with their diversity and happiness in the program were highly ranked by the black applicant with a score of 6, followed by applicants who preferred not to disclose their ethnicity with a mean score of 5.67 and Asians with a mean score of 5.63 (p=0.0001). The current faculty's diverse background and experience were ranked highest by Hispanic/Latino (a mean score of 4.20), followed by Asians and black applicants with mean scores of 4.18 and 4, respectively (p=0.0003). The program director's teaching experience and diversity factor were scored highest by applicants who marked their ethnicity as "other'' and Asians with mean scores of 4.67 and 3.82, respectively (p=0.0013). Although there is limited data directly addressing the above racial disparities for family medicine programs, there is general literature suggesting that applicants from underrepresented minority backgrounds are more likely to rank programs favorably with greater gender and racial diversity. Ku et al. conducted a study on nineteen residency programs at the Sandford University School of Medicine and concluded that for the women and minority applicants, "perceived" program diversity is an important factor in making their decision for program ranking in surgical training . FurtherOverall, the current study shows results regarding family medicine resident applicants' key factors of ranking are similar to the 2019 NRMP applicant survey. This includes the strength of program training, quality of current residents, and the geographic location of the program as the top applicant priorities overall. The NRMP's top factors for both allopathic and independent (osteopathic and IMG) family medicine applicants for ranking programs were nearly identical to each other, with "overall goodness of fit", interview day experience, and quality of residents in the program being the top three factors for both [This study also reflects that the individual family medicine residency programs can develop their own quality improvement projects for the applicants' recruitment process to ensure appropriate resource utilization for conscious recruitment and successful match. Future studies with a larger sample size involving multiple family medicine programs with representation from both community and university programs would be needed to understand the preferences of the family medicine applicants so that resources are directed to individual applicant types and eradicating any racial disparities.This study has both strengths and weaknesses. This study was performed as a quality assurance (QA) initiative. The results showed significant insights that are worthy of sharing with the general audience. This study was carried out at a single community hospital within a single specialty of family medicine, which remains a significant limitation. Additionally, the sample size was small, with a response rate of nearly half of what we anticipated from applicants. The demographics of sampled applicants are comparable to the general 2019 NRMP applicant survey. This does allow some generalizability of the results to other programs. However, with smaller sample size and with a diverse population, it was difficult to perform significant statistical sub-analysis.Although the preferences of applicants to family medicine residency programs are similar to generally reported by NRMP surveys, some key differences also exist. The program leadership should consider these preferences from the applicants' perspective for a successful match on both sides. The matching process is the beginning of a relationship between the matched applicant and the program, not merely the filling of a potential spot. Residency programs should adopt a diversity-conscious or, more specifically, URiM-conscious approach to accommodate the rapidly diversifying pool of future applicants, embracing equity and inclusion, and adapt the interview day to the applicants' specific preferences. This will ensure the appropriate distribution of costs and recruitment of highly qualified applicants. In the future, multicenter studies should be conducted to understand further the needs of URiM candidates applying to family medicine.


Appropriate management of lung nodules detected incidentally or through lung cancer screening can increase the rate of early-stage diagnoses and potentially improve treatment outcomes. However, the implementation and management of comprehensive lung nodule programs is challenging.This single-center, retrospective report describes the development and outcomes of a comprehensive lung nodule program at a community practice in Tennessee. Computed tomography (CT) scans potentially revealing incidental lung nodules were identified by a computerized search. Incidental or screening-identified lung nodules that were enlarging or not seen in prior scans were entered into a nodule database and guideline-based review determined whether to conduct a diagnostic intervention or radiologic follow-up. Referral rates, diagnosis methods, stage distribution, treatment modalities, and days to treatment are reported.The number of patients with lung nodules referred to the program increased over 2 years, from 665 patients in Year 1 to 745 patients in Year 2. Most nodules were incidental (62–65%). Nodules identified with symptoms (15.2% in Year 1) or through screening (12.6% in Year 1) were less common. In Year 1, 27% (182/665) of nodules required a diagnostic intervention and 18% (121/665) were malignant. Most diagnostic interventions were image-guided bronchoscopy (88%) or percutaneous biopsy (9%). The proportion of Stage I-II cancer diagnoses increased from 23% prior to program implementation to 36% in Year 1 and 38% in Year 2. In screening cases, 71% of patients completed follow-up scans within 18 months. Only 2% of Year 1 patients under watchful waiting required a diagnostic intervention, of which 1% received a cancer diagnosis.The current study reports outcomes over the first 2 years of a lung cancer screening and incidental nodule program. The results show that the program was successful, given the appropriate level of data management and oversight. Comprehensive lung nodule programs have the potential to benefit the patient, physician, and hospital system. Lung cancer is the leading cause of cancer-related death worldwide . Age-staIn recent years, an increasing number of centers have developed incidental lung nodule management and screening programs. Although the application of evidence-based guidelines, such as those of the Fleischner Society  or AmeriThe objective of the current paper is to describe the development and results of a lung cancer screening and incidental nodule management program in a community hospital serving 5 counties in Tennessee. This paper presents the requirements for a comprehensive program, volume for both incidental nodules and screening over time, and quarterly trends in referrals, diagnostic interventions, stage distribution, treatment modalities, and financial data.This is a single-center, retrospective chart review. Blount Memorial Hospital is a 304-bed facility serving 5 counties in Tennessee with a 16-county pulmonary referral area. Blount Memorial Physicians Group is a multispecialty private practice with a large primary care base. Lung cancer rates in Tennessee are among the highest in the United States, with an age-adjusted rate of 76.6 new cases per 100,000 people, compared to a national average of 58 new cases per 100,000 .Prior to the development of a comprehensive lung nodule clinic in 2016, the lung program at Blount Memorial Hospital consisted of a lung nodule screening service that was offered for a cash price of $99 United States dollars. A rudimentary incidental nodule program consisted of a single individual manually reviewing CT reports. There was a multidisciplinary cancer conference, but no structured program for evaluating nodules or coordinating follow-up. There was no monitoring of lung cancer screening scans, with reports only sent to the ordering physician.Beginning in 2016, a comprehensive and coordinated lung nodule program was initiated with the aims of achieving: (1) a robust cancer screening program; (2) timely capture of incidental lung nodules; (3) cohesive care between practitioners; (4) a streamlined referral program; and (5) a multidisciplinary “virtual clinic” with a nurse navigator who coordinated care across specialties.Figure All patients requiring diagnostic intervention (defined as invasive biopsy) were tracked. Most lung diagnostic interventions were conducted using image-guided bronchoscopy, including electromagnetic navigation bronchoscopy  and radial endobronchial ultrasound (EBUS) probe (Olympus). Mediastinal lymph node assessment was conducted using linear EBUS (Olympus BF-UC180F bronchoscope). Whenever possible, diagnosis and staging was conducted within a single procedure. Most patients were evaluated with both ENB and linear EBUS in the same procedure. Other procedures, such as CT-guided fine-needle aspiration and mediastinoscopy were only conducted when diagnosis and staging could not be accomplished with a single procedure. Thoracentesis was limited to patients with a pleural effusion of adequate size that was thought to be a malignant effusion. Symptomatic patients were defined as those with symptom(s) attributable to the lung nodule(s) at the time of presentation.The program goal was to improve patient outcomes, with specific objectives of achieving (1) a stage shift to earlier lung cancer diagnosis and (2) treatment initiation within 30 days of first nodule detection . The program was rolled out in a stepwise fashion, with the databases started in July 2016, followed by the formal screening program on September 1, 2016 and the incidental nodule program on October 1, 2016.Year 1 data are presented from October 1, 2016 to September 30, 2017, representing the first year of the fully operational screening and incidental management program. Available data for the “phase-in” screening period of July 2016 through September 2016 is presented separately to characterize the initial impact of screening program implementation on volume and stage shift; however, this data is more limited than the data available in Year 1 and Year 2.Analyses were performed in Excel and data were summarized by descriptive statistics (for continuous variables) or frequencies and percentages . Financial data were calculated as unbudgeted revenue generated by the nodule program, using Medicare reimbursement data for each specific year.Population-based modeling prior to program implementation predicted 1027 nodule patients in Year 1, of which 125 would require diagnostic intervention. Approximately 6000 CT scans were reviewed annually at the medical center during the study timeframe . In Year 1, 1792 patients were identified by the computer search. After review of the CT report and the CT scan, 1127 were determined to not have a nodule or to have a nodule that had been stable for more than 2 years. The remaining 665 nodules were evaluated according to ACCP guidelines and entered into the database as requiring either biopsy or follow-up . They were still included in the database to confirm that follow-up imaging was completed. The remaining 607 patients were managed by the nodule program.A total of 182 patients underwent a diagnostic intervention  while late-stage cancer was detected in 64.4% . After follow-up imaging, 9/483 (1.9%) required diagnostic intervention of which 5/483 (1.0%) had cancer. Three had negative biopsies and remain in follow-up with no progression and one had a pseudomonas infection that responded to treatment. A total of 474 patients remain in monitoring with no progression noted on follow-up imaging.During the second year, 745 patients were entered into the database and managed by the program, 169 (22.6%) underwent diagnostic intervention, and 86 (11.5%) received a cancer diagnosis. Four patients required a second procedure to complete staging or obtain a diagnosis. The remaining 576 patients were placed into watchful waiting. With follow-up scans, 5/576 (0.9%) required diagnostic intervention, of which 4/5 had cancer and 1/5 had a negative biopsy. A total of 572 Year 2 patients remain in monitoring.Figure Average days to treatment over 2 years Fig. c decreasThe number of new screening scans increased steadily each quarter, with 255 in Year 1 and 549 in Year 2. Patients with screening-detected nodules that required more than yearly CT follow-up were moved to the active database (29% in Year 1 and 20% in Year 2). Among first-year patients, 71% had follow-up scanning within 18 months of the initial scan.The total revenue generated by the nodule program in Year 1 was $2,193,994.81. Two-thirds of that revenue was generated by video-assisted thoracoscopic surgery, radiation oncology, and medical oncology treatment procedures, approximately 20% by imaging, and 15% by bronchoscopic or transthoracic biopsy. Revenue increased steadily through 2 years after program implementation, with $363,504 generated in the first quarter, increasing to a cumulative total of $3,812,461 at 18 months.Comprehensive lung nodule programs may facilitate an earlier diagnosis and a higher cure rate. While the structure and technology exist to capture and manage patients, implementation is often challenging. This report describes the first 2 years of a lung cancer screening and incidental nodule program. Several key outcomes were observed that may be informative to other medical centers considering adoption of a comprehensive lung nodule program.First, 62–65% of nodules in our program were incidental nodules. In Year 1, only 12.6% of nodules were identified through screening. Similarly, a report of a Northeastern United States comprehensive lung nodule program found that 80% of nodules were found incidentally and only 20% through lung cancer screening . Thus, aSecond, guideline-based interventional pulmonology review effectively reduced false positives and unnecessary invasive testing. Prior screening and incidental nodule studies have reported false positive rates as high as 96% . In our Third, our comprehensive program increased the proportion of early stage cancer diagnoses. Interventional pulmonology programs that include advanced minimally invasive diagnostic procedures, such has ENB, are associated with increased rates of early-stage diagnosis –27 and fFourth, our data show that a comprehensive nodule program can increase screening rates and follow-up compliance. In our practice, almost all patients meet the United States Preventive Services Task Force (USPSTF) screening criteria . While mFifth, although our analysis is limited in scope to the United States health system, our data support a meaningful financial impact for hospital and health systems of any size. We observed a total revenue of $2,193,994.81 in United States dollars in Year 1. While not all of that was “new” revenue (much of it would have been introduced into the system anyway), this number represents unbudgeted revenue generated by the nodule program. However, efficiency is also important to reduce unnecessary costs from over-management . In our Finally and most importantly, our data emphasize that careful patient navigation and oversight is essential to the success of any lung nodule program , especiaOur experience suggests that with careful oversight, a successful lung nodule program can be implemented in any type of community health system, not just large multi-hospital systems. Our initial two-year data suggest that all three stakeholders – patients, physicians, and hospitals – can benefit from a properly run, effective program. Patients benefit from earlier diagnosis and potentially better treatment outcomes, physicians benefit from the availability of high-quality multidisciplinary care that provides a “safety net” to ensure patients aren’t lost to follow-up, and the hospital benefits from downstream revenue and keeping patients within the system. Combined, this creates a “win-win-win” situation for the healthcare system and patients as a whole.


Since the giant 2011 Tohoku earthquake and tsunami, much research has focused on the distribution of coseismic slip at shallow depths during this subduction megathrust event. Here we present seismic images obtained in the immediate vicinity of the trench axis, that show thrust faults and fold-and-thrust type deformation structures near the epicenter of the 2011 Tohoku earthquake where the large coseismic slip has been inferred, and chaotic structure and the absence of thrust faults in northern and southern source areas. Seismic profiles show evidence of slope failures of the trench inner wall in a proposed tsunami source region around 39°–40° N, where the slips estimated from previous studies are in disagreement. Our results show that structural characteristics in the trench axis may be related to the occurrence of shallow megathrust slip and tsunamigenesis in the Japan Trench. One distinctive feature is a large (~ 50 m) shallow megathrust slip that occurred at around 38°–38°30′N. Another characteristic is a proposed tsunami source region at around 39°–40°N, to the north of the large shallow slip area identified by tsunami data analyses6. Seafloor geodetic observations7 and estimates of seafloor displacement based on bathymetric datasets11 are strong evidence with which to examine the propagation of fault slip towards the seafloor through the trench axis area. Research using differential bathymetry has shown that seafloor displacement during the Tohoku earthquake was > 50 m near the trench axis at 38°05′–38°35′N, where the large slip occurred during the Tohoku earthquake9, but was not significant outside of this area11  2011 Tohoku earthquake ruptured the plate boundary fault in the Japan Trench subduction zone through the shallowest part of the megathrust, and this shallow slip in particular enhanced the destructive tsunami that followed. Numerous rupture models have been provided to elucidate the characteristics of this giant earthquake using seismologicalHere we present seismic profiles acquired at the immediate vicinity of the Japan trench axis, including those obtained at the same locations as the differential bathymetry estimates. We use these results in combination with seismic profiles in the area near the proposed shallow slip and tsunami source to examine the seismic structure in the vicinity of the trench axis. We find that the structural characteristics seen in the seismic profiles are related to seafloor displacement and possible tsunami generation.12, as shown in Fig. 9, show deformation of the sediment units by thrust faults in the vicinity of the trench axis. Seismic profile HD34B  following a previous study6 ”. The amount of the slip along the plate boundary faults has been estimated as ~ 35 m4, ~ 25 m6, or 30–40 m26 around 39°30′N by tsunami data analysis. On the other hand, the differential bathymetry indicates that the trenchward horizontal displacement of the lowermost landward slope during the Tohoku earthquake, if any, did not exceed the uncertainty of the estimates. The estimated value was ~ 29 m at maximum around 39°12′N10, which is comparable to or slightly smaller than the values estimated from tsunami data. Differential bathymetry estimates suggested that the existence of localized very large slip of the megathrust in the shallow part near the trench is unlikely to explain the tsunami source, and the estimated direction of the seafloor shift was mainly trench-parallel and the trench-normal shift was small10. This predominantly trench-parallel shift might have been caused by the broader small uplift in the lower slope10. It is also possible that bathymetric features such as small bumps or valleys caused by slope failures obscured the correlation between the bathymetric data from before and after the earthquake. Taking into account the slope failures imaged in our seismic profiles of this region, we speculate that small-scale slope failure might have contributed to tsunamigenesis during the Tohoku earthquake. Note that no large-scale slope failure associated with the 2011 Tohoku earthquake was identified in the differential bathymetry result in this region, including the area of the proposed SMF24. The slope failures observed in this study is located ~ 20 km trenchward of the area of the proposed SMF24.The seismic profiles clustered around 39°30′N Fig.  that pre27  region is landward of our survey area29, the observed slump deposit might be related to the 1896 Meiji Sanriku event.Bathymetric data obtained before the 2011 Tohoku earthquake show the presence of slump deposits in this area27 Fig. , so slop30 found that the post-spreading volcanism altered the structure of the upper most part of the incoming Pacific plate in the vicinity of the trench axis around 39°N, which corresponds to the area where slope failures were identified in the seismic profiles. Magmatic intrusions and thermal metamorphism associated with post-spreading volcanism were suggested to disturb the smectite-rich pelagic clay layer in incoming sediments, and the subduction of this disturbed area was suggested to prevent giant near-trench interplate coseismic slip in this region30. Most of the seismic profiles around 39°30′N were interpreted to lack the chert layer (SU3) and suggest that the pelagic clay layer was disturbed in the shallow most part of the subduction zone. Seismic profiles  towed at 5 m depth. The data were sampled at 1 ms intervals, and airguns were fired every 37.5 m with 13.8 MPa of air pressure. We obtained the best time-migrated images from the recorded data by using conventional procedures described in Ref.12.The seismic dataset used in this study was collected during cruise KR13-11 of R/V


InCharge is a newly developed school-based health intervention aimed at older adolescents. It aims to promote a healthier lifestyle by increasing self-regulation skills. After the InCharge program’s effectiveness was previously investigated in a pilot study, the content of the program was adapted.This study describes the protocol of a cluster randomized controlled trial that aims to investigate the effectiveness of the InCharge program.A cluster randomized controlled trial including 70 classes with older adolescents (aged 16 years or older) in the Netherlands will be conducted to test the effectiveness of the InCharge program. After schools are recruited, randomization occurs at the class level. The trial consists of the following two conditions: an experimental condition and a control condition. Participants in the experimental condition will be given the InCharge intervention, consisting of four lessons of 50 minutes, with each lesson containing three assignments of approximately 15 minutes. While participants in the experimental condition will receive InCharge, participants in the control condition will receive regular academic school courses. Surveys are administered 1 week before the intervention (baseline), 1 week after the intervention (posttest), and 12 weeks after the intervention (follow-up). Variables of interest include, but are not limited to, self-regulation; predictors of snack intake, physical activity, and alcohol use; and interpersonal communication regarding these health behaviors. In addition to surveys, observations will be conducted during the first and fourth lessons, teachers will be interviewed, and focus groups will be held with a selection of students from the intervention condition.Enrollment started in September 2017. As of June 2019, a total of 1216 participants were enrolled for this trial. Findings will be published in peer-reviewed journals and presented at conferences. The trial has been approved by the Ethics Review Board of the Faculty of Social and Behavioral Sciences of the University of Amsterdam (reference no.: 2017-PC-8244).In this study protocol, the design of a cluster randomized controlled trial is described, which assesses how effectively the school-based intervention InCharge stimulates healthier lifestyles in late adolescents. We hypothesize that participants in the experimental condition will consume less alcohol, eat fewer unhealthy snacks, and be more physically active compared with participants in the control condition.Netherlands Trial Register (NL6654); https://www.trialregister.nl/trial/6654RR1-10.2196/17702 Health and mortality are strongly affected by behaviors such as excessive alcohol use, a poor diet, and physical inactivity . Some heIn the past decades, numerous health promotion programs have been developed in order to prevent the rapid increase of unhealthy behaviors during adolescence. The majority of these programs take place in the school environment, because many adolescents can easily be reached by school-based health interventions. Several studies have suggested that these interventions can indeed promote healthier lifestyles, such as a healthy diet  and physFurthermore, most health promotion programs include several different components, and in most evaluation studies, the exact relationship between these intervention components and their effects remains unclear. In order to understand which program components contribute to healthier lifestyle behaviors and which components do not, several authors conducted meta-regression analyses. For example, previous research demonstrated that self-regulation, which is defined as the capacity needed to resist temptations and impulses , was an Therefore, the school-based health intervention investigated in this protocol study also incorporated self-regulation as the main prevention strategy to promote healthier lifestyles. The intervention, called “InCharge,” is newly developed by the Trimbos Institute . The InCharge program is developed for older adolescents and aims to promote healthier lifestyles by improving self-regulation skills . The program helps students realize what is important to them and how certain temptations can hinder them from achieving these goals, teaches them that self-regulation can be used to resist certain temptations , trains self-regulation skills by formulating an action plan, and helps them practice self-regulation through various assignments. Finally, alcohol is introduced as a temptation, and students formulate action plans for responsible alcohol use in order to stimulate healthier drinking behaviors.This protocol study also investigates potential factors that determine the effectiveness of the InCharge school-based health intervention, because previous studies have found large heterogeneity in the effectiveness of these types of interventions . In partThe InCharge program is based on several psychological theories such as the social cognitive theory (SCT) , the theThe InCharge program is developed by taking into account the developmental characteristics of older adolescents. The primary developmental tasks of late adolescence are forming an identity, planning the future, and acquiring the necessary skills to transition into adulthood . As oldeThis study describes the protocol of a randomized controlled trial (RCT) including 70 classes with older adolescents, which aims to evaluate the effectiveness of the InCharge program. Baseline, posttest, and 3-month follow-up measures will be conducted to determine whether the program successfully influences the determinants of health behaviors. After completing the program, students in the intervention condition are expected to refrain more from binge drinking and eating unhealthy foods, such as snacks and sweets, and to be more physically active than students in the control condition.Other aims of our study are to understand the mechanisms through which the InCharge program has its effects. We investigate whether the effects of the program on health behaviors can be explained by increased self-regulation, which is expected to be one of the core mechanisms of the program. Additionally, as previous research has shown the importance of interpersonal communication in mass-mediated health interventions , our stuThe InCharge effectiveness study is a cluster RCT including the following two conditions: an experimental condition (the InCharge program) and a control condition (no intervention) . ParticiThe Municipal Health Services and prevention departments of regional institutions for the treatment and care of addiction help to implement the InCharge program in schools. From these Municipal Health Services and regional institutions, 47 prevention practitioners are trained by the Trimbos Institute to instruct teachers on how to deliver the program and are asked to provide us with a list of schools in their region interested in participating in the InCharge study. The research team contacts these schools to explain the objective and design of the study. First, schools receive an invitation and an information brochure via email, and thereafter, schools are contacted by phone to discuss participation in the study. Schools are allowed to select the number of classes participating in the study, with a minimum of two classes because randomization occurs within schools. To create comparable research conditions, schools are instructed that a minimum number of two classes should be from the same educational level of the field of study. Classes are eligible for inclusion in the study if they contain older adolescents and the level of education is intermediate vocational education, higher general secondary education, or preuniversity education. Students are recruited by class participation. In collaboration with the schools, students’ parents are informed about the study objectives and receive passive informed consent forms. Written active informed consent is obtained from all participating students.We randomize the participants at the class level, and classes within schools are matched on the school level in order to obtain more comparable research conditions. Classes are randomly allocated to the experimental or control condition by means of repeated coin tossing. For each selection of matched classes within schools, classes are assigned to the experimental condition if the coin toss shows heads and to the control condition if the coin toss shows tails. The coin toss procedure is continued until all matched classes are assigned to a research condition, such that as many matched classes as possible are assigned to the experimental and control conditions. Randomization is carried out centrally, with two researchers present to monitor the procedure.d=0.2) [α=.05 and a power of (1–β)=0.80, 393 students are required per condition. However, as students are nested in classes and the program is delivered to classes of students, observations of students in the same class are not independent. Therefore, the required sample size is corrected for clustering. Assuming an intraclass correlation of 0.025 and an average class size of 16 participating students, the required sample size increases to 560 students in 35 classes per condition. Calculations are made using the downloadable procedure “clustersampsi” from Stata (StataCorp).Effect sizes are difficult to estimate for the InCharge program because the program is newly developed. As school-based prevention programs generally have small effects , we based=0.2) . In ordeThe experimental intervention is a school-based program named InCharge, which has been designed for older adolescents. The main objective of the program is to stimulate healthy lifestyle behaviors, such as increased physical activity, and to decrease unhealthy behaviors, such as binge drinking and eating unhealthy foods. These objectives should mainly be obtained by means of increasing students’ self-regulation skills.The InCharge program consists of four 50-minute lessons, and each lesson contains three assignments of approximately 15 minutes. Teachers are trained on how to work with the program material by prevention professionals from Municipal Health Services or regional institutions for the treatment and care of addiction. Materials, such as worksheets, videos, and a web-based quiz tool, are available on a digital teacher platform. In addition, a specially designed app for the smartphone  is used to help students complete the program.In the first assignment of the first lesson, students visualize their goals for the next 5 years by thinking about what they want to achieve in life and why this is important to them. In the second assignment, students discuss the biggest temptations in their lives. For this assignment, the teacher clears a space in the classroom and draws an imaginary line. Students are asked to position themselves alongside this imaginary line according to how tempting they assess certain temptations, such as chocolate or sweets, alcohol, and staying in bed in the morning. After each temptation, the teacher asks the students to explain their position. In the third assignment, students visualize how their goals would be affected by repeatedly giving in to their biggest temptations or repeatedly resisting their biggest temptations.In the second lesson, the concept of self-regulation is introduced . First, students watch and discuss a video of the marshmallow experiment to understand the role of self-regulation . In thisIn the third lesson, first, the results of the 7-day challenge are discussed. During the second assignment, students reflect on their original plan to complete the challenge and discuss how these plans could further be improved. The third assignment addresses peer pressure, as peers have been shown to strongly influence the health behaviors of adolescents ,32. In oThe final lesson elaborates on alcohol use. At first, students watch a video of partying adolescents who are consuming large amounts of alcohol. The video shows a group of friends who are having fun at first, but the situation ends with a conflict and an injury as a result of excessive alcohol use. Thereafter, students discuss the role of peer pressure in excessive drinking. Second, in order to make them aware of current social norms, students discuss the social acceptability of binge drinking by means of an interactive web-based quiz. Finally, the students identify situations in their own life, where it is important to be careful with alcohol, and come up with strategies to do so. A more thorough description of the assignments can be found in An overview of all measurements is provided in The InCharge program aims to stimulate a healthier lifestyle in general among older adolescents by increasing self-regulation skills. In terms of outcome measures, the study focuses on the following three outcome behaviors: alcohol use, snack intake, and lack of physical activity. A pilot study of the program has shown that these behaviors are most prevalent among older adolescents. The main outcomes that are measured for alcohol use, snack intake, and physical activity are the actual behaviors and predictors of the behaviors. Snack behavior and physical activity are operationalized as the number of times adolescents consume snacks and are physically active in an ordinary week as well as in the past 7 days . AlcoholAdditional measures are general self-control, assessments of various temptations, interpersonal communication about the three health behaviors and the intervention, and evaluation of the program and teacher. First, we assess self-control using an adapted version of the Brief Self-Control Scale , which aObservations are conducted during delivery of the first and fourth lessons of InCharge to provide information on the classroom processes and how the program is delivered  and 3. TThe topic list of the focus groups with students includes interpersonal communication about temptations, evaluations of the InCharge program, and evaluations of the teacher. For interpersonal communication, we investigate the motivation to discuss temptations with the teacher, friends, classmates, and parents, and reasons why students did not discuss temptations with these conversational partners. The content as well as the valence of the conversations about temptations is investigated for interpersonal communication both within class and outside class. For the evaluation of the program, we aim to understand what parts of the intervention students find helpful or not, whether students actively complete in-class and homework assignments, and whether students think the intervention helps them to deal with temptations in the future. Lastly, we ask students about whether their teacher clearly explains the assignments during the intervention, who should ideally be given these interventions, and their perceptions of their teacher’s support for the intervention.The topic list of interviews with teachers includes interpersonal communication, teacher perception of students, and evaluation of InCharge. For interpersonal communication, we assess the content and valence of conversations with students about the InCharge program, both during the intervention and outside class, as well as conversations that teachers have overheard among students. For teacher perception of students, we investigate how teachers perceive students’ participation and enjoyment. Finally, we discuss the health intervention with the teacher in order to find improvements for the program.The background variables of the students  will be checked to assess whether these variables are successfully randomized across the experimental and control conditions. Unsuccessfully randomized variables will be included as covariates in further analyses to control for potential confounding. Our data have a multilevel structure because participants are nested within classes, and these classes are nested within schools. Therefore, participants in one class or school are more similar than participants in other classes or schools. To account for this potential dependency in the data, we will correct standard errors and parameter estimates for clustering.In order to determine the effectiveness of the health intervention, an intention-to-treat procedure and completers-only framework will be used to analyze the data. An intention-to-treat analysis of the data implies that participants will be analyzed under the condition that they are initially assigned, whereas completers-only framework analyzes only participants who are present at all time points. Using multilevel analyses, the three behavioral outcomes  will be compared between participants in the experimental condition and those in the control condition at posttest and follow-up to determine the effects of the program. In the experimental condition only, multilevel analyses will be conducted to investigate the influence of treatment integrity on the predictors of the three health behaviors at posttest and follow-up. For missing data in multilevel models, maximum likelihood estimation will be used. As self-regulation is an important concept in InCharge, we will test whether the potential effects of InCharge on the health behaviors are explained by self-control. Additionally, relations between interpersonal communication with teachers, friends, classmates, and parents  and health-related outcomes, such as a mediating role between the conditions and outcomes, will be tested. As the program is delivered by teachers, we will also test in the experimental condition whether teacher communication relates to the evaluations of the health intervention and the three behavioral outcomes, using structural equation modeling. For structural equation models, expectation maximization will be used to impute the data.For the qualitative data, interviews with teachers and focus groups with students will be analyzed on the strengths and weaknesses of the program in order to adapt the program. Furthermore, qualitative data will be used to determine the content of interpersonal communication with regard to the InCharge program and various temptations.No patient is involved in this study.The trial has been approved by the Ethics Review Board of the Faculty of Social and Behavioral Sciences of the University of Amsterdam (reference no.: 2017-PC-8244), and the trial is registered with the Netherlands Trial Register (NL6654). Manuscripts reporting the effectiveness of the health intervention and our other aims will be submitted to peer-reviewed journals for publication.The recruitment, inclusion, and randomization of participants  started in the spring of 2017 and was continued in the spring of 2018, and data are being collected during the following two consecutive school years: 2017-2018 and 2018-2019. As of June 2019, a total of 1216 participants have been enrolled for this trial. This study is part of a PhD project and is expected to be completed in November 2020.The goal of this paper is to describe the study protocol of a cluster RCT that aims to evaluate the effectiveness of a school-based health intervention named InCharge. The goal of this program is to stimulate healthier lifestyles by improving the self-regulation skills of older adolescents. We expect that adolescents will be more inclined to refrain from binge drinking and eating unhealthy foods and to be more physically active after following the InCharge program compared with adolescents in the control condition.Given that InCharge is a school-based health intervention, the potential to reach a large number of adolescents is an important strength of the program. Second, the underlying mechanisms of InCharge are based on several often successfully tested and relevant theories such as the SCT  and the Our study has some limitations as well. The unit of randomization was classes instead of schools. The presence of both experimental classes and control classes within the same school could result in contamination between conditions because students from the experimental and control condition could have shared information about the intervention. However, as classes within the same school are generally more similar than classes from different schools, we expect that randomization at the class level will result in more comparable research conditions than randomization at the school level. Another limitation relates to the fact that the questionnaire data are obtained through self-reports. A disadvantage of self-reported data is that participants likely want to present themselves favorably, and hence, this might influence how they respond to surveys . One exaThe InCharge study evaluates the effectiveness of the program, and its findings will help to further improve the program. The study also investigates whether including self-regulation as a key component in a school-based health intervention promotes healthier lifestyles among older adolescents. If findings reveal that improving self-regulation skills decreases unhealthy behaviors, such as alcohol use and snack intake, and increases healthy behaviors, such as physical exercise, the findings may aid the developers of health interventions in designing effective behavior change programs for older adolescents. In addition to the content of the intervention, investigating how teacher communication relates to the three health behaviors can provide information on how to communicate for a school-based health intervention. This information can be used to formulate communication guidelines in order to improve the delivery of school-based health interventions and, ultimately, healthier lifestyles. Promoting healthier lifestyles is especially important for older adolescents because unhealthy behaviors generally peak during late adolescence ,5. FinalThis protocol study uses an RCT to assess the effectiveness of InCharge, a newly developed school-based health intervention for older adolescents. As previous research has shown that utilizing self-regulation is an important prevention strategy, the goal of InCharge is to promote healthier behaviors, such as physical activity, and discourage unhealthy behaviors, such as alcohol use, by enhancing self-regulation skills. The program will be improved based on the findings of the effectiveness study.


Enhancing the sustainability of evidence-based prevention programs for mental and behavioral health requires tools for measuring both sustainability determinants and sustainment outcomes. The aim of this study was to develop the Sustainment Measurement System Scale (SMSS) and to assess its reliability and construct validity for measuring both determinants and outcomes of efforts to sustain prevention programs and initiatives.A 42-item scale comprised of items identified from qualitative data collected from 45 representatives of 10 programs and 8 SAMHSA program officers was administered to 186 representatives of 145 programs funded by 7 SAMHSA prevention grant initiatives. Cronbach’s alphas were used to determine inter-item reliability. Convergent validity was assessed by comparisons of a global measure of sustainment with current SAMHSA-funding status and continued operation in the same form. Discriminant validity was assessed by comparisons of sustainability determinants with whether or not the program had undergone adaptations.p < 0.01 to 0.001) and for community-based programs and programs with a substance abuse focus (p < 0.05 to 0.001). Convergent validity was supported by significant associations between the global sustainment measure and current SAMHSA funding status and continued operation in the same form (p < 0.001). Four of the sustainability determinant constructs  were also significantly associated with current SAMHSA funding status (p < 0.5 to 0.01). With the exception of organizational staff capability, all sustainability determinants were unrelated to program adaptation as predicted.Confirmatory factor analysis provided support for a 35-item model fit to the data. Cronbach’s alpha was .84 for the sustainment outcome construct and ranged from .70 to .93 for the sustainability determinant constructs. All of the determinant constructs were significantly associated with sustainment outcome individual and global measures for the entire sample (The SMSS demonstrated good reliability and convergent and discriminant validity in assessing likelihood of sustainment of SAMHSA funded prevention programs and initiatives. The measure demonstrates potential in identifying predictors of program sustainment and as a tool for enhancing the likelihood of successful sustainment through ongoing evaluation and feedback. Contributions to the LiteratureUnderstanding of evidence-based practice sustainment requires improved tools for measurement of determinants and outcomes.The Sustainment Measurement System Scale (SMSS) is the first tool of its kind that is designed to measure both sustainment determinants and outcomes.The SMSS demonstrates good reliability and convergent and discriminant validity.The SMSS can be used to conduct audits and provide feedback to grantees committed to sustaining prevention programs and initiatives once initial funding has ended.Sustainment is considered to be the final stage of the process of implementation of evidence-based practices, policies, and programs (EBPs) , 2. DefiIn recent years, there has been a proliferation of frameworks that focus specifically on sustainability , 4, 7–9.One of the reasons for developing a valid and reliable measure of sustainment is to provide a means of conducting an audit and providing feedback to organizations engaged in implementing innovative and evidence-based programs and practices. While monitoring and feedback are recognized as important for prevention , 20, mucFederal agencies responsible for wide-scale delivery of prevention programs, including the Substance Abuse and Mental Health Services Administration (SAMHSA), routinely collect information from their grantees to monitor progress toward completion of goals and objectives. SAMHSA supports a wide array of prevention grant programs targeting mental, emotional, and behavioral disorders including illicit substance use, suicide, and antisocial behavior. Each of SAMHSA’s prevention initiatives have specific sets of goals and objectives, and each have different prevention approaches that the administration expects will be sustained once support from SAMHSA is no longer available. As part of their initial proposal for funding, all SAMHSA grantees are required to submit a plan for sustainment of the grantee’s activities once the federal funding has come to an end. SAMHSA programs currently rely on electronic data collection systems including the Transformation Accountability (TRAC) data collection system for SAMHSA’s Center for Mental Health Services (CMHS) programs, and the Coalition Online Management and Evaluation Tool (COMET) and the Performance Management Reporting Tool (PMRT) used by SAMHSA’s Center for Substance Abuse Prevention (CSAP). This information is used to provide feedback to grantees when there is evidence of failure to achieve goals and objectives. However, there is no empirical evidence that such feedback leads to an improvement in performance or increases the likelihood of sustainment.While SAMHSA remains deeply concerned about sustainment, its mission does not allow monitoring of its grantees after funding has ended. In building a partnership with SAMHSA, the National Institute on Drug Abuse (NIDA)-funded Center for Prevention Implementation Methodology  responded to this need by undertaking the development of a sustainability measure across diverse prevention programs and test its predictive validity over time . This meThe objective of the current study was to explore and identify dimensions of the Sustainment Measurement System Scale (SMSS) that was designed to assess the sustainment of prevention programs and initiatives by generating quantitative items from qualitative content domains of sustainment outcomes and sustainability determinants and subjecting them to confirmatory factor analysis in order to discern their factor structure. Our goal was to develop a scale that focused on the determinants and outcomes of sustainment of prevention programs and initiatives and to examine its factor structure, reliability, and construct validity.Representatives from grantees supported by the following seven SAMHSA programs took part in the study: (1) the Sober Truth on Preventing Underage Drinking Act (STOP-Act) grants committed to prevent and reduce alcohol use among youth and young adults; (2) the Implementing Evidence-based Prevention Practices in Schools (PPS) grants designed to address the prevention of early childhood behavioral disorders; (3) the Strategic Prevention Framework State Incentive (SPF-SIG) grants to prevent the onset of substance use amongst youth; (4) the Garrett Lee Smith (GLS) State and Tribal Youth Suicide Prevention grants to address depression, suicidal attempts, and behavioral health problems linked to suicide; (5) the Substance Abuse and HIV Prevention Navigator Program for Racial/Ethnic Minorities  grants serving minority populations at-risk for substance use and HIV; (6) Minority Serving Institutions Partnerships with Community-Based Organizations (MSI-CBOs) grants serving communities at risk for substance use, HIV, and hepatitis-C infections; and (7) the Capacity Building Initiative for Substance Abuse (SA) and HIV Prevention Services for At-Risk Racial/Ethnic Minority Youth and Young Adults (HIV-CBI) grants to prevent and reduce the onset of SA and transmission of HIV/AIDS among at-risk populations ages 13–24.Program Officers representing the 7 SAMHSA funding initiatives provided names and email addresses of all project directors and key personal for each organization currently or previously funded under these initiatives. Researchers identified other people to contact from information provided by project directors of all PPS grantees funded by RFA SM-10-017. Names and email addresses for two to three contacts  per GLS grantee site were obtained from ICF international, the GLS contracted evaluators of the grant program, for grantees belonging to Cohorts 8 through 12, per recommendation of SAMHSA program officers. Invitations to participate in the study were sent to 528 representatives of 306 grantee sites supported by the 7 SAMHSA grants between November 2017 and March 2019. One-hundred eighty-six representatives of 145 grantees agreed to participate, resulting in an individual representative response rate of 35.2% and a grantee response rate of 47.5%.The study was approved by the appropriate Institutional Review Boards prior to participant recruitment, and informed consent was obtained prior to administering surveys. Study data were collected and managed using REDCap electronic data capture tools hosted at Northwestern University. REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry, (2) audit trails for tracking data manipulation and export procedures, (3) automated export procedures for seamless data downloads to common statistical packages, and (4) procedures for importing data from external sources .Each participant was emailed an invitation to participate including a link to the web-based survey. Participants reviewed informed consent and after agreeing to participate were able to access the survey and proceed to the survey items. Once participants logged in to the online survey, they were able to answer questions and could pause and resume at any time. The online survey took approximately 15 to 20 min to complete.Item development for the SMSS is described in detail elsewhere . BrieflyThe data sets containing sustainability elements from each of the three components of the interview  were then compared through a process of data triangulation  to identify items that were elicited from more than one data set. Items were then placed into three groups: (1) those that appeared in only one of the three data sets, (2) those that appeared on two of the three data sets, and (3) those that appeared on all three data sets. Four sustainability elements were identified by all three data sets: (1) ongoing coalitions, collaborations, networks, and partnerships; (2) infrastructure and capacity to support sustainability; (3) community need for program; and (4) ongoing evaluation of performance and outcomes). An additional 11 elements were identified by two of three data sets: (1) availability of funding; (2) consistency with organizational culture; (3) evidence of positive outcomes; (4) development of a plan for implementation and sustainment; (5) presence of a champion; (6) institutionalization and integration of program; (7) institutional support and commitment; (8) community buy-in and support; (9) program continuity; (10) supportive leadership; and (11) opportunities for staff training .Sustainment outcomes: 4 items assessing the continued operation of the program or initiative, including delivering prevention services to intended population that are evidence-based as described in the original application for funding and periodically measuring service fidelity.Financial stability: 6 items measuring funding from federal, state, or local governments and non-profit and non-governmental sources, combination of earmarked and discretionary funding, sustained funding, financial support from diverse community interests, and financial solvency.Responsiveness to community needs and values: 7 items measuring meeting needs of intended target populations and behavioral health needs of communities/populations being served, adaptability to meet these needs, consistency with norms and values of participating organizations, fit with values of sustaining organizations and communities, shared perception of project importance by participating organizations, and unacceptability of public health problem addressed by project.Coalitions, partnerships, and networks: 8 items measuring networking of grantee organization with other organizations committed to program sustainability, community engagement in development of project goals, community access to knowledge and information about the project, project support by a coalition/partnership/network of community organizatons, network expansion, commitment to continued operation of project, level of networking, and communications within organizations responsible for sustaining the project.Infrastructure and capacity to support sustainment: 9 items measuring available resources for project implementation and sustainment, integration into operations of the organization and partners, advanced development of plans for implementing and sustaining the project, execution of the project according to these plans, adequacy of staff to sustain program goals and activities, sufficiency of training available to staff and community members, staff knowledge and supportive beliefs, and staff self-efficacy to implement the project.Implementation leadership: 5 items measuring active engagement of leaders in project implementation and sustainment, involvement of community leaders in the project, appointment of someone responsible for coordinating project implementation and sustainment, support from a program champion, and process in place for sustainment in the event the champion leaves.Monitoring, evaluation, and program outcomes: 3 items measuring ongoing evaluation of progress made toward sustainment, sufficiency, and timeliness of feedback about project delivery and quality improvement, and evidence of positive outcomes.To construct this survey, the fifteen elements were condensed to create a 42-item scale divided into the following subscales:Study participants were asked to indicate level of agreement with a series of statements using a Likert scale ranging from 1  to 5  for the four items contained in one subscale measuring sustainment outcomes and seven different domains of determinants of sustainment; lower scores on all subscales indicate lower levels of agreement, while higher scores indicate higher levels of agreement with the respective statements. Each subscale score is represented as an average of the scores for each item included in the subscale. The average of the scores for each item included in the Sustainment Outcomes subscale was defined as Global Sustainment.Two sets of variables measuring characteristics of the funded programs and initiatives were included in the analysis to determine whether the association between sustainability determinants and sustainment outcome varied by one or more program characteristics and to assess the convergent and discriminant validity of the SMSS. The first set included a categorical variable describing program type (community or state) and a categorical variable describing program focus . The second set also included two variables, one describing whether the grantee is currently funded by one of the seven SAMHSA grant initiatives (yes or no), and if not, a variable describing the current status of the grantee’s program .To evaluate the psychometric properties of the SMSS, confirmatory factor analyses (CFA) with maximum likelihood estimation were conducted using EQS statistical software , 45. TheReliability of the SMSS was assessed by examining Cronbach’s alpha internal consistency for each of the subscales and the total scale. Cronbach’s alpha of .70 and above is considered good internal consistency for a newly developed scale . CorrectConstruct validity of an instrument is typically assessed by comparing constructs or measurements that theoretically should be related to one another to determine if that is the case  and by comparing concepts or measurements that are not supposed to be related to determine if they are actually unrelated  . In thisThere were no missing values for any of the 4 sustainment outcome variables; however, missing data for each of the determinant variables ranged from 15.8 to 17.9%. The same 29 to 31 individual grantee representatives did not provide responses to any of the determinant items and were removed from analyses of sustainability determinants by sustainment outcomes or determinants by current funding status or program status. As a result, 113 (77.9%) of grantees were represented by one individual, 26 (17.9%) grantees were represented by two individuals, and 6 grantees were represented by 3 (4.1%) individuals. A flow chart describing the numbers of participants from recruitment to data collection to data analysis is presented in Fig. Details of participating representatives and programs used in the data analysis are provided in Table χ2(517) = 960.23, p < .001, an RMSEA of .076 and an SRMR of .066 meeting criteria for acceptability; however, fit was not supported by a CFI of .95 or greater. The standardized factor loadings ranged from .56 to .88, and all were statistically significant . Separating the responsiveness to community needs and values subscale into two separate subscales also increased the Cronbach’s alpha coefficient from .60 to .70 for both subscales. The 35-item version of the SMSS is provided in the Results of the CFA analyses are presented in Table r = .19) were lower than the correlations among the other eight subscales (average r = .42). Of particular note, sustainment outcome was significantly associated with all 8 determinants (average r = .43), with significance levels < 0.001 except for financial stability (p < 0.01).Table Table F = 37.67, d.f. = 154, p < 0.001) as well as the four individual forms of sustainment. They also reported significantly less responsiveness to community needs , coalitions, partnerships, and networks , organizational staff capability , and evaluation, feedback, and outcomes .A comparison of sustainment determinants and outcomes by program current funding status in provided in Table F = 4.52, d.f. = 146, p = 0.035), suggesting strong support for the discriminant validity of the SMSS.The discriminant validity of the SMSS was assessed in comparisons of total sustainment outcomes and determinants by program status. The results are presented in Table Although there exist other measures designed to evaluate sustainability of evidence-based programs and interventions from the point of view of the determinants of sustainability or sustainment as an outcome, the Sustainment Measurement System Scale (SMSS) is the first instrument designed to assess both sustainability determinants and sustainment outcomes. Further, the SMSS is designed to assess different prevention programs of different types and foci. As such, it contains elements that are specific to particular programs but enable comparisons across different types of programs. It also demonstrates some potential as a tool for providing feedback to organizations, enabling them to monitor their trajectory towards achieving the final stage of state of implementation and increase the likelihood of successfully doing so.The SMSS draws from and shares several features of existing measures of sustainability. For instance, like the Stages of Implementation Completion Scale , the SMSThe SMSS is the product of a standardized process of eliciting determinants and outcomes of sustainability that are both specific to the program being sustained and generalizable to other types of programs and initiatives . This apThe design and evaluation of the SMSS also provides some important insights into sustainment as the final stage of the implementation of evidence-based prevention programs and initiatives. The process of achieving sustainment is the product of eight domains of sustainability determinants: financial stability; responsiveness to community needs; responsiveness to community values; coalitions, partnerships, and networks; organizational capacity; organizational staff capability; implementation leadership; and evaluation and feedback and positive program outcomes. Luke and colleagues  found siNevertheless, there are a number of factors that limit the findings of this study. The findings reflect the experience of specific prevention programs and initiatives that have been or are currently funded by SAMHSA; thus, their generalizability to prevention programs funded by other sources , to programs targeting treatment, or event to other SAMHSA-funded programs, is limited. The confirmatory factor analysis was constrained by the small number of programs and program representatives. Only two items loaded onto one of the identified constructs (responsiveness to community needs), while only three items loaded onto the construct of responsiveness to community values. Further research is required to determine whether or not they represent two distinct constructs.With these limitations in mind, future steps include the following: (1) evaluation of the utility of the SMSS with other types of programs in other settings, (2) use of larger samples to confirm fit of the data to the model, (3) development of guidelines for providing feedback to organizations seeking to sustain programs and initiatives based on ongoing monitoring efforts using the SMSS, and (4) consistent with the SMS process, ongoing revision of the tool itself.The SMSS is innovative in three specific respects. First, it draws upon the experience of evaluating sustainability in different types of prevention programs and initiatives with different aims and areas of emphases. This enabled us to identify a set of common elements of sustainment that can be used to generate a model and set of testable hypotheses that apply to a broad array of substance use/mental disorder/suicide prevention programs, practices and initiatives, regardless of objectives, outcomes, and infrastructure to achieve these outcomes. Second, although we did not assess the effectiveness of the various programs, practices, and initiatives participating in this study, a measure like the SMSS offers the potential to determine whether the extent to which a program or practice is evidence-based or evidence-informed determines whether it can be sustained. Such information would be invaluable to determining whether the program or practice merits initial or continued funding. Third, this measure can be used to monitor progress toward sustainment and provide feedback to stakeholders as to how to increase the likelihood of sustainment. The SMSS in particular and the SMS in general can be used as a tool for program management as well as research purposes. Although the SMSS was based on programs funded by SAMHSA, the instrument should have general applicability across diverse federal, statewide, and local prevention implementation initiatives.


In anger control ability and gratitude, there was no significant difference between groups . This program for emotion management, which is based on the broaden-and-build theory, can improve subjective happiness, anger control ability, and gratitude, by reinforcing appreciation skills and anger awareness and self-expression in male high school students.This study aimed to develop an emotion management program for late adolescent males, and investigate the effect of the program. This study is an experimental study using a pretest-posttest control group design. It was conducted from 14 May to 21 December, 2018 at a male high school in Korea. It analyzed 52 participants  Regarding subjective happiness, there was a significant difference between groups (t = 3.409,  Happiness is characterized by psychological well-being, subjective well-being, and life satisfaction, and these aspects are used as indicators of happiness from a quality of life perspective . AccordiAccording to the Youth Health Behavior Online Survey , 28.7% oFactors that influence adolescents’ happiness include adolescents’ self-understanding and personal psychological variables . Among tOn the other hand, positive thinking, defined as an attitude that promotes positive experience of life and appropriate human growth, has a cognitive feature that creates behavioral changes, and thus can impact anger and anger management, which are influenced by the cognitive domain . FurtherThe broaden-and-build theory of positive emotions states that experience of positive emotions broadens people’s attention, thoughts, and range of actions , enablinTherefore, this study aims to develop an emotion management program for adolescents based on the broaden-and-build theory, and to evaluate its effects on subjective happiness, anger control, and gratitude to help adolescents understand and appropriately manage their emotional states such that they can positively cope with stress situations and prevent potential problems.The purpose of this study is to develop an emotion management program to strengthen positive emotions and evaluate the program’s effects on subjective happiness, anger control, and gratitude among high school students.This study is an experimental study using a pretest-posttest control group design.The study population consisted of students of a male high school in Korea, who repeatedly underwent personal counseling with their homeroom teachers and school nurses due to problematic behaviors or were deemed in need of emotion management due to a possibility of exposure to crisis. This study was conducted on students who were vulnerable to a crisis or were in need of emotion management based on the recommendation of teachers who understood the purpose of the study. These students included those who had unexcused absences, who were involved in school violence, or who had difficult peer relationships. The inclusion criteria were as follows: high school students who consented to participate in the study, were capable of completing the questionnaire, and did not have diagnosed mental problems or other diseases. Of the 87 recruits, 74 students who met the inclusion criteria and consented to participate in the study were randomly assigned numbers, and the participants were divided into the intervention group and control group based on odd and even numbers, with 37 in each group. After excluding one participant in the control group who was withdrawn from the study for not participating in the post-program survey, a total of 37 participants in the intervention group and 36 participants in the control group were included in the final analysis.Twenty-seven items, including items for demographic characteristics, were used in this study, and instruments validated for use on adolescents were used with permission from the author.Subjective happiness: The Subjective Happiness Scale (SHS) developed by Lyubomirsky and Lepper  and adapAnger control ability: Anger control was assessed using the scale developed by Morganett  for use Gratitude: Gratitude was assessed using the Korean version of the Gratitude Questionnaire-6 (GQ-6), which was originally developed by McCullough, Emmons, and Tsang  and adapThis study was performed from May to December 2018. The control group and intervention group completed the questionnaire asking about their demographics and subjective happiness, gratitude and anger control prior to the intervention, and the data were collected by the researcher.The emotion management program is a group program that was developed based on the broaden-and-build theory , which bFor the components of the program, we referred to the strength of gratitude, which induces positive changes through a preceding factor that calms down anger and complaints and increases self-expression, as well as to the property of gratitude that enables phased development through training and habit formation . We inteEach session consisted of a recollection of the previous session, presentation of a gratitude journal to relax the mood, instructions about the current session, activities relevant to the topic, and wrapping up. During wrap up, participants held one another’s hands for a gratitude relay to practice positive emotional skills. After each session, feedback was provided by the session evaluation sheet. After the program, we conducted interviews with each participant. The interview questions included, “How did you think about gratitude in the past”?, and “what changes in your thoughts or behaviors about gratitude did you experience after participating in the program”? This program was administered during free study hours and not during the regular school classes, and each session lasted 50 minutes. The outline of the emotion management program is shown in Prior to the main program, we conducted a pilot study on 15 qualified students to assess the construction and methodology of the program. These 15 students are chosen at random from the recommended students by the teacher. Based on the results, we modified a part of the program and adjusted the duration of each session and other details.We surveyed participants’ subjective happiness, anger control, and gratitude two weeks after the completion of the program in the intervention group, and the same post-program survey was conducted on the control group at the same time. The delay of two weeks was intentionally given to the participants before the post-program survey to allow for their gratitude habits to form after the program. The collected data were processed by assigning a number for each participant per the guidelines for the processing of personal information, and the participants were given a small gift at data collection.t-test and the Mann-Whitney U test. To analyze the effects of the intervention program, normally distributed gratitude and subjective happiness were analyzed with an independent t-test and non-normally distributed anger control was analyzed with the Mann-Whitney U test. Statistical significance was set at 0.05.The collected data were analyzed using SPSS 25.0 software . Demographic characteristics were presented as percentages, and the homogeneity of demographic characteristics between the intervention group and control group was tested with Fisher’s exact test and Chi-square test. Normal distribution of the study variables gratitude, subjective happiness, and anger control was tested with the Shapiro-Wilk test. Homogeneity of the study variables between the two groups was analyzed with an independent This study was approved by the institutional review board of Dongyang University (1041495-201703-HR-04-01) and was performed from May to December, 2018. We adequately informed the participants of the purpose of the study, study procedure and contents, use of personal information, benefits and drawbacks anticipated by study participation, confidentiality, and ability to withdraw from the study at any time, and obtained written informed consent.There were no significant differences in age, living with parents, living with grandparents, siblings, economic status, and religion between the intervention and control groups, confirming the homogeneity of the two groups .There were no significant differences in subjective happiness, anger control, and gratitude between the two groups before the program .p = 0.001). The mean subjective happiness score increased by 0.65 after the intervention in the intervention group while it decreased by 0.08 in the control group, showing a significant difference . The mean anger control score decreased by 0.00 in the intervention group and by 0.03 in the control group, with no significant difference between the two groups . The mean gratitude score increased by 0.44 in the intervention group and by 0.31 in the control group, with no significant difference between the two groups (There was no significant difference in the mean gratitude score between the two groups (z = 0.528, o groups .This study aimed to develop an emotion management program and examine the effects of the program on subjective happiness, anger control, and gratitude in high school students, who are experiencing adolescence and an array of changes.Subjective happiness increased in the intervention group after the intervention compared to the baseline, showing a statistically significant difference between the intervention group and control group. This can be interpreted in the same context as the greater effects of a group program on happiness compared to positive emotions, psychological wellbeing, self-respect, and depression in the study by Yoo and Son . SubjectThe intervention group performed activities involving recognition and expression of positive emotions with peers. After the end of the program, the participants stated that expressing gratitude to friends was amazing and impressive and that they became closer with their friends by verbally thanking them once a day. Moreover, they were able to positively understand themselves and discover their values through activities such as coming up with desired nicknames and setting a price for their strengths, weaknesses, hidden talents, and dreams. Discovering the value of themselves as well as others and adopting a respectful attitude through the presentations and sharing their learning with the participants is speculated to have contributed to boosting their subjective happiness. Further, we can surmise that the program contributed to alleviating their tension from excessive academic stress caused by regular coursework and additional learnings, and inducing catharsis.The emotion management program developed in this study was designed to help participants recognize and express their emotions and practice positive emotional skills using the properties of gratitude, and is similar to Thompson’s emotional self-regulation . The goaGratitude refers to a generalized tendency of recognizing others’ contributions to positive experiences and outcomes and responding with a grateful heart , and it Our results showed that there was no significant difference in anger control between the intervention group and control group. This is consistent with the results of Lee and Yang  but contIn our program, anger management was dealt only in one session, so it is possible that the number of sessions directly dealing with anger was not enough for students to recognize and control their anger. This is evident in the post-program statements of the intervention group, “At first, I was so embarrassed to talk about myself in front of other people and I was frustrated because I didn’t know what to say”. We can speculate that adolescents might have had difficulty concentrating in the anger management session due to the feeling of awkwardness they had during activities encouraging them to verbally and behaviorally express their emotions. Therefore, a sufficient number of sessions should be allocated for anger exploration and management in the future. Additionally, considering that adolescents feel awkward expressing themselves, phased and individualized approaches are needed to help them naturally recognize and control anger.Candelaria, Fedewa, and Ahn  reportedThe results obtained from this study cannot be generalized because the subjects of the study were the students recommended for the emotional intervention program by a single High School. In addition, the different reasons for each recommendation was not considered in the program design. The study has limitations because the intervention program was provided without thoroughly assessing the individual psychological status of the participating students. The sample size was computed using the G-power 3.1 program. For a comparison of means between two groups, the required sample size for an α of 0.05, power of 0.80, and a medium effect size d of 0.80 was 64 for the intervention group and 64 for the control group. The choices of α, power, and medium effect size are based on Moltrecht et al. , which jBased on our results, it would be more effective to develop emotion management programs based on appropriate peer interactions, and, as an array of variables influence adolescents’ emotions, these programs should adequately consider personal and environmental factors. Considering that emotions in adolescence motivate behavior and that positive emotions and happiness experienced during growth and development can have a persistent impact on their future lives, emotion management programs for adolescents should be designed as a continuous and linked program, as opposed to a one-time program, and education for school teachers is crucial to properly manage adolescents before problem behaviors surface. Moreover, emotion management should be naturally trained through the school curriculum to help adolescents maintain and promote healthy psychological emotions, and, to enable effective implementation in the curriculum, various strategies should be explored and the programs should be linked to and run in collaboration with local communities.This study aimed to develop and validate the effects of an emotion management program to help male late high-school adolescents with their daily lives and school adaptation. The results showed that the intervention had a significant elevation of subjective happiness, but no significant influence on their gratitude and anger control. This result suggests that the emotion management program was partially effective.This study also suggests the need for further research that utilizes various personal, friend, school, and community-related variables to produce positive effects and sophisticated study designs that can contribute to improving both gratitude and anger control. We also suggest to include more number of participants in the study to satisfy the statistical requirements for all the variables under investigation.