Invest in Kids (IIK) is a Denver, Colorado, USA-based intermediary organization that works to bridge the research-to-practice gap for programs that support Colorado’s youngest children and their families. IIK supports evidence-based programs (EBPs) including three universal, prevention programs from The Incredible Years® (IY)—Dinosaur School (a classroom curriculum), Teacher Classroom Management , and Parent Program (parent training). IIK employs staff (the IIK–IY Team) to deliver implementation supports such as training, coaching, and managing implementation teams with professionals at schools and community agencies who deliver IY programs throughout Colorado. The COVID-19 pandemic presented challenges and opportunities for practitioners using EBPs, and for the IIK–IY Team. The Framework for Reporting Adaptations and Modifications—Expanded, known as the FRAME , was identified as a useful tool for documenting information about COVID-19-related adaptations to local sites’ delivery of the programs, IIK’s implementation supports, and IIK’s annual statewide evaluation of IY. This case study includes an in-depth description of the various adaptations made by the IIK–IY Team, highlighting specific examples that demonstrate how the FRAME can be used to support adaptations for numerous sites in a geographic region or state. This case study provides important lessons about what successful IY delivery and implementation supports looked like in the context of the ongoing COVID-19 pandemic, including a description of IY fidelity and program outcomes.The online version contains supplementary material available at 10.1007/s43477-022-00048-1. A central challenge of delivering evidence-based programs (EBPs) in community settings is maintaining fidelity to the program model and achieving outcomes comparable to those found in carefully controlled research settings  is a Denver, Colorado, USA-based intermediary organization that partners with communities statewide to adopt, implement, and successfully scale EBPs for Colorado’s youngest children and their families. IIK supports Nurse-Family Partnership®, Child First®, and three universal prevention programs from The Incredible Years® (IY)—Dinosaur School (a classroom curriculum), Teacher Classroom Management , and the Preschool BASIC Parent Program . The suite of IY programs focuses on strengthening parent–child, teacher–child, and home–school connections that provide a strong foundation for young children’s social and emotional development , IIK partnered with communities in 24 of Colorado’s 64 counties to support Dinosaur School, Teacher Classroom Management, and Parent Program for 7133 students , 516 teachers and educational staff, as well as 94 Parent Program facilitators and 746 Parent Program participants. At the onset of the COVID-19 pandemic in March 2020, the shift to remote work for IIK staff members, the closure of Colorado’s schools for in-person learning and transition to virtual learning for teachers and students, as well as public health mandates restricting in-person gatherings, had widespread implications for the delivery of the IY programs and IIK’s implementation supports. The IIK–IY Team led with flexibility and responsiveness to the evolving public health crisis and changing needs of program implementers and participants, including teachers, parents, and students. At the same time, the IIK–IY Team prioritized supporting the transition of program delivery to virtual spaces through the end of the program year.In the early weeks and months of the pandemic, it was unclear how long it would be before IIK and the schools and community agencies across Colorado delivering IY would return to business as usual. In August 2020, as the IIK–IY Team was busy preparing for the 2020–2021 program year, it became clear that a longer-term plan for supporting and delivering the IY programs in a virtual world was necessary. This shift involved a meeting with the program developer to tailor the core elements of the IY programs to fit a virtual context, making alterations to IIK’s implementation support, and adjustments to IIK’s annual statewide evaluation of IY. Given the number of alterations needed, and the scale and pace at which alterations had to be supported, the IIK–IY Team wanted to remain thoughtful and intentional about planning for, measuring, and monitoring such alterations. As stewards of IY, an underlying goal for the IIK–IY Team was to make appropriate, responsive alterations while also maintaining a high level of fidelity. With an eye toward documenting the alterations for the 2020–2021 program year while continuing to monitor IY fidelity and program outcomes, the FRAME . The goal of modifying the coaching context was to improve the feasibility of coaching despite virtual requirements and to improve the effectiveness of the program in the virtual context. The contextual adaptation to the coaching format and setting was to shift exclusively to virtual observations and coaching, which meant that there was no real-time, in-person modeling of concepts with the teacher.For the IIK–IY process evaluation, the shift to virtual coaching and the use of in-person, hybrid, and/or virtual delivery formats for teachers during the 2020–2021 program year led to consideration of whether adaptations were needed in order to resume assessing teachers’ Dinosaur School fidelity. The intent of the IIK–IY Team was to assess adherence to the core program elements and monitor teacher fidelity throughout the program year across all program delivery formats.Prior to the onset of the pandemic, the IIK–IY Dinosaur School fidelity checklist was used, consisting of 18 fidelity items that were rated on a 5-point Likert scale, with higher scores indicating a greater degree of fidelity to each item on the checklist. This unpublished checklist was previously developed by the IIK–IY Team to assess and support ongoing monitoring of teachers’ fidelity. There was a rating scale and a series of guide statements for each item, and the score for each fidelity item was based on the number of guide statements the IIK–IY Team member observed as completed by the teacher during the coaching visit. Example items from the fidelity checklist include “Teacher uses developmentally appropriate content in Dinosaur School large group,” and “Teacher integrates Dinosaur School visuals and/or social–emotional visuals and materials into their large group Dinosaur School lessons.” The IIK–IY Team completed this checklist three times throughout the school year .For the 2020–2021 program year, the IIK–IY Team made the decision to revise the existing Dinosaur School fidelity checklist in order to improve the fit of the checklist for assessing teacher fidelity across different program delivery formats. The proactive process to update the Dinosaur School fidelity checklist involved the IIK–IY Team examining each fidelity item and the corresponding guide statements, and identifying which fidelity items and guide statements could be assessed regardless of program delivery format. Most of the fidelity items and guide statements already met this criterion. For the few fidelity items and guide statements that did not meet this criterion, the wording was tailored so that they could still be evaluated across each program delivery format. The IIK–IY Team also identified a small number of guide statements that were only applicable to in-person program delivery, and this information was indicated in the updated fidelity checklist. In addition, a couple of the Dinosaur School fidelity items were reformatted as open-ended questions, in order to gather qualitative information about what teacher delivery of that item looked like across the different program delivery formats.With the goal of making it easier for the IIK–IY Team to rate teacher fidelity, it was decided that the fidelity checklist rating scale would also be revised. Rather than using a 5-point Likert rating scale based on the number of guide statements observed, there were instead two dichotomous response options for each fidelity item. The first question was whether the fidelity item was observed, with the response options, “yes” or “no.” The second question was to what extent the fidelity item was delivered, with the response options “full” or “partial.” “Full” indicated that all of the guide statements were completed for that fidelity item, and “partial” indicated anything less than all guide statements had been completed. The modified Dinosaur School fidelity checklist consisted of 20 total items that were fairly similar to the previous checklist, with the revised rating scale being the most substantial adaptation. See Online Resource 2 for a list of the 2020–2021 IIK–IY Dinosaur School fidelity items.Another aspect of the IIK–IY process evaluation involves the IIK–IY Team’s coaching visits. These coaching visits and key details are recorded by the IIK–IY Team and tracked in an online platform. Two questions were added to the 2020–2021 visit tracking forms to capture essential details about the program delivery format and virtual coaching at each visit with second-year Dinosaur School teachers. The first question was about the specific virtual coaching modality used to support the teacher  and the second question was about the teacher’s program delivery format . The goal of adding these questions was to monitor Dinosaur School coaching visits, accounting for program delivery format.The variability in program delivery format also necessitated adaptations to the IIK–IY Dinosaur School student outcomes evaluation. As part of this annual pre-post evaluation, teachers complete demographic information about each student, and two different measures of student outcomes at the beginning and end of the program year. The Dinosaur School student pre–post-outcome measures include 25 items that make up the Social Competence Scale-Teacher (SCS-T) Report , would be asked to fill out the second, shorter measure for each of their students.Ahead of the fall 2020 pre-test data collection, the IY Data and Evaluation Manager added questions to the pre-test about teachers’ plans for Dinosaur School program delivery during the 2020–2021 school year. These questions were designed to gather information about the timing, frequency, and format of Dinosaur School delivery at the start of the school year. Questions were also added that asked about how often teachers planned to deliver the core Dinosaur School fidelity elements. In the spring of 2021, the IY Data and Evaluation Manager added questions to the teacher post-test that requested qualitative feedback about successes and challenges delivering the Dinosaur School program during the 2020–2021 school year. Additional questions designed to capture the context of program delivery were added to the post-test, including the teaching format that was used for the majority of the school year , whether teachers and students had to transition between in-person and virtual learning at any point throughout the school year, and the Dinosaur School delivery format.It is important to note that IIK routinely collects IY outcomes and fidelity data as part of a continuous quality improvement process. The information presented in this case study is based on de-identified data and analyses that were previously performed and documented as part of the 2020–2021 IIK–IY evaluation in Colorado. Thus, there was no review by an Institutional Review Board. The IIK–IY Team partnered with Colorado communities across 13 counties to support the delivery of Dinosaur School for 4027 students and 315 teachers during the 2020–2021 program year. See Online Resource 3 for demographic information about the 2020–2021 Dinosaur School students and teachers.t-test analysis revealed that, on average, there was a statistically significant difference between the pre- and the post-test for the Dinosaur School SCS-T overall score and the Dinosaur School Social–Emotional Skills score, for all program delivery formats. On average, students who received virtual, hybrid, or in-person Dinosaur School lessons demonstrated significantly higher levels of social competence and Dinosaur School curriculum-related skills at the end of the school year, compared to the beginning of the school year. The size of the pre–post-difference for the SCS-T overall score was quite similar, on average (within a tenth of a point), for students who had received hybrid or in-person Dinosaur School for the majority of the school year. The size of the pre–post-difference for Dinosaur School Social–Emotional Skills yielded an overall score that was within a tenth of a point, on average, regardless of whether the program format was virtual, hybrid, or in-person. The adaptations to the Dinosaur School outcomes evaluation enabled IIK’s measurement of students’ Dinosaur School outcomes in context.Table The assessment of Dinosaur School teachers’ successes and challenges during the 2020–2021 program year provides additional context to the quantitative student outcomes, as well as the program adaptations. A content analysis of teacher’s open-ended feedback revealed that overwhelmingly, teachers perceived a high level of student engagement during Dinosaur School lessons, even for students who participated virtually. Regardless of program format, teachers described how much growth they saw in students’ Dinosaur School-related skills, including the ability to regulate emotions, use Dinosaur School language, and solve problems. Teachers also shared their perception that in some cases, the program helped to alleviate students’ challenging behaviors. Some noted that the Dinosaur School program content was especially well suited for supporting students’ social–emotional development in the context of the pandemic.Though some teachers mentioned that they experienced success delivering the program virtually, a considerable number of them indicated that delivering the program virtually, and/or having to transition between in-person and virtual program delivery was a challenge. In particular, it was difficult for some teachers to figure out how to use technology and deliver some of the fidelity elements virtually, such as small group lessons and role plays. Regardless of program delivery format, teachers noted that the overall context of the pandemic was taxing, as there were related issues with staffing, student attendance, and challenging student behaviors. Many teachers also reported less time to spend on Dinosaur School, which affected the frequency and duration of the lessons. Taken together, the transition to virtual Dinosaur School and related issues delivering the program in this context were the biggest barriers for teachers, but they also reported a great deal of success in terms of students’ continued engagement with the program and their growth in curriculum-related skills. It seems that teachers experienced more difficulty maintaining fidelity delivering the program virtually, but they still saw positive outcomes for students, which is consistent with the quantitative student pre–post-outcomes.Adaptations to the IIK–IY Dinosaur School fidelity checklist allowed the IIK–IY Team to monitor Dinosaur School teachers’ fidelity at three different timepoints throughout the 2020–2021 school year. The number of teachers that were observed delivering the program in person increased at each checklist, which reflects the increase in the number of schools across the year that transitioned back to in-person learning. Overall, there were a small number of teachers observed doing virtual or hybrid Dinosaur School compared to the number of teachers that were observed delivering the program in person at each timepoint. On average, the IIK–IY Team observed roughly 83% of all fidelity items for in-person Dinosaur School teachers at each checklist. For teachers delivering Dinosaur School virtually, the IIK–IY Team observed, on average, 76% of fidelity items at checklist one, 73% of fidelity items at checklist two, and 72% of fidelity items at checklist three. On average, the IIK–IY Team observed about 80% of the fidelity items at checklist two and 94% of the items at checklist three for teachers that were using a hybrid model to deliver Dinosaur School. See Table Parent Program adaptations for the 2020–2021 program year involved contextual changes, with Parent Program facilitators meeting with groups of participants in person or virtually, depending on the policies of the host agency and local COVID-19 guidance. At the start of the 2020–2021 program year, almost all of the Colorado schools and agencies that were hosting Parent Program planned to offer it virtually, and in consultation with the IY program developer, the IIK–IY Team remained committed to supporting Parent Program. Accordingly, the setting in which families were virtually engaging with the program was at home, instead of attending the in-person group at a common location, such as a school or trusted community agency.There were several adaptations to the program content; all were promoted by the program developer during summer 2020. First, virtual weekly sessions were condensed and shortened to 90 minutes, in contrast with two hours for an in-person session. With less time for the Parent Program facilitators to cover each week’s topics and objectives, the amount of content and activities covered during each session was reduced. For example, two of the core fidelity elements for Parent Program involve video vignettes that showcase examples of different parenting scenarios, and role plays where parents apply program strategies. Due to the shorter duration of each virtual session, parents were exposed to fewer vignettes and role plays. There was also a reduced group size requirement, with only six to eight Parent Program participants for a virtual group, in contrast with 7 to 15 participants for in-person groups. Given the pervasive and traumatic nature of the ongoing pandemic, facilitators were also encouraged to spend additional time checking in on families, emphasizing self-care and stress management. Thus, the structure of the group sessions was loosened to accommodate time spent checking in with families.During the IIK–IY Team’s discussions with the program developer, it was determined that a couple of the core fidelity elements, including the provision of meals and childcare for families, would be markedly different for virtual groups. Prior to the onset of the pandemic, in-person groups provided a free, family-style meal for the Parent Program facilitators, Parent Program participants, and their children, and free childcare was available on site for families to use while they participated in the group. Although the program developer was aware of the need to be flexible with these fidelity elements, it was also acknowledged that the removal of these elements was inconsistent with the program model. Nevertheless, the goals of the IIK–IY Parent Program content and contextual adaptations were to improve feasibility of virtual program delivery and maintain the reach and engagement of parents in the program amid the pandemic.During the summer of 2020 and the spring of 2021, the IIK–IY Team safely offered in-person training for new Parent Program facilitators in accordance with state and local public health mandates. The Parent Program training content, materials, and activities were consistent with prior years, but there was time dedicated to discussing guidelines for how to deliver the program virtually, including how to set up and facilitate role plays, how to show the video vignettes, and how to use incentives with parents. Parent Program coaching was only available virtually. The setting shifted from in-person observations of the parent group followed by a coaching conversation between the IIK–IY Team member and the facilitators, to virtual observations and coaching of the parent group by the IIK–IY Team member who was joining from home.Prior to the pandemic, the IIK–IY Team offered coaching to Parent Program facilitators based on their level of experience delivering the program. In anticipation of the steep learning curve for Parent Program facilitators delivering the Parent Program virtually, the team decided to offer coaching for all interested facilitators during the 2020–2021 program year. The IIK–IY Team also decided to augment their coaching support by providing more assistance to facilitators around preparation and planning, given all the logistics that needed to be considered to ensure a successful virtual parent group. A series of optional, virtual support groups with the IIK–IY Team were also available for Parent Program facilitators to join throughout the 2020–2021 program year so that they could troubleshoot challenges they experienced delivering the program in a virtual setting. The IIK–IY Team created resource videos so that facilitators could see examples of how to integrate core fidelity elements in the virtual setting, such as how to set up a virtual role play for participants. The IIK–IY Parent Program coaching protocol was tailored so that facilitators would receive coaching on all core fidelity elements when previously, the specific fidelity elements that the facilitators received coaching on depended on the facilitators’ levels of experience. A new point of emphasis was added to the coaching protocol, which involved supporting facilitators to intentionally connect with participants about their stress levels and self-care. Collectively, the goals of the IIK–IY implementation support adaptations for Parent Program were to improve feasibility of delivering the program virtually and ensuring the effectiveness of the program in the virtual context.Adaptations to the IIK–IY Parent Program process evaluation followed a similar procedure as to what was previously described for IIK–IY Dinosaur School. During the 2019–2020 program year and prior to the onset of the pandemic, the IIK–IY Team used the IIK–IY Parent Program fidelity checklist up to two times throughout each 14-week parent group to assess Parent Program facilitators’ fidelity to the program model. This unpublished checklist was previously developed by the IIK–IY Team. The Parent Program fidelity checklist consisted of 24 fidelity items that were rated on a 5-point Likert scale, with higher scores indicating a greater degree of fidelity. The score for each fidelity item was based on the number of guide statements the IIK–IY Team member observed as completed by the Parent Program facilitators during the coaching visit. Example items from the Parent Program fidelity checklist include “Parent Program facilitators use a variety of strategies that meet parent needs and keep parents engaged,” and “Parent Program facilitators ensure that homework discussion/wrap up occurs at the end of each session.” This checklist was also developed specifically for in-person program delivery. The IIK–IY Team made the decision to modify the Parent Program fidelity checklist in order to improve the fit of the checklist for virtually assessing Parent Program facilitators’ program fidelity for virtual and in-person program delivery.The IIK–IY Team examined each of the Parent Program fidelity checklist items and the corresponding guide statements. For the few fidelity items and guide statements that did not fit both in-person and virtual delivery formats, the wording was either tailored so that the fidelity item or guide statement could still be evaluated regardless of program delivery format, or its applicability to only an in-person delivery format was recorded in the checklist. In addition, a handful of Parent Program fidelity items were reformatted as open-ended questions, in order to gather qualitative information about what Parent Program facilitators’ delivery of that fidelity item looked like for virtual and in-person program delivery.The Parent Program fidelity checklist rating scale was revised in order to make it more feasible for the IIK–IY Team to rate Parent Program facilitators’ fidelity for in-person and virtual program delivery contexts. The updated rating scale was formatted to be consistent with the updated Dinosaur School fidelity rating scale, which included two dichotomous response options for each fidelity item, including whether the fidelity item was observed (“yes” or “no”), and the extent to which the fidelity item was delivered, if observed . The updated Parent Program fidelity checklist consisted of 25 total items that were similar to the earlier version, with the revised rating scale being the most significant adaptation. See Online Resource 4 for a list of the 2020–2021 IIK–IY Parent Program fidelity items.The IIK–IY process evaluation for Parent Program involves tracking the IIK–IY Team’s coaching visits with Parent Program facilitators. Two questions were added to the 2020–2021 visit tracking forms to capture information about the virtual coaching and program delivery format at each coaching visit. The first question was about the specific virtual coaching modality for the coaching visit  and the second question was about the program delivery format . The goal of adding these questions was to monitor Parent Program coaching visits by program delivery format.In addition, Parent Program facilitators administer weekly surveys to participants in their group. These brief surveys give participants an opportunity to provide feedback about the extent to which program activities and content are helpful. For the 2020–2021 program year, the IIK–IY Team added a question about participants’ satisfaction with the virtual format. The purpose of adding this question was so that facilitators could use this feedback to improve the weekly program sessions and specifically identify whether there were any barriers to participating in the program virtually.There were also a few adaptations to the IIK–IY Parent Program outcomes evaluation. Before the start of the 2020–2021 program year, the IIK–IY Data and Evaluation Manager determined that participants from both in-person and virtual groups would still be able to complete the same pre–post-outcome measures at the beginning and end of the 14-week parent group, and the program format would be tracked by group. The IIK–IY Parent Program pre–post-outcomes measures include 12 items that make up the Social Competence Scale-Parent (SCS-P) Report  as being particularly helpful. They also appreciated the discussions and hearing the perspectives of other parents that were experiencing similar challenges, and they appreciated the insights and guidance of the facilitators in a nonjudgmental, compassionate environment. Participants also indicated that the topics covered, and related program resources and materials were relevant and useful. In terms of how the program could be changed to better help participants, some suggested that it would have been helpful to spend additional time discussing certain topics. Many virtual group participants also mentioned that while they enjoyed the program, it would have been better to meet with the group in person and benefit from face-to-face interactions and discussions. The overwhelming majority reported that they would participate in virtual sessions again, although several people noted that they would prefer in-person sessions.The documentation of Parent Program facilitators’ successes and challenges delivering the program during the 2020–2021 program year offered another perspective on the program adaptations. A content analysis of facilitators’ open-ended responses revealed that they perceived a high level of participant engagement in the virtual sessions, as well as high rates of attendance and retention. They also noted that participants were able to create supportive communities within groups, even virtually. Some facilitators emphasized participants’ growth in their use of the different parenting practices and noted that participants provided positive feedback about the group. A few mentioned that the virtual format increased the accessibility of the program for families who would have otherwise been unable to participate in person.Though several facilitators considered participant engagement to be high, it was also identified as a challenge for virtual groups. Facilitators from a few groups noted that getting each participant to leave their camera on was difficult, and participants tended to get distracted during the sessions by what was going on at home. Relatedly, another barrier to supporting the program virtually was the lack of childcare for facilitators and participants. Several facilitators referenced the challenge of setting up role plays, a fidelity element, in virtual settings. Feedback from facilitators who supported participants from rural Colorado counties highlighted the urban versus rural digital divide, as these virtual groups struggled with unreliable internet connections. A few facilitators also commented on variability in participants’ comfort levels with technology and in using online meeting platforms. Lastly, some explained that it felt harder to build relationships with participants in virtual groups, compared to in-person settings. The Parent Program facilitator feedback suggests that overall, the key challenge was to deliver the virtual program with fidelity; however, facilitators also seemed to see a lot of success in terms of participants’ growth in their parenting skills. Participants’ feedback indicated that they enjoyed and greatly benefitted from the groups, regardless of program delivery format. This is largely consistent with the quantitative participant pre–post-outcomes.“yes” for that fidelity item) by program delivery format. It is interesting that the mean number of fidelity items that the IIK–IY Team observed was lower for facilitators of in-person groups compared to virtual groups. This pattern generally aligns with the Parent Program outcomes; some of the outcomes for in-person participants were slightly smaller than the outcomes for virtual program participants.The adaptations to the 2020–2021 Parent Program fidelity checklist allowed the IIK–IY Team to virtually assess Parent Program facilitators’ fidelity up to two times during the 14-week group for virtual and in-person program delivery. Overall, there were far fewer facilitators that were observed doing in-person Parent Program during the 2020–2021 program year compared to the number of facilitators that were observed delivering the program virtually. On average, the IIK–IY Team observed 79% of fidelity items being delivered by facilitators of in-person groups at checklist one and 86% of fidelity items at checklist two. On average, the IIK–IY Team observed 86% of fidelity items being delivered by facilitators of virtual groups at checklist one, and 88% of fidelity items, on average, at checklist two. See Table During the 2020–2021 program year, the IIK–IY Team partnered with Colorado communities in 20 counties to support the delivery of Dinosaur School and Parent Program for 4027 students, 315 teachers, 67 Parent Program Facilitators, and 375 Parent Program participants. The IIK–IY Team successfully adapted their implementation supports, coaching teachers and Parent Program facilitators virtually, and supported many teachers and facilitators with virtual program delivery so that children and families in Colorado could continue to benefit from IY programming. Prior to the onset of the pandemic, the IIK–IY Team had not considered supporting teachers or Parent Program facilitators with virtual program delivery, as both programs were developed for in-person delivery. The shift to a virtual world necessitated a contextual change to Dinosaur School and Parent Program delivery, and a change to the mode of the IIK–IY Team’s training and coaching. Without the use of the FRAME tool and the contextual adaptations made to the IY programs and IIK’s implementation supports, it would have been much harder, if not impossible, for IIK’s partner agencies to effectively deliver Dinosaur School and Parent Program throughout the pandemic. The IIK–IY Team’s adaptations helped to ensure that the IY programs continued to be usable innovations  made it harder to gauge the extent to which each item was delivered during the fidelity observation. However, IIK–IY coaching and fidelity observations continued to be informative for the IIK–IY Team in identifying what supports were needed and defining expectations about reasonable next steps to support teachers’ and Parent Program facilitators’ fidelity in the midst of a pandemic. Despite less clarity and precision about the degree to which fidelity was adhered to throughout the program year, IIK–IY’s decision support data system and other Competency and Organizational Implementation Drivers helped to compensate.A limitation of this case study is that the IIK–IY Dinosaur School and Parent Program fidelity and outcomes information presented does not assess the extent to which the various adaptations individually or collectively affected program fidelity and outcomes, and thus any inferences about the links between the program adaptations and the data points presented here are extremely limited. However, this case study contributes important information to the implementation science and practice literature about what large-scale, statewide adaptations to an EBP look like in community-based settings and describes the proximal program outcomes and fidelity in the context of such adaptations. Additionally, this case study provides an in-depth look at the application of an implementation science tool Supplementary file2 (PDF 14 kb)Supplementary file3 (PDF 21 kb)Supplementary file4 (PDF 15 kb)Supplementary file5 (PDF 20 kb)Below is the link to the electronic supplementary material.


BackgroundThe 2022-2023 residency match cycle will be the first cycle that program directors will have to consider some applicants with a numerical United States Medical Licensing Examination (USMLE) Step 1 score while other applicants will only report pass/fail for USMLE Step 1. Previous studies have explored how USMLE Step 1 becoming pass/fail will alter the residency selection process, but it is not yet known when program directors from each specialty expect those changes to be implemented.MethodsResidency program director’s contact information was extracted from the American Medical Association (AMA) residency program site, Fellowship and Residency Electronic Interactive Database (FREIDA). Of the 5190 programs, 4877 were determined eligible for this study of which 1274 (26.8%) responded.ResultsOf the 1274 US residency program directors included in this survey, 77.0% do not intend to adjust their usage of USMLE Step 1 as a metric in candidate evaluation until the score is no longer reported.ConclusionResidency candidates applying during the upcoming cycle can expect the majority of residency programs will not significantly alter their previous utilization of an applicant’s USMLE Step 1 score during the current 2022-2023 residency match cycle. For years, the United States Medical Licensing Examination (USMLE) Step 1 numerical score has been a tool utilized by residency program directors to assess applicants for interview and selection into their programs . As of JHistorically, metrics such as USMLE Step 1 scores, Step 2 clinical knowledge (CK) scores, Medical Student Performance Evaluation (MSPE), and grades in clinical clerkships consistently rank high among most specialties . In repoThe 2022-2023 residency match represents a unique challenge as the first application cycle that program directors will need to consider applicants with a numerical USMLE Step 1 score alongside a portion of applicants that received only a pass/fail result from their USMLE Step 1. While surveys have been conducted to assess the opinions of residency program directors about the changes to Step 1 scoring and which metrics they expect to emphasize, no studies have been conducted to assess when program directors intended to transition to metrics that do not include an applicant’s USMLE Step 1 score.This study presented our findings regarding when residency program directors expect they will begin to disregard the usage of the numerical USMLE Step 1 score in their evaluation process. It seeks to provide some clarity for residency applicants as they consider how best to optimize their transition from undergraduate to graduate medical education.A total of 4877 program directors’ email addresses were obtained from the American Medical Association residency program site, Fellowship and Residency Electronic Interactive Database (FREIDA). Of the eligible programs, 118 email addresses were returned as undeliverable and additional five programs requested not to be included in the study. Three subsequent survey invitations were emailed to residency program directors (RPDs) between March 3 and March 9, 2020. Program directors who elected to participate utilized a digital link to the Research Electronic Data Capture (REDcap) survey. After completion of the survey, program directors were temporarily able to view graphs summarizing the survey data. The survey was closed on March 14, 2020, at 12 pm CST.This study presents RPDs’ responses to the question - when is your program likely to begin disregarding three-digit USMLE Step 1 scores? - that was asked as a component of a broader 14-question online survey of RPDs (figures in Appendices). Survey responses were reviewed and analyzed in SPSS version 27 . Frequency distribution and descriptive statistics were considered to summarize participant responses as relevant to the objective and further elucidate the characteristics of the programs that participated in the study. The study was approved using exempt procedures by the Institutional Review Board at the University of Arkansas for Medical Sciences prior to dissemination of the survey .DemographicsOf the 1274 program directors that responded, 1164 selected their specialty. The specialties with the highest response rate relative to their specialty were - physical medicine and rehabilitation (46.1%), emergency medicine (40.3%), and ENT (33.1%)  of program directors across all specialties responded that they do not expect to discontinue using applicants USMLE Step 1 numerical score until it is no longer reported , concurrent programs such as MD/Ph.D., and students repeating clinical years). This study suggests that residency program directors will most likely continue to use numerical USMLE Step 1 scores for all applicants that have received one.Some limitations of this study are the inherent challenge of survey studies in that some program directors may be more or less inclined to respond, resulting in the potential for nonresponse bias. Additionally, this survey was conducted shortly after the USMLE announced the change in scoring system, which has provided the opportunity for program directors to develop new systems to adjust to the challenges of evaluating applicants that do not all have a numerical USMLE Step 1 score.Our findings suggest that residency applicants with a numerical USMLE Step 1 score, independent of specialty, can expect that their score will likely be used during their 2023 residency selection process. Those residency applicants that did not receive a numerical USMLE Step 1 score present an active challenge for program directors to objectively compare the two cohorts.


Sensory processing, along with the integration of external inputs into stable representations of the environment, is integral to social cognitive functioning; challenges in these processes have been reported in Autism Spectrum Disorder (ASD) since the earliest descriptions of autism. Recently, neuroplasticity-based targeted cognitive training (TCT) has shown promise as an approach to improve functional impairments in clinical patients. However, few computerized and adaptive brain-based programs have been trialed in ASD. For individuals with sensory processing sensitivities (SPS), the inclusion of some auditory components in TCT protocols may be aversive. Thus, with the goal of developing a web-based, remotely accessible intervention that incorporates SPS concerns in the auditory domain, we assessed auditory SPS in autistic adolescents and young adults (N = 25) who started a novel, computerized auditory-based TCT program designed to improve working memory and information processing speed and accuracy. We found within-subject gains across the training program and between pre/post-intervention assessments. We also identified auditory, clinical, and cognitive characteristics that are associated with TCT outcomes and program engagement. These initial findings may be used to inform therapeutic decisions about which individuals would more likely engage in and benefit from an auditory-based, computerized TCT program. Clinical reports of sensory sensitivities in Autism Spectrum Disorder (ASD) often refer to autistic persons’ heightened awareness of minute details that are not perceived or noticed by most. Patients may share, for example, that they can distinguish between roads while riding in a vehicle just by listening to fluctuations in the sound of their tires on different surfaces, are irritated by one person’s gum-chewing in a house rife with ambient noises, or that they dislike fluorescent bulbs because they can see the lights flashing on/off above the critical flicker frequency of 50 Hz. ASD is a complex neurodevelopmental disorder (NDD) characterized by persistent deficits in communication and interaction and restrictive, repetitive patterns in behavior, interests, and activities . AlthougSensory processing is crucial for forming reliable environmental precepts upon which other cognitive and adaptive abilities depend. Perturbations in this process may originate from ASD-related differences in low-level processing in sensory-dedicated regions in the brain ,10 and cIn recent years, the prevalence of ASD has reached an estimated 1-in-44 in the United States , and theTo date, a number of studies have examined the efficacy and utility of computer-based TCT programs in clinical and developmental samples ,38,39. AThus, the goals of our study were as follows: (1) to assess autistic response to a remotely-delivered auditory-based TCT program with consideration for feasibility and efficacy; (2) to evaluate the effects of auditory SPS on intervention engagement and compliance; and (3) to identify participant characteristics at baseline that associate with training outcomes. Our long-term objective is to inform and better tailor interventions for autistic individuals.We recruited individuals with a prior ASD diagnosis  from local clinics and the community to participate in a customized, adaptive, auditory-based TCT program.Diagnosis was confirmed by our clinical team with autism expertise through obtained medical history and comprehensive review of developmental and evaluation records. Participants with co-morbid diagnoses of ASD-related neurodevelopmental disorders  were included see ; these cPediatric Quality of Life Inventory (PedsQL) [Social Responsiveness Scale, 2nd Edition (SRS-2) [Repetitive Behavior Scale—Revised (RBS-R) [Child and Adolescent Symptom Inventory (CASI-4R) [At the pre-intervention assessment session, we collected parent-report and/or self-report measures , including demographic, family medical history, the (PedsQL) , the Soc (SRS-2) , the Rep (RBS-R) , Child aCASI-4R) . To asseCASI-4R)  and the CASI-4R)  with resCASI-4R) . The FW The neuroplasticity-based auditory-based TCT program comprised multi-level, computerized exercises designed to enhance the speed and accuracy of auditory information processing while engaging neuromodulatory systems involved in attention and reward . Particihttps://www.brainhq.com/why-brainhq/about-the-brainhq-exercises/, accessed on 12 December 2022):(a)Auditory Sweeps  target auditory processing speed (units = msec). Two successive frequency-modulated tone sweeps were presented, and participants indicated whether the frequency increased or decreased within each tone;(b)Sound Discrimination (“Fine Tuning”) targets auditory perception and processing speed (units = msec). Participants indicated which one of two confusable syllables was presented;(c)Syllable Ordering (“Syllable Stacks”) targets auditory memory (units = # items). Participants reported the order of presented syllables in a serial memory span task;(d)Auditory Spatial Match (“Memory Grid”) targets auditory memory (units = # items). Participants matched identical cards representing syllables.For the present investigation, we requested a customized research training module from Posit Science that only included the following auditory-based “games”  to more complex levels . All performance metrics were recorded throughout the training course. To improve compliance and retention, research staff scheduled weekly video/phone chat check-ins with each individual participant.The overall effects of completing the intervention were examined by conducting repeated-measures analysis of covariance (ANCOVA) tests  on pre-/post-intervention clinical and cognitive measures for participants, covarying for age and sex, who completed all levels of the auditory-based TCT program.avg.orig) = score reached the first time a participant encountered each exercise; (2) number of blocks/levels (num) = direct measure of exposure to a training exercise; (3) peak performance (avg.best) = best score reached in a training exercise at any point throughout the intervention; (4) weighted peak performance (wavg.best) = weighted average of peak performance that takes into account the number of blocks completed by each user for that specific exercise; (5) delta (avg.delta) = difference between subject-specific best and baseline performance within that block, averaged across blocks; (6) weighted delta (wavg.delta) = avg.delta divided by the standard deviation of baseline performance for that block across all study participants. The weighted peak performance metric allowed us to consider improvement in speed and accuracy for participants at varying levels of the training curriculum. In addition to serving as a measure of exposure to the TCT program, the number of blocks completed was also considered an indirect measure of participant engagement .TCT performance was evaluated according to metrics derived by Posit Science Corporation : (1) basTwo-step cluster (TSC) analyses were applied to identify subgroups of intervention exposure and engagement; the total number of levels completed was input as a continuous variable. TSC analysis employed a log-likelihood distance measure, Akaike’s information criterion (AIC) clustering criterion, and a maximum of 15 clusters. Silhouette coefficients of cohesion and separation, along with membership variables of each cluster solution, were used to assess whether outcome measurements served as accurate indicators of identified subgroups which we designated high-, mid-, and low-engagement groups. ANCOVAs were then used to compare performance metrics by cluster groups.Post-hoc, exploratory analyses using partial correlations controlling for age and sex (one-tailed significance) were conducted to identify baseline factors that are associated with level of engagement and intervention performance metrics; we posit that these factors may be used to inform therapeutic decisions about which individuals will be more likely to utilize and then benefit from this auditory-based TCT program.Weighted average change metrics were calculated for all participants (N = 25) who started the auditory-based TCT program regardless of whether they completed all levels of training; mean performance change (wavg.delta) on all four exercises reflected improvements from baseline: Sound Discrimination = 2.248, SD = 0.795; Syllable Ordering = 0.812, SD = 0.373; Auditory Sweeps = 0.265, SD = 0.265; Auditory Spatial Match = 0.803, SD = 0.624 A. In sump = 0.017 η2 = 0.489; BAC App Letter Fluency Total: F  = 5.716, p = 0.041; η2 = 0.388; BAC App Combined Fluency Total: F  = 11.044, p = 0.009; η2 = 0.551; CASI-4R ADHD-Hyperactivity/Impulsivity Subscale: F  = 6.249, p = 0.034; η2 = 0.410; and the CASI-4R ADHD-Combined Subscale: F  = 5.413, p = 0.045; η2 = 0.376. We found trend-level improvements on the BAC App Symbol Coding Subscale: F  = 4.801, p = 0.056, η2 = 0.348, and the MEFS App Total Score : F  = 4.354, p = 0.067, η2 = 0.356. Changes in the following pre-post-intervention measures did not reach significance: CASI-4R Autism Subscale: F  = 0.851, p = 0.380, η2 = 0.086; SRS-2 Total: F  = 0.159, p = 0.701, η2 = 0.019; RBS-R Total: F  = 1.561, p = 0.258, η2 = 0.206; PedsQL-Self Report: F  = 2.883, p = 0.124, η2 = 0.243.Due to COVID-19 restrictions, we were unable to collect the entire battery of post-intervention measures for some completers. For the subset of participants who provided pre- and post-intervention data, we found significant within-subject improvements in clinical and cognitive measures : BAC AppTSC analysis of the total number of levels completed (num) for all participants yielded a three-cluster solution with a robust silhouette measure of cohesion and separation coefficient (0.9644); we designated these engagement subgroups as High (Mean Levels: 640), Mid (Mean Levels: 322), and Low (Mean Levels: 42.8). Mean weighted average performance change for each TCT exercise for each TSC-Based Engagement Sub-Group suggests that non-completers (Low-/Mid-Engagers) demonstrate improvement on tasks in a pattern similar to completers (High-Engagers) B. Howevep = 0.009, η2 = 0.528); RBS-R Total  = 4.773, p = 0.009, η2 = 0.527); CASI-4R ADHD Combined  = 3.269, p = 0.037; η2 = 0.435, (pairwise comparisons n.s.)); CASI-4R Autism  = 3.004, p = 0.048, η2 = 0.414). Pairwise comparisons with Bonferroni corrections revealed a significant mean difference between High- and Mid-Engagers on the SRS-2 Total Score (p = 0.050) and RBS-R Total Score (p = 0.033) and between Mid- and Low-Engagers (p = 0.003) on the SRS-2 Total Score. A trend-level difference between High- and Low-Engagers was also found (p = 0.077) for the SRS-2 Total Score. High- and Mid-Engagers (p = 0.011) and Mid- and Low-Engagers (p = 0.035) differed significantly on the CASI-4R Autism Subscale; pairwise comparisons were not significant for between engagement groups on the CASI-4R ADHD Combined Subscale  did not differ significantly across the three subgroups. However, scores on clinical measures collected at baseline showed differentiation by TSC solution: SRS-2 Total  = 4.755, Subscale .p = 0.017, η2 = 0.491) and Sound Discrimination  = 3.528, p = 0.029, η2 = 0.454), as well as a trend level difference for Auditory Spatial Match  = 2.574, p = 0.075, η2 = 0.377). Pairwise comparisons with Bonferroni corrections revealed a significant mean difference between High- and Low-Engagers on Auditory Sweeps (p = 0.024), such that High-Engagers performed better at baseline. Trend level differences were also found between High- and Mid-Engagers on the Auditory Spatial Match (p = 0.052), Syllable Ordering (p = 0.068), and Sound Discrimination (p = 0.078) exercises indicating that High-Engagers performed more accurately than Mid-Engagers  was found between engagement groups for Auditory Sweeps  = 4.1094, Engagers .p = 0.010 η2 = 0.527). Pairwise comparisons with Bonferroni adjustments revealed a significant difference between High- and Low-Engagers (p = 0.002) and a trend level difference (p = 0.078) between Mid- and Low-Engagers (A significant main effect for the engagement group on mean improvement in performance (wavg.delta) was found for the Syllable Ordering exercise  = 4.728, Engagers .r = 0.458, p = 0.021), Auditory Sweeps , and Sound Discrimination . Significant partial correlations were also found for TCT levels completed and mean improvement on Syllable Ordering  and Auditory Sweeps ; a trend level correlation was found for TCT levels completed and mean improvement on the Auditory Spatial Match exercise . Partial correlations for TCT performance metrics are reported in Partial correlation analysis revealed significant relationships between the number of TCT levels completed and baseline performance on Auditory Spatial Match  and Syllable Order exercises . A significant partial correlation was also found for the relationship between baseline SCAN CW raw scores and mean improvement in performance on Auditory Sweeps   and the RBS-R Total Score   A. Trend = 0.068) B.The present study is among the first to examine neuroplasticity-based TCT in ASD; our initial findings support the use of web-based, adaptive, auditory-based TCT to improve outcomes for autistic individuals. Data indicate significant within-subject gains in performance on all four exercises in the adaptive training program. Moreover, participants who completed the program showed improvements in pre- and post-intervention measures of clinical and cognitive function. These data also demonstrate the feasibility of implementing an intense 12–16 week long, remotely-delivered TCT program in autistic adolescents and young adults.Given that sufficient exposure to the training exercises is necessary to effect meaningful change in target skill domains, a key study design concern is participant engagement. While we could not control the myriad extrinsic circumstances  that may affect whether or not individuals complete all study procedures, we aimed to reduce intrinsic barriers by identifying measurable endophenotypic factors associated with program compliance and outcomes. These features may be useful to consider before implementing a time-consuming intervention schedule; screening at baseline may be used to predict therapeutic goodness of fit for patients, reducing ‘drop-out’ by identifying and enrolling those who would more likely engage in and gain from a long-term, self-motivated TCT program. Accordingly, individuals who are less likely to benefit from this form of intervention may be referred to treatments that are more compatible. Interestingly, the mid-engagement group evidenced significantly fewer symptoms  than low- and high-engagers on the RBS-R, SRS-2, and CASI-4R measures at baseline. While further study is required, we conjecture that attrition from higher-functioning individuals who enrolled in the study may be related to loss of interest in the protracted, repetitive program despite the adaptive nature of the exercises.Our post-hoc analyses evaluating pre-intervention measures of auditory sensitivity and processing show promise as predictors of therapeutic efficacy. Raw scores on the SCAN FW subtest were significantly correlated with engagement and performance change metrics such that participants with more auditory processing difficulties (lower SCAN FW scores) showed greater mean improvement on the Auditory Spatial Match and Syllable Ordering exercise. Similarly, raw scores on the SCAN CW subtest were significantly correlated with weighted average change on the Auditory Sweeps exercise, indicating that lower SCAN CW scores were associated with greater improvement in performance. These pre-intervention SCAN findings align with baseline performance data on Auditory Sweeps, a training task designed to target “bottom-up” auditory processes , such thThe presented findings data underscore the need to shift towards precision treatments that take individual endophenotypes into consideration when developing a therapeutic regimen. Heterogeneity within the ASD diagnosis precludes the efficacy of generalized approaches, yet, the burden of extensive, personalized characterization and treatment can be prohibitive. The present study provides a promising approach to streamlining screening for auditory SPS with objective task-based measures for auditory SPS. While we administered the SCAN FW and CW subscales during in-person assessments, remote administration is also feasible with video-chat technology and headphones. Similarly, the web-based auditory-based TCT program allows for fast electronic data upload and scoring of baseline performance. These brief, remotely-delivered measures allow for expedient decisions about whether an individual is more or less likely to benefit from the intervention.post-hoc analyses of the relationships between factors are correlational and should not be over-interpreted without further investigation. (5) Finally, given the remote-delivery design of the study, we did not include direct observations or assessments of adolescent peer interactions. Nevertheless, we do posit that the demonstrated “near-transfer” of cognitive gains on auditory perception, memory, and speed of processing may lead to “far-transfer” in other domains. For example, we hypothesize that faster processing of auditory information may be beneficial for individuals who have difficulty distinguishing temporal cues in conversation. Future study designs will need to include longer-term follow-up of participants as well as direct measures of potential “far-transferred” skills in practice.Our study was limited in a number of ways: (1) the time commitment required to complete the TCT program was lengthy for this population (12–16 weeks); consequently, adolescents with academic, extra-curricular, or away-camp activities were less likely to participate in a study requiring sustained engagement. Moreover, our data collection was hindered by mandatory constraints on active research during the COVID-19 pandemic; hence, our sample size was modest. Fortunately, most of our TCT program could be completed online, and our researchers were able to maintain weekly check-ins via phone/video calls in order to provide support as participants progressed in the program. (2) Future studies will need to include autistic participants asked to play non-adaptive games  as well as non-autistic participants as comparisons to complete the TCT program . In this initial TCT study, since our objectives were to better understand individual differences across the spectrum of autism so that we might inform future interventions, we chose to focus on within-subject effects. (3) Our study sample includes a high (5:1) male:female ratio; while this imbalance is roughly comparable to the broader ASD population, the lack of female participants precludes generalization to autistic females. (4) While we posited the predictive potential of pre-intervention measures in determining the suitability of auditory-based TCT, our One of the primary objectives of this investigation was to ascertain which baseline measures associate with intervention outcomes in an effort to better predict which individuals might (A) comply with the demands of a program that requires a significant time commitment and willingness to engage and (B) benefit from the program if they are exposed to enough of the intervention. Given that all participants demonstrated some gains in the TCT skills, we suggest that compliance and engagement may be the limiting factor in intervention success. Participants who demonstrated more auditory processing deficits during the pre-intervention assessment  showed greater average improvement over time, and these gains were associated with the number of levels completed. However, poorer baseline TCT program performance also correlated positively with the number of levels completed. These data suggest that those who might benefit more from greater exposure to the TCT program also had the most trouble staying engaged. As such, future studies will need to evaluate whether individuals with low baseline performance scores can gain more from the TCT program if additional support is provided to increase engagement. Notably, we found that our high engagement group  showed symptom levels between mid- and low-engagers on the RBS-R, SRS-2, and CASI-4R measures at baseline. These findings show that individuals with “mid” level symptoms may be a good fit for TCT as they are more likely to complete the program. Overall, these findings suggest that first encounters with the TCT module provide valuable insight into the potential utility of the auditory intervention for each individual.Neuroplasticity-based, computerized TCT has emerged as a promising approach to improve outcomes in multiple clinical and cognitive domains. The appeal of evidence-based TCT programs is their broad accessibility (remote delivery), cost-effectiveness, and potential for individual tailoring of treatment targets. Program use and outcomes can be monitored remotely, which allows clinicians and researchers to confirm engagement, provide assistance, and adjust training as needed. Our findings are among the first to provide support for the efficacy and feasibility of long-term, auditory-based TCT for autistic adolescents and young adults. We were also able to identify subsets of participants with similar patterns of symptoms and traits who engaged with and responded to the intervention program; further delineation of these clusters may be used to triage individuals into more appropriate therapeutic plans and inform the development of precision medicine approaches.


ARID1A-mutated ovarian clear cell carcinoma (OCCC) is a relatively indolent and chemoresistant ovarian cancer histotype. In OCCC-derived cells ARID1A simultaneously drives GLS1 expression and metabolism reprograming. In ARID1A-mutated OCCC-derived mouse models, loss of ARID1A corresponds to GLS1 upregulation and increases sensitivity to GLS1 inhibition. Thus, targeting of GLS1 with CB839 has been suggested as a targeted approach for patients with OCCC with tumors harboring ARID1A mutations. Here, we investigated whether GLS1 is differentially expressed between patients with OCCC whose tumors are ARID1A positive and patients whose tumors are ARID1A negative. In clinical specimens of OCCC, we found that GLS1 overexpression was not correlated with ARID1A loss. In addition, GLS1 overexpression was associated with better clinical outcomes. Our findings have implications for human trials using experimental therapeutics targeting GLS1.Targeting glutamine metabolism has emerged as a novel therapeutic strategy for several human cancers, including ovarian cancer. The primary target of this approach is the kidney isoform of glutaminase, glutaminase 1 (GLS1), a key enzyme in glutamine metabolism that is overexpressed in several human cancers. A first-in-class inhibitor of GLS1, called CB839 (Telaglenastat), has been investigated in several clinical trials, with promising results. The first clinical trial of CB839 in platinum-resistant patients with ovarian cancer is forthcoming. GLS1 differential expression in patients with OCCC with or without ARID1A mutations is significant because a clinical trial with a GLS1 inhibitor is forthcoming. Tumors without ARID1A have low levels of GLS1 and GLS1 expression is associated to better outcome. Thus, blockade of GLS1 could be counterproductive for patients with OCCC. ARID1A, which is found in approximately 50% of patients  is an indolent form of ovarian cancer associated with a poor prognosis. The principal reason for such a dismal prognosis is resistance to standard-of-care chemotherapy used in general for ovarian carcinomas, including taxane- and platinum-based agents . OCCC ispatients . ARID1A patients . Patientutations . Thus, ein vitro and in a preclinical model of ARID1A-mutated OCCC  and anti-GLS1 .Archival tissues were used with approval from the Institutional Review Board (IRB) of the University of Minnesota (STUDY00006529). Demographic information and patient characteristics are reported in ARID1A mutation screening and as a method that has been shown to reliably predict ARID1A mutations with 100% sensitivity and specificity. GLS1 immunoreactivity was scored using an H-score: staining intensities  were multiplied by their respective percentages of cells stained . Brightfield images were acquired with a Zeiss Axio Scan.Z1 system (Zeiss) at 40× magnification.Representative cores from the formalin fixed paraffin embedded (FFPE) blocks of archival tissues were selected and arranged in (two) tissue microarrays (TMA) containing a total of sixty clinical specimens. Fifty-four specimens had enough quality tissue for both ARID1A and GLS1 staining. Thus, the correlation between GLS1 and ARID1A expression was calculated based on 54 (37 + 17) of these specimens. Clinical information was available for 55 ARID1A-stained specimens  and 54 GSomatic DNA was extracted from FFPE tissue samples. Next-generation sequencing (NGS) libraries were prepared with 50–250 ng of purified DNA following the Illumina NGS library preparation procedure and enriched by PredicineCARE NGS panel (Predicine) using a hybrid capture method and deeply sequenced by Illumina pair-end sequencing. Raw sequencing data (BCL files) were fed through DeepSea, a Predicine proprietary NGS analysis pipeline. Paired-end reads were first merged as single-strand fragments, then the consensus bam file was built by merging fragments with the same start and end mapping locations. Raw variants were called from mismatches with the reference genome in the consensus bam file and were then filtered on the basis of variant background , repeat regions, and other quality metrics. Variants with mutation allele frequency (MAF) ≥ 5% and hotspot variants with MAF down to 2% were reported.Total cellular protein (10–30 μg) from each sample was separated by SDS-PAGE, transferred to polyvinylidene fluoride (PVDF) membranes and subjected to Western blot analysis using the specified antibodies, as previously shown . Amido bt tests for continuous measures and Fisher exact tests for categorical measures. In this analysis ARID1A was treated as a binary variable using the previously mentioned cut-off of 5.5, while GLS1 categorical groups were created based on low (<100), moderate  and high (≥200) expression levels. Survival curves were generated using the Kaplan–Meier method. Univariate Cox regression models were used to determine the statistical significance of ARID1A and GLS1 on progression-free survival (PFS), cancer-specific survival (CSS), and overall survival (OS). After identifying possible confounders, models were also generated to adjust for factors such as stage and endometriosis  and raw data were not available at the time of publication due to Predicine's legal terms and conditions. Only the mutation calling data was provided by Predicine. All available data will be shared upon reasonable request from the corresponding author.ARID1A mutations are found in 50 and 30% of patients, respectively , CSS (P = 0.003), and OS (P = 0.004). Importantly, the correlation between ARID1A loss and worse outcome remained significant even after adjusting for stage and endometriosis in multivariate analysis . Classification of early stages (I+II) versus late stage (III–IV) is common practice both clinically and in retrospective and prospective studies involving patients with ovarian cancer because these two groups typically have very different outcomes. In fact, consistent with other studies , with early stage ARID1A-negative patients having a prognosis similar to that of III/IV ARID1A-positive patients . Taken together, these results demonstrate that our cohort can reliably be used to investigate the correlation between ARID1A status and GLS1.We then sought to determine the correlation between ARID1A expression and clinical outcome. As shown in  studies  an even P = 0.001). Evaluation of ARID1A expression via IHC has been shown to reliably predict ARID1A mutations  as compared with patients who did not carry ARID1A mutations. The fact that in the OCCC cohort for which we had performed next-generation exosome sequencing, patients whose tumors carried ARID1A mutations had a dramatically lower ARID1A expression levels (H-score) as compared to patients who did not carry ARID1A mutations further speaks for the reliability of IHC staining to determine ARID1A mutational status.Recent reports have shown that GLS1 is overexpressed in both OCCC and high grade-serous carcinoma (HGSC) of the ovaries, compared with cells derived from normal surface epithelium of the ovaries and fallopian tubes . Furtherutations . To expeutations , patientP = 0.0259). This finding is novel and unanticipated. We then compared GLS1 expression levels between ARID1A-negative OCCC and normal surface epithelia from ovaries and fallopian tubes. We found that GLS1 expression in normal epithelia is similar to expression levels detected in ARID1A-negative OCCC (P = 0.828). Moreover, GLS1 levels were not correlated with other clinicopathologic parameters, suggesting that the ARID1A status was the only determinant of GLS1 levels in our cohort  tumors have higher levels of GLS1 as compared with ARID1A-mutated tumors  for which we had paired frozen specimens available . As showP = 0.013; CSS, P = 0.014; OS, P = 0.021). The correlation of GLS1 overexpression with better clinical outcome was novel and unanticipated. The absence of correlation with the clinicopathological characteristics was consistent with previous findings in other cancers , and GLS1 only showed a trend toward improved OS , but significance was lost in the ARID1A- subgroup , probably due to the very small size of the moderate and high expression groups .Next, because we found that loss of ARID1A immunoreactivity acted as a negative prognostic factor in our set of patients, and that this correlated with low levels of GLS1, we included the ARID1A status in multivariate analysis to exclude the hypothesis that it may have acted as a confounder. Both the factors tended to remain significant, but ARID1A did not reach significance for PFS  in the necessarily different microenvironment between cancer cells growing in cultures, cancer cells growing in a mouse and cancer cell growing in situ in patients’ tumors, and b) in the difference in proliferation rate of cancer cells in vitro, in a mouse model and in a patients’ tumors. This is consistent with recent work showing that in vitro glutamine metabolism is highly dependent on experimental conditions  is a key enzyme in glutamine metabolism that has been shown to be overexpressed in cancer cell lines and tumor models and to be required for the survival of senescent cells . Furthernditions . FurtherImportantly, our findings have implications for human trials using experimental therapeutics targeting GLS1. The ability to add ARID1A as a correlative predictive biomarker using IHC and/or next-generation sequencing techniques should be strongly considered for current and forthcoming trials to provide prospective validation of our findings. More importantly, to best stratify patients and limit use of targeted drugs that are not likely to be effective in patients and which may cause more harm than benefit, our findings should be examined carefully before launching iterations of this therapeutic strategy.Noteworthy, while in this study we have focused on ARID1A as it represents the most frequently mutated component of the SWI/SNF complex in OCCC , other cc) IHC to determine ARID1A levels; in the era of genomic profiling, IHC has been proven to be highly reliable to distinguish between ARID1A WT and ARID1A-mutated tumors  the retrospective approach; as always, prospective evaluation with appropriate pre-powered statistical design is indicated and necessary to confirm these findings, (ii) sample size; we had available total of sixty clinical specimens in our study. This is an appropriate number considering that ovarian cancer is a rare disease and that OCCC represent 5%–10% of all ovarian carcinomas. However, and as always, a bigger cohort would allow for validation and refining of our findings, d tumors . Furtherted OCCC . Finallyted OCCC  this maySupplementary Figures S1, Tables S1-S2Figure S1, Table S1, Table S2Click here for additional data file.


Outdoor falls present a significant challenge to the well-being of community dwelling older adults. There are a number of existing evidence-based programs that address fall risk, including multifactorial and exercise-based programs. However, despite the difference in risk factors for indoor and outdoor falls, no existing program specifically targets outdoor falls. To fill this gap, the Stroll Safe program was developed and refined based on a prior feasibility study. The 7-week group-based manualized program is focused on promoting safe behavioral strategies to reduce the risk of outdoor falls. In addition to presentations and community mobility coaching by the group leader, an occupational therapist, the program includes group discussion and problem solving, capitalizing on the life experiences of participants. Action planning facilitates implementation of the strategies discussed. Given that the neighborhood environment impacts both risk and fear of falling, a walkability audit (i.e. the SWAN) focused on environmental hazards is included and is used to promote awareness of hazards and as a tool for self-advocacy. Data science and machine learning facilitate the creation of resources for route planning. Vision Zero resources help to identify hotspots for pedestrian injuries, and a map of shadow is used to create a user friendly map of potential hotspots for black ice. In this symposium, we will present 1) Findings from the efficacy trial for Stroll Safe, and 2) Describe data science research that can be used to inform outdoor falls prevention programs.


F. tularensis. We determined F. tularensis-specific IgM and IgG antibody titers by an LPS-based ELISA assay on five occasions one to twelve months after onset of ulceroglandular tularemia in 19 individuals. Peak IgM antibody titers were observed at the one-month time point and peak IgG antibody titers at the two-month time point. Both IgG and IgM antibody levels declined linearly thereafter with rather similar kinetics. Compared to the average one-month antibody titers, average IgG titers were not significantly lower before the 12-month time point and IgM titers before the 4-month time point. All, but one average titer, were significantly increased compared to the cut-off of the assay. Average IgG and IgM titers were significantly lower for the group = 69 years old compared to the group < 69 years. Collectively, the data demonstrate a persistence of F. tularensis-specific IgM and IgG antibody titers for at least 12 months after ulceroglandular tularemia. Thus, low, but significantly elevated F. tularensis-specific antibody titers are of limited diagnostic value since they are not indicative of ongoing tularemia.Serological analysis is the predominant method used to diagnose tularemia, a zoonotic disease caused by the highly virulent bacterium  Francisella tularensis is the etiological agent of the zoonotic disease tularemia and it is a facultative intracellular, highly infectious, Gram-negative bacterium (acterium . Diagnosacterium . Tularemacterium .tularensis (type A), if inhaled, causes a disease with high mortality if untreated, whereas the other clinically important subspecies, holarctica, very rarely causes fatal disease, although individuals may become severely ill, in particular due to the respiratory form  and serum prepared as described by the manufacturer. Serum samples were obtained at one, two, four, six and 12 months after first sign of illness. The serum was stored at 4°C and analyzed by ELISA within two days after sampling.F. tularensis LPS was a generous gift from Dr Wayne Conlan, NRC-CNRC, Ottawa, Canada. The LPS was purified as described  diluted in 0.05 M of carbonate-bicarbonate, pH 9.5 and incubated overnight at 26°C. The antigen was removed and wells filled with PBS + BSA 0.5% and stored at -80°C until use. Uncoated wells were included to control for non-specific binding. Before addition of serum samples, the plates were washed four times with PBS + 0.05% BSA. The patient sera were diluted 1,000-fold in incubation buffer (PBS + 0.05% of Tween-20). One hundred µl of the diluted serum was added to coated and uncoated wells. The plates were incubated at 26°C for 3 h and thereafter washed. The anti-IgG- and anti-IgM-conjugated alkaline phosphatase antibodies were diluted 5,000-fold in incubation buffer and 100 µl added to each well. After overnight incubation at 26°C, plates were washed three times before 100 µl of alkaline phosphatase substrate was added to each well. The reaction was stopped by adding 50 µl of 3.0 M of NaOH per well when wells incubated with the calibrator serum had reached an OD of 1.0 A405. The relative titer of each serum sample was calculated by subtracting the value of the non-coated wells and thereafter multiplying the value with the/correction factor and the dilution factor. The correction factor was calculated by dividing the nominal value of the calibrator serum, 1.0, with the obtained value of wells with the calibrator serum. Based on an extensive collection of sera from donors with no known exposure to F. tularensis, 100 - 200 was considered to be a threshold value and a value of > 200 as positive.An ELISA was used to determine IgM and IgG antibody titers. The wells of Nunc MaxiSorp plates (Sigma Aldrich) were coated with 100 µl of Since 2010, approximately 14,850 serum samples had been analyzed by the method. Of these, 1,801 were IgM positive and 2,520 IgG positive. Based on these results, it was determined that the cut-off of 100 had a sensitivity of 100% for both IgM and IgG and a specificity of 93.6% for IgG and 97.3% for IgM. The PPV for IgG was 97.0% and for IgM 99.6% and the NPV for both 100%. For the cut-off of 200, the sensitivity was 97.3% for IgG and 98.3% for IgM. The specificity was in both instances 100.0%. The PPV for both were 100% and the NPV was 99.5% for IgG and 99.8% for IgM. The intra-assay CV was 4.6% and the inter-assay CV 9.6%.t-test were used. A P value of < 0.05 was considered significant.Student`s Sera from 19 individuals with ulceroglandular tularemia were obtained on 5 occasions, 1 -12 months, after onset of disease and analyzed for the presence of IgM and IgG antibodies. Peak average antibody levels were observed at the one-month time point for IgM, 537 ± 54, and at the two-month time point for IgG, 789 ± 56 Figure 1Specific knowledge regarding the kinetics of humoral immune responses after infection is of much relevance from a clinical standpoint, since the information can be utilized to pinpoint when infection occurred and therefore be diagnostically important. Serological responses to many infections are characterized by IgM titers peaking within a couple of weeks and thereafter rather rapidly declining, sometimes disappearing within weeks or a few months, whereas IgG titers peak more slowly, but persist for much longer, at least many months . SerologThe use of serology for diagnosis of tularemia is well established and there are numerous studies demonstrating its utility, reviewed by Maurin . A numbeF. tularensis, also in individuals vaccinated with the live vaccine strain of F. tularensis, cell-mediated immunity was found to persist for three decades without evidence of decline (It is well documented that cell-mediated and humoral immune responses after tularemia are very long-lived. In fact, cell-mediated immunity may be life-long since > 95% of individuals were found to present with significant cell-mediated immune responses 25 years after tularemia . At that decline .The present study provides additional information regarding the kinetics after tularemia and demonstrated that there were elevated IgG and IgM antibody titers for at least 12 months after infection. Although the titers waned over time, compared to the average one-month IgG titer, a significant decrease for the group was not observed until the 12-month time point. Compared to the one-month IgM titer, there was significant decrease at the 4-month time point. Notably, the ≥ 69 year-group showed significantly lower antibody levels, a finding not reported previously. This is in agreement with the impaired B-cell responses occurring during ageing and concomitant increased susceptibility to various bacterial and viral infections among elderly . A limitCollectively, the data demonstrate long-term persistence of IgM and IgG antibody titers, both of which are significantly elevated during a period of 12 months. From a clinical perspective, this means that low-levels antibody titers are not indicative of acute or recent tularemia. Thus, serological diagnosis has to rely on the demonstration of significant increase of the antibody levels between two samples taken one to four weeks apart, concomitantly with a clinical presentation compatible with tularemia. Other authors have also made the same conclusions .The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.The studies involving human participants were reviewed and approved by the Regional Ethical Review Committee, Umeå 2016/335-31 and the Swedish Ethical Review Authority, 2019-01567. The patients/participants provided their written informed consent to participate in this study.HL and KE performed the experiments. AS, JE, and HL designed the study. HL and AS analyzed the data and wrote the manuscript. HL, JE, KE, and AS reviewed the manuscript. All authors contributed to the article and approved the submitted version.


Facilitator insights into the experience of implementing the living skills program and the perceived alignment of the COPM and LSP-16 with recovery-oriented practice are reported.Outcomes from participating in a six-week small-group living skills program, periodically implemented over two years with twelve adult mental health consumers, are reported as a case study. Occupational therapy and recovery-oriented approaches underpinned the development and implementation of the program thus it was important that outcome measures aligned with these perspectives. Data were gathered pre and post program using the Canadian Occupational Performance Measure (COPM) and the Life Skills Profile-16 (LSP-16) in each iteration of the program. Two occupational therapy program facilitators were interviewed about their experience of running the program and using these measurement tools. COPM occupational performance ( Mental illness is a leading global health concern  which was developed from a well-established occupational therapy framework, the Canadian Model of Occupational Performance and Engagement (CMOP-E). This framework considers the interplay between person, environment and occupation and the resultant impact on occupational performance  explore outcomes following participation in a living skills program for adult mental health consumers using the COPM and LSP-16; and (ii) capture the experiences of the program facilitators, including their perspective on the alignment of these tools with recovery-oriented practice.The need for a living skills program for adult mental health consumers at risk of homelessness was identified by a community service agency in regional Victoria, Australia. The agency services were guided by a strengths-based, recovery-oriented approach thus the living skills program needed to adopt individualised and recovery-oriented practices in its design, delivery and evaluation. Undergraduate occupational therapy students developed the living skills program in collaboration with agency personnel and professional practice educators to ensure alignment with the agency values. The program was piloted in 2017 with a small group of young adults who were living in supported community housing and evaluated using the COPM and LSP-16. The overall goal of the pilot program was to increase participants’ independence in everyday living skills and participation in the community. Participants undertook activities designed to develop skills in cooking, nutrition, financial planning, budgeting, work readiness and social interaction. The weekly program content which ran for six weeks was designed and structured around participant identified living skills goals with a focus on building on existing strengths and applying developed skills in real-world settings. The group program was reviewed after the first implementation and subsequent programs were supplemented by individual sessions addressing specific participant goals such as supermarket shopping and accessing community venues such as the local library. This enabled the consolidation of skills learned in the group setting for individuals seeking experience in applying their skills in a real-world context. In 2019, the program was expanded to include adults who had already experienced homelessness but were now living in supported community housing.For all participants, the COPM interview set COPM performance and satisfaction goals related to everyday living and was instrumental in shaping the individualised content of each program. Prior to commencing the program, consenting participants met with the program facilitator, an occupational therapist or occupational therapy student, to undertake the COPM interview and set their individual goals. Progress made towards achieving these goals by the end of the program was used as an outcome measure. Agency staff familiar with the participants completed the LSP-16. The program content was structured around the living skills goals common to a small group (four-six in each program) that could reasonably be undertaken in the group setting twice per week over a six-week period supplemented by individual sessions where necessary and possible. Common occupational domains addressed through the program included shopping, meal preparation and budgeting.The first research questions focused on individual outcomes for participants of the Living Skills program: What change occurred following participation in the program with regard to (i) participant occupational performance and satisfaction as measured by the Canadian Occupational Performance Measure (COPM), and (ii) participant general functioning as measured by the Life Skills Profile-16 (LSP-16)?The second group of research questions explored the experience of the occupational therapists running the program, specifically seeking to understand their perspective on (i) facilitating the program, (ii) the alignment of the program with recovery-oriented practice and (iii) the suitability of the outcome measures to capture change and align with recovery-oriented practice.Given the contemporary “real world” context of the living skills program, a case study approach to reporting the findings was considered most appropriate. Data were gathered from multiple sources: participant self-ratings via the COPM, support worker ratings via the LSP-16, and experiential reports via program facilitator interviews. Multiple sources of data collection converging in a triangulating manner are well suited to real-world evaluations Yin, . COMP peEthical approval to conduct the research was provided by the Human Research Ethics Committee [2017-335 H]. Data collection occurred intermittently from November 2017 to August 2020.Convenience sampling was used to recruit participants from adult consumers at risk of homelessness residing in supported accommodation provided by a community mental health service in regional Victoria. Inclusion criteria required participants to be adult consumers of the organisation with identified needs for support in everyday living who were willing to be referred to the program by their case manager. To maintain a safe environment, participants were unable to attend group activities if they were under the influence of drugs or alcohol. No other exclusion criteria were applied. Information letters and consent forms were distributed via the service coordinators and case managers. Although individuals were able to participate in the living skills program without participating in the research, the same evaluation processes were undertaken as part of the program implementation. Written informed consent was obtained and measurement tools and data analysis processes were structured to enable enhanced understanding of participant results and the experience of program facilitators. All participants of the program consented to have their data included in the research project but three did not complete the program.The Canadian Occupational Performance Measure (COPM). The COPM is an occupation-based goal setting tool that can be used pre and post intervention to measure change in occupational performance and occupational satisfaction . The abbreviated form of the Life Skills Profile  and satisfaction  significantly improved from pre-intervention to post-intervention. Individual case analysis outlining occupational performance is presented in Fig. COPM scores in occupational performance , nor across any domains from pre intervention to three-months post intervention (see Table 2). The LSP-16 was completed by the same mental health practitioner pre and post program for each individual participant.The LSP-16 showed no significant change in the total scores One male and one female occupational therapist participated in the semi-structured interviews. One facilitator, pseudonym MS, was an occupational therapist with two years of service with the organisation. The other facilitator, pseudonym NP, was an occupational therapy student when implementing the program in 2019 and had since begun working as a registered occupational therapist outside of the organisation.‘it’s not just cooking’, ‘environment affects occupational performance’, ‘person centeredness’ and ‘a need for realistic measurement tools’. Written informed consent was obtained from participants to include direct quotes. Figure Four themes were generated: Facilitators explained that a range of skills were developed through participation in the program. Cooking was outlined as a therapeutic and complex task with “a whole other set of skills underneath”. Skills developed in the program were seen as being able to be “transferred” into other settings and areas of life.Cognitive skills such as “planning, delegating and problem solving” were reported to consequently develop through participation. Such skills often complimented other participant goals such employment and organisational skills. The program occurred at a set time each week which provided structure and “routine”. NP expressed that some participants “struggled with getting to an appointment”. Thus, the program provided opportunities for development of organisational skills.Opportunities for “leadership roles”, “teaching others” and “conflict resolution” were reported. Social interaction is embedded in the program through group participation and provides a safe space to develop social skills. Furthermore, facilitators expressed that “confidence” was a personal attribute that developed through participation in the program. NP described participants as “proud” and noticed an increased motivation to cook outside of the program: “one (participant) went away and cooked her own meal……another went to the library and printed out a bunch of recipes”.Facilitators emphasised that participating in the program is much more than “just cooking” or budgeting. NP reported that the program emphasised personal development and ‘sense of self’, explaining that “you’re not just doing sessions for the sake of it being another week, you’re more so doing them because you’re going onto the next step.” He reported to challenge participants with questions such as “where do you want to go next, what’s the next step for you?”. He elaborated “it allows them to continue to develop to areas they wouldn’t have outlined in their goals”, “because it’s a confidence thing, it’s about going into those environments that would potentially trigger you and coming out with not only a skillset, but the confidence and the motivation to continue on with what the programme’s taught you.”Facilitators expressed the significant impact of the environment for consumers. The environment was conceptualised through an occupational therapy CMOP-E lens whereby the ‘environment’ is the context in which an individual performs occupations and includes physical, social, cultural and institutional elements  which situates participants as experts in their own experience and encourages them to guide the approach to their own support needs. Program facilitators reported a perceived increase in participant confidence, self-determination and self-efficacy over the duration of the living skills aligning somewhat with the ‘redefining self’ element of Davidson’s model.Facilitator interviews also indicated an increase in living skills program participants’ self-confidence and self-worth. The group nature of the program provided participants with opportunities for peer interaction, mentorship and leadership and participants regularly referred to ‘being supported by others’. Eftimovska-Tashkovska et al.,  found a It is difficult to attribute change solely to the living skills program, partly because of changes to life circumstances experienced by some participants during the program, reflecting the complexity of evaluating ‘real-world’ programs, and partly due to broad and holistic measurement tools such as the LSP-16 which may not be sensitive to change in this context. Recovery is by its nature idiosyncratic, and measurement tools which capture change in elements indicative of recovery are necessary if we are to understand the process of recovery from an everyday perspective. In this study, the program facilitators also noted a positive impact on some participants when they entered the organisation and began to access services simply because of this change in their life circumstances. Several participants transitioned from being homeless to housed at the same time as they participated in the living skills program, with corresponding improved access to social and physical supports in addition to the specific opportunities the living skills program afforded them. Henwood et al.,  noted thThe consumer movement has created momentum within the mental health sector and places emphasis on a recovery-oriented service focus and a strong consumer voice. Andresen et al.,  acknowleThere are limitations to this study. The research was conducted at one organisation across two sites with a total of 12 participants, with relatively small datasets for the two measurement tools used. Having only two participants for the facilitator interviews limited the breadth and depth of analysis and although using principles of thematic analysis which allows for flexibility of dataset size and composition (Braun & Clarke, Nevertheless, the findings indicate that a living skills intervention that is individualised and incorporates participant choice and group interaction is able to support people to achieve their goals. The results suggest that self-report measurement tools which actively engage participants such as the COPM, are able to demonstrate meaningful change. The findings invite occupational therapists and other allied health professionals to consider the applicability and relevance of measurement tools for recovery-oriented programs conducted in real-life community settings, such as the living skills program.Recovery is complex and multifaceted and influenced by more than one support or intervention, thus the importance of identifying measurement tools most closely aligned to recovery-oriented practice is paramount. It is noteworthy that the tool most closely aligned to the intervention in this instance, the COPM, was also the tool that demonstrated significant change across the participant group. The holistic, individualised nature of recovery-oriented practice has synergies with occupational therapy in the mental health sector (Kelly et al.,