The study aimed to evaluate the effectiveness of a program based on the telementoring model [Extension for Community Healthcare Outcomes (ECHO)] for primary care physicians in diagnosing and treating patients with dementia.The ECHO model was adapted through 12 live sessions of 2 h every 2 weeks consisting of a didactic presentation by the expert, peer-led new case discussions, and follow-up discussions. In addition, there were 10 h of self-paced e-learning and interim assignments. The impact was examined by noting participation, session ratings, monthly clinical reports, and comparing knowledge and competency scores before and after the course.t-test revealed a statistically significant improvement (p < 0.001) in self-rated skills and confidence in diagnosing and treating dementia with an effect size of 1.25 and 1.37, respectively. No change in the knowledge score was observed throughout the course. A considerable increase in dementia-related clinical practice was observed during four monthly summary of clinical cases. Due to the limited data of monthly reports during the COVID pandemic, no statistical analysis was attempted.Among the 63 participants, 39.7% attended at least 80% of the sessions; completing the program successfully. The ratings for all sessions ranged from “good” to “excellent.” The paired sample The ECHO model appears to have a positive immediate impact on the clinical ability of primary care physicians to diagnose and treat dementia. Its direct impact on patient health and at the community level should be aimed at in future studies. Population aging is gradually taking the global form. In some low- and middle-income countries (LMIC), the proportion of older people could double in 50 years, which had happened over 150–200 years in high-income countries (HIC). In India, too, people over the age of 60 years are projected to increase from 8.6% in 2011 to 19.5% of the total population in 2050, probably leading to an absolute number of 320 million people. With the less availability of resources at the national level and the slower recognition and response of the government to the dependence of older people, this faster rate of growth of the aging population amplifies the scarcity of resources for elderly healthcare .Research reveals a delay in the diagnosis of dementia, a delayed initiation of dementia-specific medications, and inadequate pharmacological and psychosocial management within the primary care system . The treThe Project Extension for Community Healthcare Outcomes (ECHO) is an innovative model based on the concept of telehealth to bridge the gap between a specialist and a primary care physician. Unlike telemedicine, where the specialist is in charge of patient care, the ECHO model trains and enables the primary care physician to provide specialist care in a community setting. It follows a virtual hub-and-spoke educational model using telementoring, case-based discussions, and didactics to access the best evidence-based practices and improve the expertise in specialty conditions of primary care physicians . A recenP < 0.03) with a reduced hospitalization  certificate course on dementia at the NIMHANS Digital Academy conducted for 10 months between October 2019 and July 2020. Anonymity was maintained and informed consent was obtained from all participants. Institutional Ethics Committee approval was granted. All participants were physicians who completed at least MBBS and were certified in any program on the basics of mental health  or who underwent a 2-/3-year residency course in psychiatry. All of them also needed to work with older adults at the primary care level, some of whom are people with memory problems or are being diagnosed with dementia. Doctors who were specialized in any specific field of medicine or surgery and worked with patients with complaints restricted to the respective field were excluded.1.During the participant enrollment process, we collected details about their qualification, clinical work profile, and clinical setting.2.routinely ask about memory complaints to my elderly patients.” The section “Knowledge” included a theoretically based evaluation such as “Which of the following is the preferred measure to handle psychiatric/behavioral symptoms that develop in dementia” and providing options related to specific antipsychotics and behavioral management for selection.Online (internet-based) assessment: It consisted of questions to assess the knowledge, clinical skills and confidence of participants in the screening, diagnosis and treatment of patients with dementia. This questionnaire was initially administered and then after completing the entire course of live sessions. All sections used the self-rated assessment pattern. For example, participants had to select one of the five options related to the frequency of the clinical skill suggested as “I 3.Monthly summary of clinical work related to older adults: This information was collected through an online form at the end of each month from October 2019 to July 2020. It examined the clinical aggregates of older adults seen by the doctors in their clinic and, among them, the number of older adults with memory problems, diagnosed with any dementia, MCI or Alzheimer’s disease, and the treatment provided for dementia in terms of cholinesterase inhibitors, psychotropics, and non-pharmacological treatment.4.Session ratings: Participants were asked to anonymously rate each session using a five-point internet-based Likert scale at the end of the live session. The ratings considered were as follows: “poor, average, good, very good, and excellent.”5.Attendance was recorded for all live sessions.Participants underwent a digital training titled “Dementia: Diagnosis and treatment for physicians” conducted by a team of psychiatrists providing clinical services and training for geriatric psychiatry at NIMHANS, an academic institute with tertiary-level healthcare facilities. Following the hub and spoke model, our team had 4 key experts on dementia, PSi, MV, PTS, and SG. They work primarily with geriatric patients with dementia and other psychiatric disorders. We also had PC, SN, and PSa with expertise in implementation of an ECHO based program. The dementia expert group finalized the course curriculum after peer review by dementia experts outside of NIMHANS. It covered the screening for cognitive problems, the history and mental status examination of an older adult, the structured assessment of dementia, the salient characteristics of mild cognitive impairment and Alzheimer’s disease, distinguishing them from other dementias, the management of behavioral and psychological symptoms in dementia, the utility of cognitive enhancers and non-pharmacological treatment in dementia, and prevention of dementia.This certificate course was conducted in association with Project ECHO–India. Participants were briefed on the principles and process of ECHO, the program curriculum, and the requirements for course certification before starting it. They were encouraged to ask questions, although no workshops were conducted to train on these aspects in advance. The curriculum was covered through a blended teaching technique that included virtual live video conference sessions every 2 weeks and self-paced e-learning. Live sessions were held for 2 h on the first and third Mondays and accounted for 24 h with 12 sessions. In self-paced e-learning, participants were involved through Google Classroom for a total of 10 h. Curriculum-based resource materials, assignments, and self-assessments were administered through this platform. This asynchronous e-learning is an enhancement to the synchronous learning of ECHO tele-clinic. It has been adapted to all programs of NIMHANS ECHO with the objective of having more self-controlled learning, specifically factual once , 21.The live session had a didactic presentation on a predetermined topic related to one of the domains mentioned above. It lasted 30–45 min, including the topic-related interaction between the expert and the participants. The remaining time of the 2 h live session was dedicated to case discussion. Here, one or two participants presented their respective deidentified case of suspected or diagnosed dementia, which they noted in clinical work, and faced challenges in diagnosis and treatment. The specific case presentation form was developed to cover all relevant history and examination of cognitive, psychiatric, and medical aspects together with a structured cognitive evaluation. Then a discussion was followed with the expert and other participants and ended with the plan for further evaluation and management of the case. The cases discussed were also followed up in subsequent sessions to see the result of the planned recommendations. All sessions were conducted using a web-based videoconferencing application that is compliant with the privacy and protection of health information according to the “Digital Information Security in Healthcare Act (DISHA), 2018” of the Government of India . All livThe outcome evaluation of the course was performed with the Moore’s continued medical education (CME) evaluation framework .–Level 1, Participation: A minimum of 80% attendance for live sessions was taken as one of the criteria for awarding certificates. The proportion of participants who achieved this criterion was noted.–Level 2, Satisfaction: The satisfaction of the participants was assessed by an anonymous rating of each live session provided by them.–Level 3, Learning and Knowledge: To receive certification, the participant had to secure a minimum of 80% in all interim assignments in 5 attempts over 2 weeks and present 2 cases during live sessions or otherwise to experts. Additionally, knowledge was assessed using the online questionnaire conducted before and after the course.–Level 4, Competence: Participants conducted a self-assessment of clinical skills and confidence using the online assessment questionnaire before and after the course.–Level 5, Performance: This was assessed using a monthly summary of the clinical cases seen by the participants. Details are described above.t-test was used to compare scores before and after the course. Data were evaluated at a 0.05 significance level.Normality tests were applied to examine the distribution of the data. The frequency distribution of the demographic characteristics of the attended and accredited groups was compared with the chi-square to find any significant differences between the two groups. The knowledge, skills, confidence, and interest scores in learning were summarized in mean and standard deviation. The Paired Samples n = 25) of these 63 physicians completed the program and were accredited. 53 (84.1%) physicians had previously attended one or more ECHO programs . The sociodemographic characteristics of both groups are described in A total of 108 physicians had expressed interest in attending the program. Among them, 72 were eligible for participation in the program and 63 attended one or more sessions. 39.7%  compared to the accredited group (24%). However, the chi-square analysis did not reveal significant differences between the two groups for any characteristics. Most of the attending physicians and those certified were men and worked in government hospitals in an urban setting. The largest work profile in both groups was the medical officers of primary health centers or district hospitals.Participants were asked about the expectations of the program prior to its onset. The main expectations were as follows.–Improve skills in screening for dementia.–Learn how to diagnose and treat patients with dementia.–To understand cognitive problems in older adults.–To learn specifically about dementia due to Alzheimer’s disease.a.n = 33) of the physicians attended at least 6 of the 12 sessions, that is, 50 and 39.7% (n = 25) attended 10 sessions, that is, 80%.Participation: 52.4% (b.Satisfaction: 45–90% of participants in different sessions gave an “excellent” rating. Other scores received for the sessions were “very good” and “good.” None of the sessions received a “poor” rating from any participant. The detailed distribution is presented in c.Learning and Competence: The case presentation and interim assignments were satisfactorily completed for certification by 25 participants. The cases presented ranged from mild cognitive impairment (MCI) to severe dementia. Many of the cases were presented to discuss the management of associated behavioral and psychological symptoms. There were also difficulties in diagnosing MCI, differentiating dementia from psychiatric disorders, and addressing metabolic factors associated with dementia. Otherwise, the type of dementia was limited to Alzheimer’s disease and Vascular dementia. The cases presented also had comorbidities ranging from diabetes, hypertension, and nutritional deficiency to ischemic heart disease, stroke, and psychiatric disorders such as depressive disorder, bipolar affective disorder, and schizophrenia. These comorbidities presented challenges in pharmacological and non-pharmacological treatments and were addressed accordingly.t-test revealed a statistically significant improvement in the skills and confidence of certified participants with an effect size of 1.25 and 1.37, respectively. The level of knowledge and interest in learning did not change from the pre- to post-phase of the program.All accredited participants completed the pre-and post-assessment questionnaires. The paired sample d.n/N = 231/293) compared to those in October 2019 (n/N = 93/185). There was also a gradual increase in patients managed with pharmacological and psychosocial treatments during these 4 months.Performance , 3: We pWith the advancement in videoconferencing technology and its ease of availability in recent years, the scope of education and training through telementoring has gained momentum in the medical and surgical fields , 25. We The results showed a significant improvement, that is, approximately 1.5 times, in the clinical skills and confidence of accredited physicians in the screening, diagnosis, and treatment of dementia after completion of the program. The results reflect the comprehensiveness of the course in providing clinically relevant material. The regular provision of assignments would have helped participants develop self-reliance to handle elderly with cognitive problems, which is also reflected in the monthly reports of the participants showing a trend of improvement in clinical practice. For 4 months, there was an increase in the number of older adults screened for memory disturbances identified with Alzheimer’s disease and other dementias and the use of pharmacological and psychosocial methods in treatment. Other studies based on the ECHO model dealing with complete geriatric medicine , 14 or spro forma provided in this training program. Instead of a weekly schedule as in the original ECHO model, this program had fortnightly sessions to provide adequate time to grasp self-learning materials and assignments. This frequency has been adopted in some ECHO programs, including dementia care-related programs  received in 80–90% of the responses in all sessions. The participants liked the sessions related to the treatment aspects and the didactics presented through case vignettes.According to Moore’s CME evaluation framework, this program achieved good results in satisfaction, learning, competence, and the participants’ performance . The 25 The program was designed to enable physicians to diagnose and provide comprehensive treatment for patients with dementia. This approach is similar to the earlier Dementia 360 telementoring program conducted for clinicians in the Oregon state of the United States . Other pThis study is based on only one cycle of the program. Therefore, the program had a restricted number of accredited participants, i.e., 25. A smaller proportion (40%) of the total participants were accredited, which led to a small sample size. Increasing the number of accredited participants at the cost of a possible effective course and fidelity to the ECHO program was not preferred. This small sample prevented us from effectively analyzing the impact of the program on various parameters and could not have enough power to detect the change in other aspects of the evaluation of the program. Changes related to COVID-19 further limited long-term follow-up of physician clinical activities. Immediate changes in this program related to patient health and community level, levels 6 and 7 of Moore’s expanded CME framework, were not considered. This change would require regular updates on the specific patient outcomes of accredited participants. In this regard, adding an e-consult, such as constant support and feedback to the accredited group from experts through online consultation, may help achieve the outcome. As dementia involves subjective evaluation of behavioral problems, an e-consulting facility for physicians to reach out could help them improve their clinical skills and help monitor patient health more effectively. Additionally, community-level changes would require an increase in the number of accredited participants from the confined region, which can be planned in later cycles of the dementia ECHO program. Including the training of other mental health service providers, such as home care providers and psychiatric social workers, in our program would improve community-level outcomes. This study could also encourage the use of ECHO models to train accredited physicians in this program on other dementias and various other aspects of geriatric health.The ECHO model appears to be a promising innovation to bridge the treatment gap in dementia. In a country like India, where the prevalence of dementia is expected to increase drastically, but experts are limited, the ECHO model shows that videoconferencing-based telelearning and clinics can train and empower a large number of primary care physicians remotely in different parts of the country. The collaborative network based on the ECHO model of experts and primary care physicians should be built and further evaluated for its impact at the patient and community level.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by Institute Ethics Committee, Behavioural Sciences, National Institute of Mental Health and Neurosciences (NIMHANS). The patients/participants provided their written informed consent to participate in this study.PSi oversaw the training program mentioned here, supported by the equal contribution of SN and PSa, MV, PTS, PC, and NG. PSi and SN prepared the manuscript, with support from the other authors. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Adolescents with physical disabilities have higher rates of mental health conditions and issues than adolescents without disabilities, and this disparity was exacerbated by the onset of the COVID-19 pandemic. They also have limited access to on-site programs and nearby peers.This pilot aims to investigate the potential effects of a low-dose multiplayer virtual reality telegaming program on depression, socialization, and loneliness among a cohort of children with physical disabilities. A secondary aim is to describe feasibility metrics, namely, recruitment and adherence rates and perceived program enjoyment and satisfaction. The tertiary aim is to describe behavioral mechanisms that affect participant adherence and social participation in the classes.t tests as appropriate.This study is a single-group pre- and posttest–designed trial. A single cohort of 12 children with physical disabilities will pilot a 1-month program that includes 2 supervised 1-hour sessions per week of group-based exergaming. Participants will complete questionnaires before and after the program. The primary aim measures will include the Children’s Depression Inventory 2 Short Form, a measure of feelings of depression, and the UCLA Loneliness Scale, a measure of both loneliness and social isolation. Secondary aim measures will include three posttest Likert scale questionnaires: perceived program enjoyment, program satisfaction, and satisfaction with multiplayer experiences. At postintervention or dropout, participants will undergo semistructured interviews to identify behavioral mechanisms that underlie participation. Data will be reported descriptively and be supported by Recruitment procedures started in July 2022. All data are expected to be collected by January 2023. Full trial results are expected to be published by March 2023. Secondary analyses of data will be subsequently published.This trial tests a peer-to-peer virtual reality telegaming program that includes a completely remote enrollment, assessment, and intervention protocol. This program is accessible and short in duration and frequency, allowing it to be integrated into other interventions. Knowledge obtained from this study will inform the development of a larger trial for improving the mental health and well-being of adolescents with physical disabilities.ClinicalTrials.gov NCT05259462; https://clinicaltrials.gov/ct2/show/NCT05259462PRR1-10.2196/42651 For people with physical disabilities, adolescence is a critical period for school educators and health professionals to provide quality education and health services. In adolescence, people with disabilities build their self-identity and adopt vocational skills and health behaviors that increase the likelihood of living healthy independent lifestyles as they transition into early adulthood -3. PriorDisappointingly, reports from the COVID-19 era have found that stress, depression, isolation, and physical activity participation have substantially worsened since the outbreak of COVID-19 among adolescents with disabilities -17. A loCOVID-19–created barriers to physical activity opportunities; virtual reality group exercise-filled voids in exercise participation and human connectedness that were created by the COVID-19 pandemic; and the virtual reality headset fostered immersion, which increased their motivation to game. In summary, group virtual reality gaming provided an accessible opportunity for adolescents with disabilities to build new friendships with peers, which enhanced their mental health. There is a need to start confirming these benefits through quantitative investigation, particularly among adolescents living in the Southeast, a geographic region with some of the lowest rates of health care access across the United States  [In April 2020, one author (BL) and an adapted fitness instructor conducted a home-based community virtual reality program for adolescent members of a fitness facility for people with disabilities. The program included low-cost virtual reality equipment, and people could participate from home during the height of the COVID-19 pandemic amid social distancing mandates, occupancy limits, and facility closures. The program was 1 month in duration and included two 30-minute sessions per week of synchronous group exercise. The program included a total of 10 participants, completed in two intervention waves (n=5 per wave). Since this project was not a research study, the anecdotal resultant themes were ndation) .This study has 3 purposes. The first purpose is to examine the effects of a home-based virtual reality multiplayer gaming program on depression, socialization, and loneliness among 12 adolescents with physical disabilities (ages 13-19 years). The second purpose is to describe the acceptability of the program as measured by participant adherence, total play time, exercise time, and perceived program satisfaction and enjoyment. The third purpose is to describe behavioral mechanisms that affect participant engagement. We will qualitatively explore participants’ perceptions of barriers and facilitators that affected their adherence and social participation in the classes.This study is a single group pre-to-post design trial that will last 1 year and include 2 waves (n=6 per wave). Participants will be recruited from the medical and billing record databases at the Children’s Hospital, as well as through the collaborative network of the university and an adapted fitness facility for people with disabilities . We plan to recruit a convenience sample of 12 people to satisfy minimum recommendations for a feasibility study that aims to inform sample size considerations for a larger trial .The protocol and informed consent and assent forms were approved by the Institutional Review Board for Human Use of the University (IRB-300008913) on May 23, 2022. Prospective participants and their caregivers will be mailed a consent and assent form, along with the study questionnaires. Participants will be instructed to review the consent and assent forms carefully, sign them, and then complete the study questionnaires. Once completed, they will be instructed to mail the documents back in a return envelope to research staff. Should the participants not want to participate, no action will be needed. Consent and assent documents will be written in English.Eligibility criteria will include self-reported mobility disability , between the ages of 13-19 years , access to a Wi-Fi internet connection in the home, and a caregiver to support the child if <18 years of age. The exclusion criteria will be as follows: physically active (defined as >150 minutes/week of moderate-to-vigorous intensity exercise), cannot use their arms for exercise or operate the controller buttons using their fingers, and complete blindness or deafness.Before starting the intervention, participants will be mailed consent/assent forms and baseline surveys. Once the forms are received by research staff, participants will be mailed the intervention equipment. Post intervention (week 5), participants will be mailed another packet of surveys to complete and return to research staff. Participants will then be asked to participate in a one-on-one semistructured interview via phone call or Zoom  with the principal investigator, which will be audio recorded and transcribed for qualitative analysis. Participants will receive an electronic US $40 gift card for each packet of surveys returned .learn-as-you-go approach based on participants’ preferences.The program will last 4 weeks and include two 1-hour sessions per week of supervised peer-to-peer gaming. Further details of the program prescription are shown in Baseline participant characteristics will include age, sex, and ethnicity. Aim 1 measures will include the Children’s Depression Inventory 2 Short Form, a measure of feelings of depression with strong psychometric properties among adolescents with and without disabilities ,24, and Aim 2 will be evaluated through feasibility metrics: recruitment rates, implementation and resource management issues, participant attendance in class  and outside of class (play time not within the scheduled class sessions), and program satisfaction and enjoyment. Two components of satisfaction will be measured, namely, satisfaction with program delivery and satisfaction with playing with others. Both components will be measured by Likert scale surveys . Survey Aim 3 will be measured through one-on-one semistructured interviews of participants’ perceptions of the program. Interviews will be conducted via phone call with the participant and principal investigator (BL). Interviews will last up to 30 minutes and contain 10 general questions, examining barriers and facilitators to adherence, likes and dislikes with the program and equipment, and recommendations to improve the program . GeneralThe research team’s philosophical assumptions will align with dialectical pluralism . This pat tests to compare pre-to-post changes in survey scores. Aim 2 feasibility metrics will be reported in a descriptive manner with no a priori criteria for acceptability.As a preliminary study, the study is not powered for efficacy. Instead, the findings will provide outcome estimates that will inform sample size and design considerations for a larger trial. Descriptive statistics for all quantitative outcomes will include means, SDs, effect sizes, box and whisker plots, and 95% CIs as appropriate. Aim 1 will include substantive contribution [coherence will be sought by ensuring that qualitative study procedures throughout the project align with the goals of the study [transparency will be maintained throughout the study by receiving in-depth feedback from a critical friend [Aim 3 will be analyzed using a qualitative 6-step process of thematic analysis  by two mribution , that ishe study . Third, l friend . The quaThis study was approved by the university institutional review board on May 23, 2022. The study was initiated in February 2022, and the first participant was enrolled in July 2022. Recruitment of the last participant is anticipated in October 2022.This study will investigate the preliminary benefits of a home-based mindfulness program that is combined with peer-to-peer virtual reality gaming on mental health among adolescents with disabilities. A program that is beneficial toward depression, anxiety, and isolation, and connects peers remotely at the home using low-cost equipment could greatly benefit this population, which has experienced poorer mental health since the outbreak of the COVID-19 pandemic . The quaIn addition to concerns of poor mental health among adolescents with disabilities, health professionals are often concerned with maintaining or improving physical function ,36 or alAn innovative component of the study is that it incorporates remote study procedures. Due to difficulties with allocating transportation, a remote study procedure increases accessibility by eliminating the need for on-site visitation. This study has limitations. The sample size is small, and the results will need to be confirmed with larger clinical trials. Additionally, this study requires participants to be able to use the handheld controllers and view the screen of the virtual reality headsets, which may not be appropriate for some people who have functional or visual impairments.Should the findings of this study suggest a potential benefit toward some aspects of mental health, the study may discover an innovative and, most importantly, scalable method for addressing mental health among children with physical disabilities.


Performance in early courses significantly predicted the performance in advanced courses and experiential performance (ρ 0.48–0.61). Conclusion: The correlations between early and late course performance demonstrated the cohesiveness of this program. Further study is needed between the predictors of success using non-cognitive admission criteria.Background: To establish the predictors of success in an international-trained PharmD (ITPD) program between admission criteria and academic performance. Methods: The primary outcome of this study was the correlation of admission criteria with didactic and experiential grade point averages (GPA) for the first 5 years. Candidates meeting the minimum criteria completed a competency exam or the US-Foreign Pharmacy Graduate Equivalency Exam (US-FPGEE). Tests of English language proficiency (TOEFL(R) and ACTFL’s Oral Proficiency Interview) plus interview with faculty, students, and alumni were also required. Scores were correlated with both didactic and experiential GPAs. Results: The 23 students admitted to the ITPD program had a cumulative GPA of 3.72. There was a significant correlation between total admissions score and the median pharmacy and healthcare course category GPA (ρ 0.53), but not other categories. The composite TOEFL did not predict any performance but TOEFL writing and speaking did correlate with advanced pharmacy practice experience (APPE) performance. The OPI scores were associated with higher GPAs overall, in advanced integrated clinical sciences, and APPEs. The admission interview scores consistently and significantly correlated with preceptor-rated APPE GPA, practitioner skills, and professionalism (ρ > 0.5;  As the profession of pharmacy continues to evolve, it becomes imperative that pharmacy education considers approaches that diversify the profession, globalize pharmacists’ patient care, and supply advanced pharmacy practitioners that meet the needs of local healthcare communities across the world ,2. AlthoThe Accreditation Council for Pharmacy Education (ACPE)-accredited International-Trained PharmD (ITPD) program was established in 2014 at the University of Colorado Skaggs School of Pharmacy and Pharmaceutical Sciences (CU SSPPS) to address the need for pharmacists’ patient care training internationally. This advanced-standing entry-level Doctor of Pharmacy (PharmD) program provides a combination of didactic and experiential course work designed for pharmacists educated and practicing outside of the United States. The overarching goal of this program is to encourage the advancement of pharmacists’ patient care in the student’s home country.In accordance with the 2016 ACPE accreditation standard 17 appendix 3, PharmD programs should conduct correlations between admission variables and academic performance [The ITPD curriculum of the CU SSPPS was designed as an advanced-standing entry-level PharmD curricular pathway for practicing pharmacists from around the world who aim to advance pharmacists’ patient care in their local healthcare communities. While a separate pathway, it is aligned with the school’s traditional PharmD pathway by sharing ability-based outcomes, faculty members, and a majority of the didactic content. The curriculum includes 90 semester credit hours delivered in a hybrid format. This encompasses online didactic classwork and two in-person sessions of interactive didactic classes that correspond with elements of the experiential curriculum. It also includes in-person introductory pharmacy practice experiences (IPPEs), an advanced introductory pharmacy practice experience (aIPPE), and advanced pharmacy practice experiences (APPEs). As the program is designed to be flexible, time from program admission to completion spans 3 to 6 years. Prior publications have detailed success in tailoring educational methodologies to learning outcomes in this program ,9.Advanced standing into the program is assured through requiring a baccalaureate degree (or higher) in pharmacy and that candidates have practiced as a pharmacist for at least one year. Applicants also have to successfully pass two foundational competency exams in biomedical and pharmaceutical sciences. These exams align with coursework commonly covered in baccalaureate pharmacy programs and the first and second years of the CU SSPPS entry-level Doctor of Pharmacy Program. They include topics such as: biochemistry, cell biology, mechanisms of disease, pharmaceutics, medicinal chemistry, microbiology, pharmacology/toxicology, pharmacokinetics, drug design and drug action, immunology, and basic anatomy. In place of these exams, applicants may demonstrate competency via the passing of the US-Foreign Pharmacy Graduate Equivalency Exam (US-FPGEE).®) exam and the American Council on the Teaching of Foreign Languages (ACTFL) oral proficiency interview (OPI) exam. Upon meeting these qualifications, applicants are invited to video-conference interviews with multiple faculty members, a current ITPD student, and an ITPD alum. Interviews assess methods of learning and researching, critical thinking and decision making, adhering to principles and values of the profession, commitment to pharmacists’ patient care in their home country, and communication and interpretive abilities . These were assessed using a quantitative and qualitative standardized rubric. Scores on each category ranged from 0 to 3 and 0 to 9 through an internally created rubric, with the final interview score calculated as a percentage of possible points, and the multiple interviewer scores averaged. In addition to numeric score, each interviewer provided a qualitative rationale for scoring as well as a qualitative overall impression of each candidate, which were reviewed with candidate scores through our admissions committee.The ITPD applicants are required to demonstrate English language proficiency through the Educational Testing Service’s Test of English as a Foreign Language  as a secondary research study with a waiver of informed consent. The primary outcome of this study was the correlation of individualized student quantitative admission criteria with academic performance  completed for the first 5 years of the program’s offering with analysis completed at year 5 of the program’s offering (Summer 2019). The admission criteria evaluated included a cumulative admissions score, TOEFL scores , OPI scores, FPGEE scores/performance on biomedical and pharmaceutical sciences competency exams, and interview scores. The academic performance criteria included mean cumulative programmatic grade point average (cGPA) on a 4.0 scale and median grade point average (GPA) within major didactic course categories . Correlations that did not express a monotonic relationship were excluded from the analysis. Other comparisons  were conducted using the non-parametric Wilcoxon rank-sum test using the same threshold for statistical significance. All statistics were conducted using JMP Pro Statistical Software .Since a majority of the data were assumed to be non-parametric in nature, a Spearman’s rank-order correlation test was used to assess the strength and significance of each correlation. The strength of correlation was expressed in ρ values, with 0.10–0.39 considered to be a weak correlation, 0.40–0.69 moderate, 0.70–0.89 strong, and >0.90 as very strong . CorrelaTwenty-three students had been admitted to the ITPD program in its first five years (2014 to 2019). Students earned their original pharmacy degree from 14 unique countries outside of the United States. At the time of the writing of this manuscript, 21 students have graduated and 4 are completing APPE rotations. Two students were unable to continue the program due to non-academic reasons and they were not included in the analysis. At admission, six students had a graduate-level degree and four were employed at Joint Commission-accredited healthcare facilities. The applicants had a mean of 5.5 years (±SD 4.73) of prior pharmacy practice experience.These students earned a cGPA of 3.72 (±SD 0.31). Median GPA for major course categories was 3.92 (±IQR 0.24) for foundational integrated clinical sciences (fICS); 3.86 (±IQR 0.48) for pharmacy and healthcare; 3.84 (± IQR 0.6) for professionalism and communication and informatics; 3.67 (±IQR 0.36) for advanced integrated clinical sciences (aICS); and 3.52 (+/− IQR 0.67) for integrated clinical sciences-pharmacotherapies (ICS). Experiential GPAs were similar to those of didactic coursework, with a median GPA 4.0 (±IQR 0.2) for IPPEs (including aIPPEs) and 4.0 (± IQR 0) for APPEs.p = 0.01) but not cumulative cGPA or other course categories (p = 0.03) and cGPA , and TOEFL writing scores were significantly correlated with fICS GPA  and APPE practitioner scores . Both the TOEFL reading and listening scores, however, were not correlated with any didactic, experiential, or cumulative GPAs. The OPI scores were associated with a higher aICS GPA , APPE GPA , and a higher cGPA . The interview scores were not significantly correlated with any didactic course categories or cGPA but were the strongest predictor for nearly every aspect of APPE, but not IPPE, experiential assessment. The significant correlations included interview scores and APPE GPA , APPE practitioner skills , and APPE professionalism scores  and a non-significant trend toward APPE preceptor rating of communication .Full results for the assessment of the primary outcome are presented in tegories . Althougp = 0.01) and communication . Of students who took the FPGEE (n = 5), there was a very strong and significant correlation between FPGEE score and foundational and integrated sciences GPA  and cGPA . The interview scores were not significantly correlated with performance in any major didactic course category GPA or cGPA. In addition, performance on the biomedical sciences and pharmaceutical sciences competency exams did not predict cumulative GPA or course category GPAs.When considering assessments of advanced standing, the foundational competency exams showed limited correlation with academic performance. The performance on the biomedical sciences competency exam had no correlation with any didactic or experiential course, or cumulative GPA, while the pharmaceutical sciences competency exam scores had a significant negative correlation with APPE preceptor ratings of a student’s practitioner skills . The presence of a previous graduate degree (p = 0.16) or employment at an international joint commission certified health care facility (p = 0.18) was not associated with a significant difference in cGPA. However, students who had prior graduate degrees performed significantly better in the aICS course category compared with those who did not have a prior graduate degree . A similar trend was noted among the pharmacy and healthcare course category and prior graduate degree . Conversely, students employed at joint commission certified facilities performed significantly lower in advanced ICS courses vs. those who were not employed at such an institution . Employment at an international joint commission certified facility or the presence of a prior graduate degree did not significantly predict any experiential grade performance (p > 0.05).Assessment of secondary outcomes demonstrated that years of past experience had no correlation with GPA of any major didactic or experiential course categories with the exception of a negative correlation between past experience and the pharmacy and healthcare GPA , interprofessional ethics , public health , instructional methods , professional skills development , and management and leadership . The TOEFL composite score significantly predicted higher performance in management and leadership , communication , clinical skills foundations , clinical reasoning and decision-making , and clinical capstone . Within the individual components of the TOEFL exam, TOEFL writing scores predicted success in communication , drug information , clinical skills foundations , capstone , and professional skills development . TOEFL speaking predicted performance in management and leadership , communications , GI/Nutrition , and clinical reasoning and decision making . OPI score significantly predicted higher performance in both the pharmacotherapy of endocrine, hematological, pulmonary, urological disorders and women’s health course  and the clinical capstone course .In terms of individual courses, total admissions score also predicted higher performance in the infectious diseases course  and advanced integrated clinical sciences  Similar significant correlations were noted for the pharmacy and healthcare courses and pharmacotherapy courses  as well as the advanced pharmacotherapy courses . Significant correlations were also noted between the communications and informatics courses and pharmacotherapy courses , and the advanced pharmacotherapy courses . IPPE experiential GPA was significantly correlated with performance in the course categories of foundational integrated sciences  and advanced integrated clinical sciences . The performance in all major didactic course categories correlated well with APPE rotation performance .Once admitted to the program, higher performance in the ITPD foundational course categories significantly predicted success in advanced courses and the experiential curriculum . The perEstablishing effective admission criteria is essential to the success and function of a pharmacy program. Prior pharmacy school admissions studies have shown that traditional cognitive measures  had strength of correlation of less than 0.30 with didactic performance ,12,13. AOur results suggest that the select aspects of our admissions criteria predict performance in specific aspects of the curriculum as opposed to a holistic prediction of cumulative GPA in an international group of pharmacy students. For example, there was a very strong and significant correlation between performance on standardized testing (FPGEE) score and GPAs in the program. Our non-cognitive criteria, such as interview scores, were one of the strongest factors predicting success in the experiential component of our program. This reflects the need for assessment of these “soft” non-cognitive skills as part of the admissions process and substantiates prior literature suggesting these criteria may better predict experiential performance . ImportaThe reasoning for the variability we observed in our correlations may be multifaceted. These results may reflect the complexity of the profession, requiring clinicians to operate on high levels of critical thinking, data integration, and communication to successfully practice. Results from a previous study have suggested testing of English language understanding does not predict academic success within a program in non-native English-speaking students. That study is limited in scope as it solely evaluated composite TOEFL and non-native English-speaking students who came from only one geographic region . Our resOur results also suggest that, overall, our weighted admissions score was unable to predict success across all aspects of our program. Yet, our broad criteria are able to identify global pharmacy professional applicants who can succeed in a US-based PharmD program. This may be reflective of not only our admission criteria, but also the common foundational pharmacy education and motivation of the individuals successfully admitted. The correlations between both early and late didactic courses reflect the educational cohesiveness of our program. Some of the weaker correlations between didactic coursework and experiential performance suggest that skillsets in the classroom may not fully translate to a practice-based setting, but predictive ability did remain valid for major course categories. This is consistent with prior research suggesting that targeted assessments and situational judgment testing are more likely to predict experiential readiness than GPA or pre-pharmacy performance (PCAT or pre-pharmacy GPA) .This study is met with several limitations. We could not perform a multivariable regression due to the smaller sample size of this study, prohibiting the number of predictor variables for an effective regression model and as such, our correlations should be considered associative in nature. Secondly, because ITPD students performed exceedingly well in the program, there was a small degree of variance in cumulative GPA. This is mirrored by prior studies showing exceptional performance of non-traditional PharmD pathways . This, cThis study demonstrated that specific aspects of the TOEFL, OPI, and other admission criteria predicted academic success in didactic coursework for the ITPD program. This study adds to the literature regarding predictors of programmatic success in the didactic and experiential setting in a multicultural student population. Further research is needed regarding the predictors of academic success in healthcare programs with international student populations.


Digital mental health interventions have the potential to increase mental health support among isolated older adults. However, the older adult population can experience several barriers to accessing and using digital health resources and may need extra support to experience its benefits.This paper aimed to understand what older adults experience as an important aspect of support during engagement in a digital mental health program. The program entailed 3 months of staff support to participate in digital literacy training and engage with the digital mental health platform myStrength, which offers support for a range of mental health challenges, including depression and anxiety.A total of 30 older adults participated in surveys and interviews to assess their experience of participating in a digital mental health program provided by county mental health services. As part of the program, participants attended 4 classes of digital literacy training, had access to the digital mental health platform myStrength for 2 months with staff support (and 10 months after the program without support), and received support from program staff during the entire 3-month program. Survey data were analyzed using descriptive statistics, and interview data were analyzed using thematic analysis.A thematic analysis of the interview data revealed that participants valued ongoing support in 3 main areas: technical support to assist them in using technology, guided support to remind them to use myStrength and practice skills they had learned, and social support to enable them to connect with others through the program. Furthermore, participants reported that social connections was the most important aspect of the program and that they were mainly motivated to participate in the program because it was recommended to them by trusted others such as a community partner or because they believed it could potentially help others.Our findings can be used to inform the design of future digital mental health programs for older adults who may have unique support needs in terms of dedicated technical support and ongoing guided support to use technology and social support to increase social connectedness. Mental health is an increasing concern among the older adult population. Even before the global COVID-19 pandemic, social isolation among older adults  was a significant public health issue . In addiDigital mental health interventions (DMHIs) have the potential to remotely increase access to mental health support. Mental health support refers to any type of support aimed at protecting or promoting psychosocial well-being , such asTo better understand older adults’ perspectives toward the concept of mental health technologies, the study by Andrews et al  held a dExposure and awareness alone appear to be insufficient for older adults to engage with DMHIs. A recent report by the Substance Abuse Mental Health Services Administration highlighted the need for support and digital training when offering DMHIs to older adults . It is wPrior recommendations to provide support have been mainly evaluated from an operational  perspective, with the aim of facilitating successful implementation of an intervention and achieving desired outcomes, rather than from a user’s perspective. For example, several recent studies have focused on the effectiveness of DMHIs with older adult participants and have provided support to older adults in the form of therapists, health professionals, research assistants, and coaches ,17,19. SIt is important to consider not only what types of support older adults need but also what they value, as participants’ experiences affect real-world use and can be crucial in whether they would continue to engage with a digital intervention beyond a research setting.This paper reports on a study situated within a digital mental health program with older adults. The aim of this study was to evaluate older adults’ experiences with the program and specify what they experienced as important aspects of support while they were taking part in the program. The research question we aimed to address was as follows: What do older adults experience as important factors of support during engagement in a digital mental health program?Thirty older adults participated in a digital mental health program for 3 months. The aim of the program was to engage English-speaking and Spanish-speaking isolated older adults with technology and to improve their well-being and sense of social connectedness. Participants experienced several types of isolation. First, everyone experienced some degree of social isolation as the program took place during the global COVID-19 pandemic. Second, older adults were selected based on geographic isolation, and finally, Spanish-speaking participants were also selected based on perceived isolation by program staff due to language, culture, and generational barriers.Participants received digital literacy training as part of the program. Participants were given free access to the digital mental health platform myStrength for 1 year, and those who did not have the required resources to use myStrength were provided with tablets and an internet connection.Participants were given 1-year access to myStrength, a digital and mobile-based mental health platform . myStrenDuring the program, participants received weekly phone calls from the program staff to check in and to encourage them to use the platform. In addition, participants received at least one in-person visit from nurses to help them access and use technology. myStrength is offered in both English and Spanish. The Spanish version offered some content such as videos in Spanish, but the platform was not fully translated into Spanish .To address our research question, we conducted a mixed method study, in which participants took part in 3 surveys and an interview during the program, and 1 follow-up survey 8 months after the program  and Hispanic, with a population of approximately 259,000 people, of which approximately 22% are aged ≥65 years . The proPhase 1: Onboarding participants by giving them access to required resources, such as a device and an internet connection, to be able to participate in the program (4 weeks)Phase 2: Providing digital literacy training consisting of four 2-hour web-based classes to establish or improve participants’ digital literacy skills (4 weeks)Phase 3: Providing participants free access to the myStrength platform and nurse promotores support (8 weeks)Phase 4: Debriefing participants at the end of the program and conducting surveys and interviews (8 weeks)Phase 5: Providing continued free access to myStrength after completion of the program (10 months)When referring to the program, we refer to phases 1 to 4 as the 3-month period during which staff support was provided to the participants.There were 5 key data collection points : 1 surveThe data collection instruments were developed in English. The first 3 surveys and the interview guide were translated into Spanish by an external translation vendor. These translations were reviewed for accuracy and readability by 3 Spanish-speaking members of the research team. For example, vendor translation sometimes used formal words that are not typically used in everyday language, or the intent of a question was lost. These translations were updated to better encapsulate the original intent of the questions. The fourth follow-up survey was translated directly by the Spanish-speaking members of the research team.Digital literacy training is offered in both English and Spanish. The training consisted of four classes on the following topics: (1) computer basics, (2) internet basics, (3) email basics, and (4) myStrength. All digital literacy classes were voluntary for both participants and program staff.promotores (volunteers from the North Marin Community Service’s Promotores Program that work closely with the Spanish-speaking community in Marin County). Marin County chose to partner with nurse interns and promotores because they were trained and experienced community partners; furthermore, promotores were trusted community leaders in the Spanish-speaking community. The promotores network works to bridge and connect the community with health services and community engagement . The suTechnical support, such as helping participants get connected to Wi-Fi,support during digital literacy training, such as teaching participants how to use their devices and Zoom app to participate in digital literacy classes,checking in with participants about their participation in the program, andconnecting participants to resources and mental health services.Participants were also offered the option of having a staff member visit them in person or provide remote support. Support was ongoing throughout the program, with weekly check-in calls from staff, nurse interns and promotores, and in-person visits as needed for participants who experienced more challenges with technology. The number of visits ranged from 1  to 10 visits.To help recruit older adult participants for the program, the county approached existing community-based organizations and agencies working with isolated adults, such as West Marin Senior Services, Meals on Wheels, and organizations that work closely with the Spanish-speaking older adult community. They also approached divisions in the county that already organized health programs for older adults, as they had the infrastructure and relationships in place to reach the older adult population.. One of the county staff members was a promotora who had full access to the network. These partners also helped identify isolated older adults.Recruitment was primarily done by sharing information about the program in partnership with West Marin Senior Services and the Telehealth Equity Project in the Marin County Division of Aging. In addition, information was shared by the promotores within their network to recruit Spanish-speaking older adultsThe program was specifically intended for isolated English- and Spanish-speaking older adults, as they were identified by county staff as underserved populations in the county. Participants expressed interest in the program by completing a screener form to assess their eligibility . This foA total of 30 participants were enrolled in the study. The first survey (phase 1) was completed by 28 participants, the second survey (between phases 2 and 3) was completed by 25 participants, the third survey (phase 4) was completed by 23 participants, the fourth survey (after phase 4) was completed by 18 participants, and 24 participants participated in the interview.One participant did not complete the demographic survey. Among the 29 participants who provided demographic data, their ages ranged from 60 to 89 years, with a mean age of 72 (SD 7.8) years, and 27 (93%) participants identified as female. Furthermore, 28% (8/29) identified as non-Hispanic White, 24% (7/29) as Mexican or Mexican American, 17% (5/29) as Central American, 10% (3/29) as South American, 10% (3/29) as other White, 3% (1/29) as other Hispanic or Latinx, 3% (1/29) as Black or African American, and 3% (1/29) as other or African. In addition, 48% (14/29) of participants indicated that English was their preferred language, and 45% (13/29) of participants reported Spanish as their preferred language. One participant indicated a preference for Vietnamese, and 1 participant did not indicate a preference. One Spanish-speaking participant did not have reading proficiency and completed the survey verbally with staff assistance.At the start of the program (phase 1), participants were asked to rate 2 statements related to digital literacy  taken from the Mental Health Literacy Scale . They weParticipants were also asked about their technology use on the screener form. The purpose of these questions was to assess the extent to which support was needed for the onboarding participants.Loneliness was measured at 3 time points: at the beginning of the program (phase 1), after digital literacy training (phase 2), and at the end of the program (phase 4). Loneliness was measured using the Three-Item Loneliness Scale , a shortSocial isolation was measured at three time points: at the beginning of the program (phase 1), after digital literacy training (phase 2), and at the end of the program (phase 4). The social interaction subscale of the Modified Duke Social Support Index ,32 was uAt the beginning of the program (phase 1), participants were asked a multiple-choice question on whether they had ever experienced or were currently experiencing any mental health challenges. Participants indicated that they had not experienced any challenges, had been diagnosed with a mental health condition, had not been diagnosed but were experiencing challenges, or could self-describe their experience. The survey explained that many different terms could be used to refer to these experiences, including mental health challenges, emotional distress, and psychological disorders.Before engaging with myStrength (before phase 3), participants were asked what they considered important aspects of mental health technologies and what they would like to do using technology. For each of these questions, they were instructed to “select all that apply” from a list of 9 and 8 options, respectively, or give an answer in their own words. These lists were based on common needs from the literature regarding mental health technology , as wellAfter engaging with myStrength (after phase 3), participants were asked about their experience using myStrength . Participants were asked to rate 15 statements  on a 5-point Likert scale ranging from strongly disagree (1) to strongly agree (5). The survey items were based on the Unified Theory of Acceptance and Use of Technology questionnaire , which iParticipants were contacted for a follow-up survey 8 months after the program was concluded (phase 5). The aim of the final survey was to learn more about participants’ continued use and experience of myStrength after the end of the program and staff support and asked questions similar to survey 3 (phase 4) applied to the period after the staff support had ended. Survey questions assessed people’s self-reported use of, and intention to use, myStrength, reasons for use and nonuse, challenges using myStrength, people’s perception of myStrength, facilitating conditions that would make it easier to use myStrength, and the overall impact of the program.Participants were invited for an interview at the end of their participation in the program (phase 4). The aim of the interviews was to gain a more in-depth understanding of participants’ experiences with the program. The research team developed an interview guide with discussions and sample questions. The topics covered in the interview guide included the following:Overall experience with the programHealth outcomes Experience with program staffOther resources and strategies used to support mental health and well-beingIf applicable, reasons for dropping out of the program earlyThe study was approved by the University of California, Irvine, Institutional Review Board (review number 20195406).Before each survey and interview, the participants were emailed a study information sheet that was reviewed and approved by the Institutional Review Board. For interviews and surveys over the phone, the sheet was reviewed at the start of the phone call with an opportunity for participants to ask questions. Participants were asked for their permission to audio-record the interviews at the start of the phone call; participants were still able to participate if they declined to be recorded. In addition, participants were asked to sign a participation agreement and a device user agreement by Marin County at the start of the program. The participation and device user agreements were reviewed and approved by the county’s legal department. All collected research data were stored securely and confidentially on a password-protected secure server. Participants received a US $10 gift card for each survey that they completed and a US $20 gift card for completing the interview. The survey took about 20 minutes to complete, and the interview took approximately 40 minutes.t tests.We analyzed the survey data using descriptive statistics, such as frequency counts. The analyses were performed using SAS statistical software  . ChangesAudio recordings of the interviews were transcribed verbatim in the language they were conducted in. The transcripts were analyzed by 2 members of the research team (one of whom was fluent in both English and Spanish) using thematic analysis. The themes were derived as follows: the interview protocol was developed to ascertain the overall experience with the program based on some collectively agreed upon outcomes among the research team, the county, and the broader collaboration that supported the Help@Hand Project; thus, these constructs  were used as a starting point for the thematic analysis, and interview questions were grouped based on construct . However, the interview also allowed for emergent ideas either not mapped directly to the outcomes of the project or high enough that important subthemes could emerge . Transcripts were first independently reviewed by each research member. The 2 members then met to discuss patterns and notes and iteratively developed a codebook that fit within the topic categories and captured the patterns identified in both English and Spanish transcripts. Spanish transcripts were then translated into English by an external translation vendor, and transcripts were coded by 1 researcher and discussed with the other research team member. The desktop version of the qualitative analysis platform Dedoose  was usedThe following section presents an overview of the survey and interview results. The results are organized by theme rather than by method . Themes were based on interview themes . Throughout this section, we report the number of people who provided a particular survey answer and the total number of people who responded to the survey question.Participants reported that they became aware of the program primarily through word of mouth and recommendations from others. The reasons for deciding to participate were as follows: (1) participants trusted the program based on recommendations from others who had participated in other programs offered by Marin County, (2) they wanted to contribute to a program that may help other people, and (3) they were interested in learning something new.Before engaging with myStrength, participants, overall, reported positive expectations about myStrength on the survey: they believed that it could be useful in their everyday life  and improve their mental health , and around half  of them believed that it might meet their mental health needs. However, during the interviews at the end of the program, it was revealed that not everyone knew beforehand what the app did or had a clear image of what “digital mental health” was: “I had no idea what it was about, so I had no expectations. I didn’t know about it. She told me about it, and so it’s hard to have expectations on something you don’t know about.”Although participants were excited about participating in the program, they had varying degrees of readiness to begin the program. For example, before starting the program, 48% (11/23) of participants reported that they did not have the required devices to use myStrength and participate in the digital literacy classes; these participants were given a tablet by Marin County to participate in the program. In addition, 21% (6/28) of participants needed support to access Wi-Fi, and 21% (6/28) of participants said they had never accessed the internet before. The technical support available through the program was experienced by participants as very useful, as expressed during the interviews: “The one-on-one technical [support]...[opened the door to new technology].”Participants indicated different levels of confidence in using technology in the surveys, and some not only needed support to access technology but also needed support to learn how to use technology. Almost half  of the participants reported that they were not confident about using technology at the start of the program. Furthermore, 76% (19/25) of participants did not know how to download a mobile app on their phone, and 62% (16/26) of them reported they were not comfortable using technology to support their well-being: “Todo era nuevo para mí. No sabía manejar estas cosas. Simplemente, el celular... ”However, participants were interested in learning new technology skills and improving their digital literacy. Specifically, participants were keen to learn how to use smartphones , the internet , computers , and how to use technology to support their mental health .Overall, 80% (24/30) of participants participated in at least one of 4 digital literacy classes. Even though participants had positive experiences with digital literacy training and gained digital skills, not everyone felt confident enough after the training to use web-based tools on their own for mental health. Participants reported feeling overwhelmed by the digital literacy training at times and the variety of options they were exposed to and noted the challenges of trying to teach and accommodate different needs in a group setting:Well, for a long time, I found it quite confusing. And somehow I guess I was feeling negative at first and thinking, Oh, dear...It’s sort of a new speak, you know? [laughs]...But later, I began to see some of the value in it [digital literacy training] and realized it wasn’t really the way I was feeling it would be...Well, it’s certainly helped to be able to be in touch with family. And so, that was lovely.To apply the skills they learned, participants not only needed human support at the beginning of the program but also continued support during the program. This ongoing support was experienced as helpful throughout the program:Oh, I thought it was great. The nurse—Yes, she was wonderful. She just went overboard to do the training as best she could...Oh, yes, they were very willing to do whatever I needed help with...She was continually there. And if I needed her to help me with something, she went out of her way to come over and help me at that particular time...[helped] in person...Yes, exactly. If I needed help, I need someone right here.A total of 5 participants reported that they did not have the support of family and friends to remind them or help out if they had any technical difficulties, and staff, therefore, checked in with participants one on one throughout the program. This personal support was experienced as very helpful: “The one-on-one contact was very good, excellent.”Social connection was an important aspect of the program for participants: 75% (21/28) of participants reported that they wanted to learn how to use technology to connect with friends and family. This was echoed during the interviews as follows:I think it’s WhatsApp or something like that?...Yeah, they, my friends have been telling me, so that way you don’t have to pay long distance phone calls...So that’s what I remember. So I think, you know, it’s good to have that.A total of 38% (11/29) of participants reported experiencing mental health challenges at the beginning of the program. During the interviews, participants described how the global coronavirus pandemic had affected them, including increased isolation, lack of social connection and activities , safety measures, and even some having experienced the virus. In addition, many participants reported feeling isolated or lonely. For some, health conditions impacted feelings of isolation:Sí, ahorita con la pandemia, sí me siento sola. Y como que sí me da miedo quedarme sola porque, primero, hay tantas cosas que me han pasado... 21=3.04; P=.006). A total of 17 participants scored high on loneliness at the beginning of the program; at the end of the program, only 9 participants scored high on loneliness.Overall, 78% (18/23) of participants indicated on the survey at the end of the program that they felt more connected to others because of the program. To compare any changes in loneliness and social isolation during the program, we considered 22 participants who answered the survey questions on loneliness and social isolation at baseline and at the end of the program. There was a statistically significant decrease in loneliness from the beginning of the program  to the end . A total of 7 participants were considered socially isolated at the beginning of the program and 4 were considered socially isolated at the end of the program.There was also a statistically significant decrease in social isolation from the beginning of the program  to the end .” Participants were satisfied with the program and hoped that more programs like this would be offered in the future.Participants also felt connected because they learned how to use technology to reach out to family and friends during the digital literacy classes: “I was pretty much at home most of the time and alone, so that was nice to be able to get into the technology, and reach out to more peopleMost  participants were satisfied with the digital literacy training, and participants reported various benefits from participating in the training; they  reported that they were more likely to use technology and felt more socially connected , and the percentage of participants who felt confident using technology increased from 31% (8/26) before training to 73% (19/26) after training.Overall, participants had a positive experience with myStrength: they recommended myStrength , found it useful , and found it easy to use . Benefits of myStrength were that it changed how they thought about mental health and helped them recognize symptoms:Maybe it made me more aware so that when I got in this bad mood a couple weeks ago, that...I didn’t know. [chuckles] One of the sections of myStrength was—I think it was Depression...And I’ve never had depression. So, I don’t know what it’s like. But I think it was last week, all of a sudden for about two days, I think I felt what might be depression...And so, I mean to go back and check that out.In addition, 70% (16/23) of participants indicated that they had used myStrength for 2 months at the end of the program, and 52% (12/23) had said they used it several times a week or daily. Furthermore, 57% (13/23) participants self-reported that they had stopped using myStrength at some point during the study. The main reasons participants mentioned for not using myStrength were that they had no time , health reasons such as chronic fatigue , technical issues , and they did not think they needed it : “No funciona mucho, falla el Internet. Por eso a veces no lo ocupo. ”Although  participants agreed that myStrength valued cultural differences, several Spanish-speaking participants mentioned that many aspects of the Spanish version were available only in English:Pero cuando yo entré a myStrength, todo es en inglés...porque, como te digo, si sería en español, entonces iría más rápido al video...Ahorita estoy como poquito a poquito...en la tablet, son en español las instrucciones. Pero cuando voy a myStrength, todo sale en inglés...lo que yo digo de ellos es que si estuviera en español el programa, la mayoría de los programas, o sea, en inglés y en español, sería maravilloso. When thinking about mental health technology beyond myStrength, important aspects for participants were that the data should be kept private , the app would be free , and the app would not have a negative effect on their device, such as draining their battery .Eight months after the program and staff support ended, 16 (53%) of the 30 participants completed the follow-up survey. During the program, participants overall had a positive experience with myStrength, reported high use, and reported that they were more likely to use technology to support their well-being; only 1 participant reported that they were still using myStrength at the time of the follow-up survey, even though participants still had free access to the platform. Furthermore, 75% (12/16) of the participants reported that they experienced technical issues that prevented them from continuing to use myStrength. Technical issues included forgetting how to use a tablet, smartphone, or computer and not knowing how to log back into the platform after being logged out. In addition, 19% (3/16) of the participants also reported significant life events, such as illness or injury, contributing to nonuse. Although 50% (8/16) participants reported that they intended to use myStrength again in the future, 44% (7/16) of participants also indicated that ongoing support would make it much easier to use myStrength.This paper sought to share what older adults consider important factors of support when engaging in a digital mental health program. We found that participants valued different types of support to stay engaged in the program: technical support, guided support to be reminded to engage with the program, and social support. Social connections were the most important aspect of the program for participants. In addition, even though participants were excited to participate in the program, they did not have a clear image of digital mental health and were mostly motivated to participate in the program because it was recommended by someone they trusted, such as a community partner with whom they were familiar or peers who had participated in similar programs offered by the county. Next, we discuss these findings in more detail and the implications they may have for the future design of digital mental health programs for older adults.Although the program showed high engagement and an improvement in loneliness and social connectedness, it is important to understand these results in the context of considerable hands-on staff support, rather than solely attributing these improvements to mental health technology. When contacted 8 months after the program—and thus the support—had ended, most respondents had stopped using myStrength.Nonuse of a digital health app does not always have to be negative ; for exaProviding continued support from a trained person was shown to be important in reminding older adults to use the technology and troubleshoot in case of technical issues during the program. Furthermore, it was important to provide ongoing personal support. In a separate publication , we discThese findings indicate that when integrating DMHIs into existing programs, typical measures of success, such as efficacy or changes in mental health outcomes, may not be sufficient to achieve a long-term beneficial impact. The real-world success of DMHIs in older adults may depend on the presence of intensive and ongoing support. In addition, this support may have benefits for both sustaining engagement and alleviating social isolation.To ensure that this resource-rich support is sustainable, it is important to assess support needs and build wrap-around services around DMHIs for older adult populations. For example, there may be an opportunity to build and maintain close partnerships with local agencies that already work closely with older adult communities, such as home nursing services.The facilitating factor of personal guidance is widely known in the digital mental health literature, and guided DMHIs typically have higher engagement than unguided DMHIs. For example, 2 studies that evaluated a self-guided DMHI found that participants regularly forgot to use the intervention ,42. In cHowever, human support is resource intensive and may not always be feasible. Therefore, different types of support may be used to address the different types of challenges people experience. For example, for older adults with higher digital literacy, email and text reminders could remind people of the technology at regular intervals; automated reminders have been found to facilitate engagement with the general population and have been experienced positively . In addiBoth feelings of loneliness and feelings of being socially isolated significantly decreased from before to after the program. This decrease may indicate that participation in the program reduced social isolation in some participants. Participants indicated that they liked being able to connect with people such as other participants, digital training teachers, county staff, nurses, and promotores during the program, and some even took part in the program because they wanted to help others in improving the program for future deployment among older adults.This is in line with prior research highlighting that offering social opportunities is an important facilitator of DMHI use among older adults . For exaGiven the great value placed on a social component, previous work has recommended that DMHIs should incorporate a feature or ability to connect with others . HoweverFuture digital mental health programs could host sessions where participants may be able to meet others taking part in the program and connect with others and may focus their digital literacy initiatives on teaching skills on how to connect and communicate with loved ones.Our study also revealed the importance of collaborating with community partners who have a close relationship with the community to get a digital mental health program that starts with older adults. Participants primarily indicated that they became aware through word of mouth and decided to participate because they trusted the person who recommended it to them. Efforts to recruit and engage participants without the assistance of partners were largely ineffective. For instance, promotores helped recruit almost all Spanish-speaking older adults, whereas other partners were instrumental in recruiting English-speaking older adults. This finding is consistent with prior work that highlighted that community-based partnerships can be crucial for populations who may face barriers to accessing care . For exaPromotores may have also played an important role in engaging Spanish-speaking participants throughout the program, as they have the cultural sensitivity and experience of the community. There is a need for mental health programs to be culturally appropriate , which iTechnology adoption is often influenced by the beliefs and recommendations of people close to the user, such as family, friends, or health providers . This maCommunity partners can further play an important role in sustained engagement, as indicated by the finding that a large number of the participants discontinued myStrength after the program and support had ended. There may be opportunities in the future to make this type of support and community engagement an essential part of future staff job roles. In addition, prior work with a vulnerable youth population recommended integrating mental health services into services that youth are already receiving, such as schools . There mOne limitation of our study pertains to missing data as not all participants completed each survey or survey question. Engaging older adults in research activities is a known challenge , and parIn addition, most of the participants were female. Men are often underrepresented in older adult health research programs, and future efforts may need to tailor recruitment strategies to increase male participation, for example, by featuring men in promotional material .The digital mental health program studied in this paper was particularly focused on isolated  older adults in specific communities with little access to mental health resources; therefore, caution must be exercised when generalizing the results. That said, we expect our findings to generalize to isolated older adults with low digital literacy.The program offered staff support and myStrength to all participants and there was no comparison group. We did not know if the participants would have experienced similar benefits in the absence of this intervention.Finally, this study was conducted during the COVID-19 pandemic, which may have exacerbated the level of isolation. It also added the challenge of providing support and digital literacy training remotely when participants were already struggling to get on the web in the first place. Future programs may offer in-person classes and provide more in-person support.DMHIs have the potential to improve mental health among older adults; however, factors such as low digital literacy and lack of support can form a barrier to starting and sustaining engagement. We found that ongoing support  not only may be important from an implementation perspective to retain people in a digital health intervention but also may be more crucial in improving people’s experience and supporting their mental health than the technology itself. Having a social component and being able to participate in a program with other people may address loneliness and motivate people to enroll in a program that can help them stay connected with family and friends and improve their technical skills.Future research should keep this in mind and provide a social support component, whenever possible. For example, web-based Zoom sessions can be held to connect with others using the platform, or a DMHI can provide the option to provide individual support and guidance from staff or coaches.