The post-program quitters were at lower T2D risk  and were more likely to regress to normoglycemia  compared with the controls in the time-varying analysis (n = 532). We demonstrated that the FIT2 program was negatively associated with long-term T2D risk and positively associated with the probability of regression to normoglycemia compared with usual care. To prevent T2D development, we recommend simultaneously promoting smoking abstinence and lifestyle coaching for PCWG restriction.Post-cessation weight gain (PCWG) facilitates short-term type 2 diabetes (T2D) risk in prediabetic smokers in the absence of complementary measures. In this shared decision-making-based non-randomized controlled trial, prediabetic smokers joined the Fight Tobacco and Stay Fit (FIT2) program or received usual care. The 16-week FIT2 program combined smoking cessation therapy with individualized coaching in diet and physical activity strategies for PCWG restriction . During a mean follow-up period of 1316 days, 217 participants (36.8%) developed T2D, and 68 (11.5%) regressed to normoglycemia. In the intention-to-treat analysis ( Type 2 diabetes (T2D) has been linked to cardiovascular comorbidities, malignancy, and high mortality ,2,3. To We recruited prediabetic smokers receiving medical care at two hospitals in northern and central Taiwan, where a systematic identification system was applied to classify the tobacco addiction status of every patient in the outpatient clinics . This stWe estimated to recruit at least 596 prediabetic smokers, 33% (199) of whom decided to join the FIT2 program, to reach 90% power and a two-sided 95% CI for the detection of a 50% risk reduction, assuming 30% to be the risk of incident T2D during follow-up in the usual care group .1c levels, lipids, alanine aminotransferase, and creatinine levels were determined. The estimated glomerular filtration rate was calculated using the four-variable version of the Modification of Diet in Renal Disease Study equation for Chinese Patients [Body height and weight were measured using a standard stadiometer, and body mass index was calculated. Blood pressure was measured with an electronic sphygmomanometer while the patient was seated and after resting for at least five minutes. Hypertension was defined as a history of hypertension according to the medical records or repeated systolic blood pressure ≥ 140 mmHg or a diastolic blood pressure ≥ 90 mmHg. Each participant underwent laboratory testing after fasting for at least ten hours. Plasma glucose and HbAPatients . DyslipiThe assignment of this trial was based on shared decision making. At baseline (from August 2013 to January 2017), all eligible prediabetic smokers were asked if they would like to join the FIT2 program or just receive usual care. From enrollment, the smoking status, breath carbon monoxide levels, anthropometric indices, and blood tests were recorded every six months. The intention-to-treat analysis was performed among all participants joining the FIT2 program and receiving usual care. To reduce the immortal time bias in the survival analyses, the time-varying analysis was adopted among the participants with documented abstinence status at six months .www.hpa.gov.tw, accessed 19 March 2021). It was initiated at 0.5 mg once daily for the first three days, increased to 1 mg once daily from day 4 to day 7, and then increased to 1 mg varenicline twice daily from day 8 to the end of the 16 weeks. After 16 weeks, varenicline was no longer administered. During the therapy course, the physicians could adjust the varenicline dosages according to tolerability. Participants joining the FIT2 program were encouraged to either set their quit day 8 days after starting the varenicline; or freely choose their quit day for any time between days 8 and 35. Nicotine replacement therapies were prohibited during the study period.The FIT2 program was a 16-week program that combined pharmacotherapy for smoking cessation with individualized counseling focused on both smoking cessation and weight-control techniques. Behavioral counseling in tobacco cessation techniques were delivered to every FIT2 participant involving one session per week . The phaIn addition to varenicline prescription and smoking cessation counseling covered in conventional smoking cessation services, the FIT2 program also offered individualized weekly coaching in diet and physical activity to restrict PCWG. The counseling sessions allowed opportunities to identify obstacles to lifestyle change and discuss approaches with a professional panel of dietitians and certified personal trainers. The dietitians instructed the FIT2 participants to choose minimally processed, whole grains, vegetables, whole fruits, nuts, healthful sources of protein, and plant oils; rather than sugared beverages, refined grains, potatoes, red and processed meats, and other highly processed foods. The FIT2 participants were also encouraged to do at least 150 min of moderate-intensity  aerobic physical activity throughout the week. Emphasis was placed on checking the weekly diary for body weight, food, and physical activity through protected cellphone messages between the participants and the assigned panel professionals. For those who gained weight, more intensive ways of calorie restriction and physical activity  were instructed. Post-program abstinence status was recorded weekly from 16 weeks to 6 months by self-reported 7-day point prevalence of abstinence and a breath carbon monoxide level of less than 6 ppm. Post-program quitters were those who successfully quit at 16 weeks after the FIT2 program and had maintained their non-smoking status until six months (achieving post-program abstinence). For the time-varying analysis, participants who failed to stop smoking after the FIT2 program were reassigned to the control group, while those who relapsed after six months were still classified as post-program quitters.Usual care was provided for prediabetic smokers who decided not to join the FIT2 program. Usual care comprised an interpretation of laboratory results and encouragement to quit smoking and initiate a therapeutic lifestyle change for T2D prevention at each visit. In the time-varying analysis, the control group contained participants joining the FIT2 program who failed to achieve post-program abstinence and all participants receiving usual care, including those who quit smoking on their own. Given that the post-program abstinence was established at six months after enrollment, for the time-varying survival analysis, the follow-up time experienced by the controls also included the first six months experienced by all post-program quitters to reduce immortal time bias .1c levels ≥ 48 mmol/mol (6.5%) [1c changes at 6 and 12 months and. Participants who regressed to normoglycemia should meet all the following conditions for more than six months and maintain such status until end of the study: (1) plasma glucose levels < 5.6 mmol/L (100 mg/dL) in the fasting state; (2) plasma glucose levels < 7.8 mmol/L (140 mg/dL) two hours after a 75 g oral glucose load; or (3) HbA1c levels < 39 mmol/mol (5.7%), in the absence of antidiabetic drugs. This study expects to collect other preregistered secondary outcomes until December 2022.The primary outcome was new-onset T2D, defined as random plasma glucose ≥ 11.1 mmol/L (200 mg/dL) with hyperglycemic symptoms or repeated results in at least one of the following parameters: (1) plasma glucose levels ≥ 7.0 mmol/L (126 mg/dL) in the fasting state; (2) plasma glucose levels ≥ 11.1 mmol/L (200 mg/dL) two hours after a 75 g oral glucose load; (3) HbAl (6.5%) ; or currWe collected the parameters of the physical activity level, depression scale score, and sleep quality at baseline. The physical activity level within the last week was calculated as the average daily energy expenditure in Kcal/day . Being p2 test or Fisher’s exact test. Continuous variables were compared using the two-sample Student’s t-test. Statistical significance levels were determined by two-tailed tests (p < 0.05). Linear regression analyses were used to estimate the association of clinical variables with HbA1c changes at 6 and 12 months compared with the baseline. We performed Cox proportional hazards regression to explore the association of the FIT2 program with progression to T2D and regression to normoglycemia in the intention-to-treat analysis. Kaplan–Meier failure plots of T2D risks of the two groups were drawn. To reduce immortal time bias, we also adopted time-varying analyses in Cox proportional hazards regression and all participants were considered unexposed from baseline until six months when the post-program abstinence was established [For the descriptive analyses, values are presented as either numbers (percentages) or mean ± standard deviations. For the univariate analyses, categorical data were compared by the χablished . Modifieablished . We assun = 279), 207 (74.2%) achieved post-program abstinence at six months . At the end of this study, there were no serious adverse events reported.A total of 589 eligible prediabetic smokers were enrolled. Among the participants participating in the FIT2 program , weight gain at six months was positively associated with HbA1c changes at 6 and 12 months . The FIT2 program was inversely related to HbA1c changes at 12 months  while not related to HbA1c changes at six months compared with usual care in the multiple linear regression analysis , weight gain at six months was positively associated with HbA1c changes at 6 and 12 months , while post-program abstinence was only inversely related to HbA1c changes at 12 months  compared with the controls in the multiple linear regression analysis  , 217 participants (36.8%) developed new-onset T2D, and 68 (11.5%) regressed to normoglycemia. In the intention-to-treat analysis, participants joining the FIT2 program group had a lower risk of incident T2D compared with the participants receiving usual care shown in the Kaplan–Meier failure plots (= 0.008) . The FIT= 0.008) . Weight p = 0.008.Kaplan–Meier failure plots show that participants joining the FIT2 program had a lower risk of incident type 2 diabetes (red line) compared with participants receiving usual care (blue line). The log-rank test was significant with p = 0.016)  . The pos= 0.016) . Weight p = 0.016.Modified Kaplan–Meier failure plots show that the post-program quitters at six months had a lower risk of incident type 2 diabetes (red line) compared with the controls (blue line). The log-rank test was significant with n = 207), an increased PCWG degree at six months was associated with an increased T2D risk (p for trend = 0.010)  . Post-prThis non-randomized controlled trial pioneered a shared decision-making based program combining smoking cessation therapy with individualized behavior coaching in diet and physical activity for PCWG restriction in prediabetic smokers. The intention-to-treat analysis demonstrated that the FIT2 program was negatively associated with long-term T2D risk and positively associated with the probability of regression to normoglycemia compared with usual care. The time-varying analysis consistently revealed that the post-program quitters had a lower long-term T2D risk and were more likely to regress to normoglycemia compared with the controls. In addition, weight gain at six months was positively associated with T2D occurrence and negatively associated with regression to normoglycemia in the both intention-to-treat and time-varying analyses. Among the post-program quitters, an increased PCWG degree at six months was associated with an increased T2D risk. Our integrated FIT2 program emphasizing both abstaining from smoking and restricting PCWG through individualized behavior coaching in diet and physical activity would reduce long-term T2D risk and increased the likelihood of regression to normoglycemia.Although data from a nationally representative study showed that the mortality risk is smaller in overweight or obese ex-smokers than normal-weight smokers, primary care clinicians have been faced with challenges in fighting smoking and obesity together ,21. PCWG1c elevation and new-onset T2D compared with participants receiving usual care. Furthermore, PCWG ≥ 2 kg at six months was associated with a much higher T2D risk than PCWG < 1 kg at six months in post-program quitters. Taken together, this study provides insights into conducting future multi-center clinical trials to test this integrated program on long-term cardiometabolic outcomes.The present study chose varenicline as the cessation medication, as it has been shown to excel in facilitating smoking cessation without evidence of an increased risk of neuropsychiatric or cardiovascular adverse events ,27. HoweSome concerns and limitations of this study should be addressed. Immortal time bias is a challenge often overlooked in studies other than randomized controlled trials. It arises when the start of follow-up and treatment do not coincide . In the Our integrated FIT2 program with the concerted concept of quitting smoking and restricting PCWG through individualized behavior coaching in diet and physical activity is negatively associated with the risk of new-onset T2D compared with usual care. Additionally, prediabetic smokers who successfully quit and maintained their non-smoking status at six months may achieve better long-term glycemic outcomes than the controls. More long-term health outcomes should be explored using shared decision-making-based study designs. To cut challenges in long-term T2D risk and increase the likelihood of regression to normoglycemia, we propose promoting organized tobacco cessation and weight control programs together for prediabetic smokers.


To assess the preparedness of Iranian hospitals against earthquake.In this systematic review, a query was carried out on PubMed, Scopus, Web of Sciences, Sid, Irandoc, Google scholar, and Magiran databases for articles published between 2000-2019. Statistical analysis was performed using Chi-square and I2 tests at a confidence interval of 95%. Finally, out of 7458 studies, 10 related articles were analyzed. As evidenced by the obtained results, the highest readiness was obtained at 0.709 (95% CI: 0.49-0.88) in “disaster plan”, while the lowest readiness was reported at 0.455 (95% CI: 0.23-0.68) in “structural safety”. The overall earthquake preparedness of these hospitals was calculated at 0.47 (95% CI: 0.37-0.56). The results of the present study pointed to the moderate level of earthquake preparedness in Iranian hospitals. This finding highlights the necessity of a training plan and implementation of a national program in all hospitals of the country to increase earthquake preparedness. Large earthquakes are one of the most destructive natural disasters, often resulting in massive casualties and high mortality . As repoPreparedness is defined as activities set up to build a mechanism for rapid responses to limit the risks and effects of disasters  and is rConsidering the high incidence of earthquakes almost everywhere in the world, it is important to address the issue of earthquakes and prepare the health care system to provide care in critical situations. The present study aimed to systematically assess the preparedness of Iranian hospitals against earthquakes to help integrate all the key components (10 variables) of preparedness and give the reader a general view of these components and ways to upgrade them. It is hoped that the findings of the current study will assist the managers and authorities of hospitals and medical centers in increasing earthquake preparedness.In the present systematic review and meta-analysis, the relevant studies were selected, and the results were reported based on the PRISMA guidelines .Precise Definition of the Study Question and Identification of Its Components At this stage, a detailed definition was provided for the following: participants in the baseline study, type of intervention, comparison group, outcome, type of baseline studies, as well as the time and place of review. The appropriate keywords were selected according to MESH  and EMTREE. The queries were carried out with high sensitivity to ensure the identification of all preliminary studies. The search strategy for each of the databases is explained in the following. The current research was performed using some keywords, including “hospitals”, “earthquake”,” disaster”, “preparedness” and “Iran”. The Persian equivalent of these terms and all possible combinations in the Persian national databases were also searched. Search for Basic Studies The query was carried out on both International databases  and Persian national databases  for published articles on the preparedness of Iranian hospitals. Duplicates were removed after searching and logging the findings into the Mendeley software to find relevant articles. Inclusion and Exclusion Criteria for Study and Selection Process Two authors independently performed all the stages of study selection, separation by title, the abstract or full text of articles, and data extraction. Any disagreement between the two authors was resolved by asking the opinion of the third author. The inclusion criteria entailed all studies published as an original article. On the other hand, the exclusion criteria were as follows: 1) studies without original research , 2) poor-quality studies that did not report a good index of association between hospital preparedness and earthquake, 3) insufficient data, and 4) studies published in Persian without an English abstract. Primarily, 7433 articles were retrieved, and 24 papers were excluded due to duplication. Out of the remaining 7,401 articles, 7,309 papers were excluded for the following reasons: irrelevancy, non-original study, pilot study, and repetition of studies. Those articles indexed in several databases were regarded as one. Out of the remaining 90 articles, 69 papers were removed due to the absence of study design specification and insufficient information about component data. The other 21 articles were removed since they had qualitative synthesis. In the event of incomplete or ambiguous data, the author(s) were requested for more detailed information via email. Initially, the authors read the titles of articles and selected the relevant ones. At this stage, they were sensitive and reviewed the full text of the articles related to the STROBE and MOOSE or NEWCASTLE - OTTAWA (NOS) tools in terms of quality. Data Extraction Finally, 10 related articles were reviewed. The necessary data were accurately acquired via a data extraction form on the basis of title, year of publication, type of study, region (province), and sample size; subsequently, they were entered the pre-prepared forms. Synthesizing and Integration of Data and Meta-analysis p-value less than 0.05 was considered statistically significant. Two researchers independently reviewed the collected articles based on inclusion and exclusion criteria, and in case of any disagreement, another author examined them. Finally, the collected information was entered into the software. The prevalence of each component was reported with a 95% confidence interval. Weighted averages were calculated for each study based on sample size and variance. Two models of static effect (DerSimonian and Laird method) and random effect (Mantel and Haenszel method) were used according to the results of the heterogeneity of the studies. Statistical analysis was performed using the Chi-square test and I2 at a 95% confidence level. In addition to statistical tests, cumulative and Galbrais graphs were also used. In the accumulative graph, the amount of heterogeneity was visualized to find outlier data. Begg’s funnel plot and Egger’s linear regression test were used for assessing publication bias. All the statistical analyses were performed using Stats Direct software using a two-tailed test. A In the present study, out of 7458 studies,10 articles were reviewed . The res2 pointed to the superiority of the Random Model over the Fixed one. Begg funnel plot and Egger’s linear regression test was used for the evaluation of publication bias. The results of this test (p>0.05) and graphs indicated no publication bias among studies (The Forest Chart of these three components is based on Random and Fixed models in  studies . FurtherIn the majority of studies (n=5), data about the preparedness of hospitals against earthquakes were collected via the earthquake preparedness assessment checklist in eight planning dimensions. This checklist encompasses the following issues: 1) safety of equipment and hazardous materials, 2) mitigation of construction, 3) hospital evacuation and field treatment, 4) necessary medical and nonmedical equipment and consumable goods, 5) hospital environmental health proceedings, 6) hospital curriculum, 7) disaster plan management, and 8) support planning of critical services. In this 123-item questionnaire, according to the earned scores, the situation of earthquake preparedness in each planning dimensions was categorized into three groups: A) weak (0-<50%), B) moderate (50%-<75%), and C) good (75%-100%). In other studies, researcher-made questionnaires were used.et al., [et al., [The present study aimed to evaluate earthquake preparedness in Iranian hospitals. In this systematic review, we analyzed the overall preparedness of 134 Iranian hospitals for 10 components against earthquakes. The results pointed out that the earthquake preparedness in Iranian hospitals is at a moderate level (pooled proportion=0.470). In a systematic review performed by Kazemzadeh et al.,  the prep[et al.,  stated tet al., [In the present study, the highest level of preparedness is related to the component of Disaster plan management (pooled proportion=0.709). This program identifies policies, protocols, and ways in which hospitals respond to the crisis. In the same context, in their study, Rabeian et al.,  reportedet al., [In addition, based on the results of the current study, the lowest level of earthquake preparedness was related to the dimension of structural safety (pooled proportion=0.455). Hosseini Shokouh et al.,  explaineet al., [Concerning the safety of equipment and hazardous materials, the findings of the present study pointed to the moderate level of preparedness in Iranian hospitals in this dimension (pooled proportion=0.528). Along the same lines, in their study, Hekmatkhah et al.,  attributet al., [In terms of hospital evacuation and field treatment, Iranian hospitals have been reported to be at a moderate level. In their study, Afkar et al.,  rated hoet al., [Regarding support for critical services, as displayed in et al.,  attributNon-structural vulnerabilities and human resources are other important aspects of earthquake preparedness in Iranian hospitals which have been scarcely studied (two articles). Non-structural elements generally include mechanical and telecommunication, while electrical components encompass water supply, heating and cooling systems, fire detection, and containment systems. In this regard, Iranian hospitals were moderately prepared (pooled proportion=0.588). In terms of human resources, the preparedness of Iranian hospitals is at a moderate level. Insufficient attention to the needs of staff and their families in times of earthquake and lack of proper staff training can be among the weaknesses of hospitals in this regard. From this perspective, the notable strengths of hospitals included completing and updating the staff contact list, assessing and monitoring staff presence and absence, and prioritizing personnel need .et al., [et al., [Furthermore, studied hospitals were also at a moderate level regarding the necessary medical and nonmedical equipment, environmental hospital health planning, and hospital curriculum. In the field of essential supplies, Afkar et al.,  identifi[et al.,  stated tIn the dimension of planning environmental health measures, the studied hospitals demonstrated a low level of preparedness. This unsatisfactory level can be attributed to the failure to plan for food control, disinfect different parts of the hospital, construct temporary facilities, plan for proper collection of wastewater, and plan for providing water and hygienic materials in different parts after the earthquake. In addition, there is an absence of planning for chemical and biological quality control of hospital water, as well as the lack of a potable water treatment system for disinfection of water after the earthquake , 22.The assessment of earthquake preparedness in Iranian hospitals demonstrated that although many components have been studied, some components, such as surge capacity, disaster recovery, triage, communication, safety, and performance of earthquake maneuvers have been scarcely evaluated and require more studies in these areas. There is also a need to focus more on hospital coordination and connection with other relief groups, such as emergency medical services, the Red Crescent, and the fire department. As a final note, one of the main missed elements in earthquake preparedness is the failure to recruit, train, and deploy volunteers during the earthquake which requires careful planning by the authorities.As evidenced by the results of the present study, the earthquake preparedness in Iranian hospitals was reported to be at a moderate level. This finding highlights the necessity of training and implementation of a national program in all hospitals of the country to increase earthquake preparedness. The mere development of earthquake prevention programs is not effective, rather it should be implemented and practiced regularly and continuously. Furthermore, it was suggested to increase structural and nonstructural safety, provide the necessary medical equipment, establish the appropriate communication infrastructure, continue staff training courses, and promote coordination and collaboration with other relief teams to increase hospital readiness.


The traditional informed consent (IC) process rarely emphasizes research participants’ comprehension of medical information, leaving them vulnerable to unknown risks and consequences associated with procedures or studies.Virtual Multimedia Interactive Informed Consent (VIC) for advancing the IC process and compared the results with traditional paper-based methods of IC.This paper explores how we evaluated the feasibility of a digital health tool called Using digital health and web-based coaching, we developed the VIC tool that uses multimedia and other digital features to improve the current IC process. The tool was developed on the basis of the user-centered design process and Mayer’s cognitive theory of multimedia learning. This study is a randomized controlled trial that compares the feasibility of VIC with standard paper consent to understand the impact of interactive digital consent. Participants were recruited from the Winchester Chest Clinic at Yale New Haven Hospital in New Haven, Connecticut, and healthy individuals were recruited from the community using fliers. In this coordinator-assisted trial, participants were randomized to complete the IC process using VIC on the iPad or with traditional paper consent. The study was conducted at the Winchester Chest Clinic, and the outcomes were self-assessed through coordinator-administered questionnaires.A total of 50 participants were recruited in the study . The participants in both groups had high comprehension. VIC participants reported higher satisfaction, higher perceived ease of use, higher ability to complete the consent independently, and shorter perceived time to complete the consent process.The use of dynamic, interactive audiovisual elements in VIC may improve participants’ satisfaction and facilitate the IC process. We believe that using VIC in an ongoing, real-world study rather than a hypothetical study improved the reliability of our findings, which demonstrates VIC’s potential to improve research participants’ comprehension and the overall process of IC.ClinicalTrials.gov NCT02537886; https://clinicaltrials.gov/ct2/show/NCT02537886 Informed consent (IC) is essential for upholding ethical conduct in research and medical treatment. The goal of the IC process is to provide research participants with sufficient information about the proposed research so that the participants can make an autonomous decision regarding their health and well-being -4.Despite the implications and importance of the IC process, the Joint Commission for Transforming Healthcare has reported that an estimated 60% to 70% of individuals do not read or understand the information contained in the consent form, and 44% of the participants signing the IC documents do not understand the nature of the proposed procedure . AlthougSeveral studies have compared the efficacy of digital innovations in IC versus traditional paper consent. A systematic review conducted in 2019 to compare the 2 methods in published studies between 2012 and 2018 concluded that 67% of the included studies reported a positive effect on at least one of the studied outcomes. The efficacy of innovative interventions appeared high for interactive multimedia, with a positive effect on participants’ comprehension and no negative effect on satisfaction or participation [Another study that compared different methods of providing the same information in different formats to participants found that presenting plain text to participants with only audio narration proved to be the least effective when compared with other methods such as animated video or comics .Several digital platforms have been developed to improve the consent process, and although their findings have been shown to be more effective than paper IC, either in a real study or a hypothetical study, some of these systems lacked 1 or more of the features that make Virtual Multimedia Interactive Informed Consent (VIC) unique -16. ThesVIC [Existing research suggests that the use of digital health interventions, such as virtual coaching and mobile apps, along with interactive audio and visual elements in a participant-centered IC, can increase the participant’s interest and retention ,18-21. WVIC , and comThe initial concept of the VIC tool was developed on the basis of previous work, literature findings in IC research, participant input, and subject matter expert interviews . The theOur tool presents IC materials to participants with the option of going back and forth through each section as well as the ability to click on links within the sections, if desired, to drill down for more information. In addition, the VIC tool has an option to assess participant comprehension with automated quizzes, which can help emphasize the information presented. VIC provides many features and functions, including internet access to the consent, retrievable electronic records of IC, electronic signatures, and the potential for seamless integration with the electronic health record . MoreoveAlthough more than half of the feasibility studies in the existing literature use hypothetical scenarios to test enhanced IC, we believe that the use of an actual research study improves the validity, accuracy, and reliability of the results ,27. ThisThis study was a randomized controlled trial to test the feasibility of VIC in an ongoing, real-world biorepository research study titled “Yale Center for Asthma and Airway Disease Mechanisms and Mediators of Chronic Lung Disease Study” (GenEx 2.0). GenEx 2.0, the parent study for VIC, evaluated the pathophysiology and heterogeneity of airway disease in participants using several procedures, including a coordinator-administered, self-assessed questionnaire, lung function testing, blood draw, and hypertonic saline–induced sputum induction at the Winchester Chest Clinic (WCC). A detailed IC document was required so that the participants understood not only the risks and benefits of the procedures but also the ramifications of contributing biological and genetic samples to the GenEx 2.0 study.Using the parent study’s existing infrastructure and participants, the VIC trial recruited individuals and randomly assigned them to receive either the existing GenEx 2.0 paper consent document (control arm) or consent on the tablet through the VIC tool (intervention arm). Of the eligible participants, the allocation ratio for each group was 1:1. For both arms, the study coordinator responded to any questions from the participant regarding the IC process. After the consenting process, participants from both groups started the GenEx 2.0, which took an average of 2 hours to complete. The outcomes for the VIC trial were self-assessed using coordinator-administrated questionnaires at the WCC. Immediately following the GenEx 2.0, we surveyed the participants regarding their comprehension and satisfaction of the study to evaluate the effects of the VIC intervention.The GenEx 2.0 study recruited participants with lung disease from the WCC at Yale New Haven Hospital in New Haven (CT) and healthy individuals from the community using fliers. We approached the participants who were considered for the parent study and asked if they were interested in participating in the VIC trial in addition to their participation in GenEx 2.0. For VIC participants, the trial included web-based components in addition to face-to-face components for the survey, or was conducted primarily face-to-face depending on the arm to which the participant was assigned.The participants were eligible for both GenEx 2.0 and the VIC trial if they (1) spoke English, (2) were older than 21 years, (3) provided an email address, and (4) were willing to use an iPad. Computer literacy was not required for eligibility. The participants were excluded from the GenEx 2.0 study for (1) having a smoking history of more than 10 packs a year, (2) being active smokers within the past year, or (3) having other chronic lung disease or asthma variants. In addition, GenEx 2.0 participants were excluded from participation in the VIC trial if they were (1) not able to safely undergo the studies required for participation, (2) were unable to read or understand English, (3) refused to participate, or (4) had participated in the GenEx 2.0 trial in the past.No participants withdrew during the study period from the VIC trial.Eligible and consenting participants were randomized to receive IC through standard paper consent or digital consent through the VIC tool. Because of the small sample size, we used the method of minimization  first toIndividuals who were interested in and eligible for the VIC study consented to participate in the study and completed an initial demographic survey. The participants were then randomized and scheduled for their GenEx 2.0 study visit, which usually occurred within 1 to 2 weeks of enrollment in the VIC study. If assigned to paper consent for the GenEx 2.0 trial, the individual received a copy of the paper consent in the mail to review before the study visit. Because VIC is a web-based application that provides access to the consent before, during, and after the study visit, the individuals who were assigned to the VIC received an email with a link to the remote web-based version of the VIC consent session. This remote “review” link allowed VIC participants to preview what would be seen on the iPad during the study visit, but did not give access to the final signature needed to complete the consent process. Only VIC-assigned participants had access to the application itself during this time. If VIC participants had questions or concerns, they could leave comments in the “Notes” section of each slide, which would be addressed during the study visit. Remote sessions were separately tracked in the system to differentiate between the remote and study visit sessions. 10 of 25 participants accessed the remote session before their study visit.During the GenEx 2.0 study visit, individuals completed the consent process according to the arm to which they were assigned. Individuals assigned to the control arm  completed the consent with a study coordinator who explained each section of the consent by reading it out loud and answering any questions during the process. The individual then signed the consent form if he or she wanted to participate in GenEx 2.0. Participants were not blinded, as it was not possible for the purposes of this study, and the study coordinator was aware of which group each participant belonged to as they oversaw both groups separately. The participants were also aware that the VIC was the comparator of interest for this trial.Individuals assigned to the intervention arm (VIC tablet-based consent) completed the visit with a study coordinator who provided the individual with an iPad along with disposable headphones so that the individual could listen to the audio instructions comfortably. The individual would then go through the consent process alone and sign the consent on the iPad at the end if they were interested in participating in the GenEx 2.0 study. The format of the VIC process on the iPad allowed for the presentation of content to be displayed 1 section at a time with a “Continue” and “Back” button that participants could press to move forward or backward. For text-only sections, no more than approximately 90 words at a time were displayed, with an average of 76.6 words per slide overall. Some sections were transformed into interactive multimedia components, including animated videos that explained study procedures  as well as videos specific to privacy and withdrawal information. In addition, these multimedia components also allowed interactivity with a simple menu that could pause, play, rewind, mute, and enable closed captioning if needed. Some sections were followed by interactive quizzes that emphasized key information to enhance participant comprehension, which would give automatic feedback to the participant on the answer and allow them to go back to the key section and revisit the material, or move forward, regardless of their answer. This method did not inhibit the participant, but rather encouraged the active retention of the material. The participants were able to either continue through each section  in order, or access a menu that allowed the participant to view any section of the consent in the order they wished. The VIC required that the participant views each section of the study’s IC before allowing access to the signature portion of the tool. Once the IC was signed, the tool then ended and emailed a copy of the consent with the signature to the participant’s enrolled email. Once the consents were collected, participants from both groups started the GenEx 2.0 study procedures.After completing the GenEx 2.0 study session, the participants who enrolled in the VIC study from each arm were provided with a paper exit survey that asked questions about the feasibility of their respective IC process. After completing the survey, an incentive of US $60 was provided to thank them for their time in completing both the VIC trial and the GenEx 2.0 parent study. This incentive was the same for those who did not enroll in the VIC trial and completed only the GenEx 2.0 parent study.The exit survey provided at the end of the GenEx 2.0 study procedures was designed to assess comprehension and satisfaction of the participants from each IC method . It inclThe remaining 12 survey questions were administered using a 7-point Likert scale. Of these 12 questions, 4 questions assessed the participants’ self-assessed understanding of the IC general concepts, with scores ranging from 1  to 7 (I understood this very well); 3 questions assessed satisfaction with the IC process, with scores ranging from 1 (Very dissatisfied) to 7 (Very satisfied); 1 question assessed the perceived length of the IC process, with scores ranging from 1 (Very long) to 7 (Very short); 1 question assessed the perceived difficulty in completing the IC process, with scores ranging from 1 (Very difficult) to 7 (Very easy); 2 questions assessed the likelihood of participating in future clinical trials, with scores ranging from 1 (Very unlikely) to 7 (Very likely); and 1 question assessed the importance of the IC process in the participant’s decision to ultimately participate in the trial, with scores ranging from 1  to 7 (Very important). Participants were also asked to provide an estimate of their perceived time (in minutes) to complete the IC process.We measured six outcomes to assess the feasibility of VIC compared with the standard paper consent in a real-world clinical research study, which were recorded through a paper survey after completing the IC process. The study outcomes included each participant’s (1) comprehension of the GenEx 2.0 study IC content, measured through a 13-question comprehension quiz; (2) satisfaction with the IC process, ranked on a 7-point Likert scale; (3) perceived time required to complete the IC process; (4) perceived ease of the IC process; (5) perceived likelihood of participating in future clinical trials; and (6) perception of the importance of IC in the decision to participate in clinical trials.Data were analyzed according to intention-to-treat analysis . Baseline data are reported as mean and SD for continuous data and as counts and percentages of total for discrete data. The 13 comprehension questions were summarized as mean (SD) and proportions (SE). Differences in the mean values of correct answers between VIC and paper consent were presented with 95% CIs. The proportion of correct answers for each of the 13 individual questions was analyzed using risk ratios (VIC relative to paper) with 95% CIs. Likert scale data for each treatment group were summarized as mean (SD), as well as the perceived time to complete the IC process, and differences in the distribution between VIC and paper were tested using the Cochran–Mantel–Haenszel statistic. One participant in each treatment group reported 120 minutes as the perceived time, which were likely outliers and were removed from the mean calculations. Analyses were performed using SAS .The VIC trial protocol was approved by the Yale University Institutional Review Board. The research team included a chair of the institutional review board, who assisted with human rights perspectives. We anticipated potential risks with study participants consenting to a study via a digital medium and took multiple precautions to ensure comprehension of the digital IC for the GenEx 2.0 trial. The tool was tested with multiple users and subject matter experts before the study to determine its usability. The research coordinator had the ability to review the quiz results of the participant, as well as the time taken to read each section of the consent. The research coordinator evaluated each participant’s quiz scores and the time spent going through the consent to determine if the participant was properly informed. If the research coordinator did not feel the participant understood the nature of the study, the coordinator had the ability to withdraw the consent.The VIC tool was hosted on a secure server located on the Yale University network and only accessible to those with explicit administrator access via the Yale University network through Yale’s Central Authentication System. Databases storing participant data were also maintained on the Yale University network and hosted on separate servers from the tool itself to increase security measures. The databases were backed up and stored securely in a separate location on the Yale University network with access granted only to the system administrator and principal investigator. As with any digital application, we planned and anticipated common issues such as bugs, and although no bugs occurred during the trial itself, the developer was available to maintain the system and ensure stability of the application.A total of 91 individuals were approached for the study, of whom 25 were ineligible because they had already participated in GenEx 2.0. The remaining 66 individuals completed the initial screening questionnaire, and of them, 16 participants were deemed ineligible because they either did not provide an email address (n=4), refused to participate (n=5), or other unknown reasons (n=7). A total of 50 individuals were ultimately enrolled in the VIC trial, and 25 were randomized to the VIC intervention arm and 25 to the paper IC control arm .The mean age of the participants enrolled was higher in the VIC arm  than in the paper arm ; with a mean difference 9.4 (95% CI 0.9-17.9). Overall, other demographic characteristics, such as race, ethnicity, household income, and relationship status, were comparable between both arms . RegardiOverall, participant comprehension was high in both the IC process groups. For the comprehension outcome , VIC parP=.05).The participants were also asked questions to ascertain their level of satisfaction and perception with the 2 consent processes . OverallP=.02 and P=.045, respectively). When asked about their level of understanding on how to withdraw from the study, VIC participants had a mean score of 7.0 versus 6.7 for paper consent participants. Furthermore, VIC participants reported a lower mean perceived time to complete the consent process, 12.9 (SD 7.6) minutes for VIC participants versus 16.6 (SD 9.7) minutes for paper consent participants; mean difference of −3.7 (95% CI −9.0 to 1.5) minutes. In terms of satisfaction with completion time, VIC participants had a higher mean score than paper consent participants . Regarding overall process difficulty, VIC participants scored the process as less difficult, with a mean score of 6.3 compared with paper consent participants with a mean score of 5.9 (P=.02).Furthermore, on average, VIC participants had a slightly better understanding of who in the study had access to their PHI and what would happen if they chose to participate in the study (mean scores of 7 for both), although paper consent participants scored a mean of 6.6 and 6.7, respectively  than participants from the paper consent group (6.4 VIC vs 5.9 paper). Both IC groups reported that they were very likely to participate in the future clinical trials (6.4 VIC vs 6.3 paper) and that the IC process was relatively important in their decision to participate in the clinical research study (5.0 VIC vs 5.7 paper).The study was conducted to test the feasibility of the VIC digital consent tool that enhances the IC process with interactive content on tablets. We built a VIC tool with a reusable infrastructure that allows for integration into the IC process for future research studies and may improve the clinical workflow through a more efficient IC process. The VIC tool can assess participant comprehension through automated quizzes and self-tests, emphasize topics using multimedia, and allow individuals to view demos and presentations. The participants can also listen to comments and explanations, get customized information, click on links to drill down for more information, ask questions, receive answers, and rewind and replay audio and visual components as needed. The VIC tool also provides a retrievable electronic record of the IC document, including the electronic signature, which can be integrated into modern electronic health records. Although not performed for the purposes of this study, VIC can also be performed remotely and securely with the electronic signature feature, and on a PC or smartphone if needed.These innovative, dynamic features enhanced the overall participant experience in the ongoing research study, GenEx 2.0, compared with the standard paper IC process. In our study, the VIC trial participants reported significantly better understanding of the use of their PHI and participation in the study and greater satisfaction in completing the IC process because of the shorter duration. Previous systematic reviews have suggested that enhanced consent forms and extended discussions are most effective in improving participant understanding, and our findings confirm this claim ,28-32. TSystematic reviews have also suggested that the integration of audiovisual elements into the IC process can improve participant recall with no adverse impact on satisfaction or anxiety , and we A particular strength of our study was the option of having the participants provide consent to participate in an actual ongoing parent study in which they were enrolled prospectively, rather than asking them to imagine that they would take part in a hypothetical study. The real-world aspect of the study ensured that we were actively collecting data instead of relying on retrospective analyses, which have inherent limitations ,27.One comprehension question that provided indication of poor content delivery in both arms was the question of whether participants believed that they or their insurance carriers would be expected to pay for the costs of the treatment if they were injured while participating in the study. In the VIC arm, 64% (16/25) of the participants answered incorrectly compared with 52% (13/25) of those who answered incorrectly in the paper arm. In contrast, the other 12 questions indicated moderate to high comprehension of the content in both arms. This may be a reflection of the difficulty conveying language typical to the consent form regarding this specific section, because our results indicate it is atypical to the positive trend.There are some limitations with regard to the nature of this study, which may have affected certain findings. One limitation of this study was that we did not have masked research staff administering the survey to participants after the IC process. The study coordinator for GenEx 2.0 was the same person who administered the paper-based and VIC IC processes and collected study surveys, which limits our control over observer bias. Another limitation was that we did not directly and independently measure the time to complete the IC and instead relied on the participant-reported perceived time to complete the IC. We also limited participation in this study to individuals who spoke English, had an email address, and were willing to use the iPad. Although most participants were confident in using technology, we believe that these conditions could have potentially limited the generalizability of the findings because 6% (4/66) of the screened individuals did not provide an email address. Although the GenEx 2.0 study topic difficulty level could be described as moderate with very few study procedures involved, VIC may perform significantly better with more complicated topics with the use of multimedia and dynamic features to reinforce information, such as the “teach-back” quiz feature. In addition, our sample size of 50 participants could have affected the ability to detect differences between VIC consent participants and paper consent participants, which we plan to address in future research regarding the tool.Potential future research topics that would be important to consider and explore with the VIC tool would be expanding future research using VIC to include participants of other languages other than English, as well as population with various hearing and vision impairments. The integrated text-to-speech and other audiovisual components of VIC are innovative language integration tools that would make switching to consent in other languages a very feasible task. Moreover, the tablet-based methods of VIC also offer various accessibility tools that can be considered to reach a more inclusive target population, and with VIC’s primary feature prioritizing a customized level of information, we feel that there is great potential for future topics to maximize its usability. Although cost analysis was not performed for this study, we believe that in future studies, the cost of creating a dynamic, digital IC may lessen as the tools become scalable and feasible to others.Our team plans to disseminate the VIC tool in collaboration with other institutes to facilitate the adaptation of digital consent platforms with the goal of creating a scalable, dynamic, and effective IC process.This study found that the VIC tool is feasible when integrated into a real-world research study, and the use of multimedia and other interactive features via a tablet-based IC process led to greater satisfaction in delivering important content compared with the standard paper process. VIC participants reported a lower perceived time to complete the IC process and higher comprehension, as well as higher overall satisfaction compared with the participants in the control group. Our preliminary findings suggest that compared with the standard paper consent process, dynamic digital IC processes can enhance comprehension and satisfaction and transform the consent process for human-based research studies.


Travel restrictions amidst the COVID-19 pandemic reshaped interviewing for fellowships into a predominantly virtual process. How this impacts Obstetrics and Gynecology (OB/GYN) resident approaches to fellowship application and Match navigation is largely unknown.We performed a cross-sectional survey study of fourth year OB/GYN residents in the United States who participated in at least one virtual fellowship interview in 2020. We collected information regarding demographics, application strategy, perceived strengths and weaknesses of virtual interviews, and confidence with rank list creation. Descriptive statistics were used for categorical variables and responses pre- and post-Match were compared using Fisher’s exact test.n = 45, 60%) or understanding the culture of programs  to create a rank list. Cost savings were the main benefit of virtual interviews , and inability to get a true “feel” for a program was the biggest limitation . A majority (46.7%) advocate for a future hybrid interview process.Seventy-five out of an estimated 490 applicants (~ 15% response rate) completed the survey. Of the respondents, 65.3% felt they interviewed at more programs virtually than they would anticipate completing in person, but perceived less confidence in having the necessary information (OB/GYN residents pursuing fellowship reported interviewing at more programs during the virtual season, but had less confidence with rank list creation. Cost savings benefits are weighed against difficulty getting a “feel” for programs virtually. Most would advocate for a future hybrid interview process.The online version contains supplementary material available at 10.1186/s12909-022-03113-3. Although restrictions imposed by the COVID-19 pandemic changed the process of Obstetrics and Gynecology (OB/GYN) subspecialty fellowship recruitment to an exclusively virtual experience for the 2020 recruitment season, some have long questioned the necessity and cost of traditional in-person interviews [In a 2014 survey of first-year Maternal-Fetal Medicine fellows, the cost of in-person interviews amounted to almost $7000 U.S. dollars per applicant. This monetary cost did not include the toll associated with the physical burden of travel and time away from clinical duties, which often includes a complex system of arranging clinical coverage. Previously, programs have made small steps toward “regionalization” of interviews  and have considered the role of virtual interviews and the idea of a centralized “job fair” (where fellowship administrative personnel gather in one place to meet with a crowd of applicants), but innovation has been slow without impetus [However, while financial and logistical advantages of virtual interviewing may be obvious, OB/GYN resident perception of virtual fellowship interviews, strengths and limitations of virtual interviewing, and the impact on creating rank lists is largely unknown . FurtherThe aim of this study is to assess OB/GYN resident strategies in fellowship application, perception of the strengths and limitations of virtual fellowship interviews, and overall confidence with decision-making in creating their rank list.p-values were calculated and the significance level was set at < 0.05. Open-answer questions responses were reviewed for themes. The study was approved as exempt by the Institutional Review Board at Women and Infants Hospital in Providence, Rhode Island.We performed an anonymous cross-sectional survey study of fourth-year OB/GYN residents in the United States applying into subspecialty fields including Family Planning, Female Pelvic Medicine and Reconstructive Surgery, Gynecologic Oncology, Maternal-Fetal Medicine, Minimally Invasive Gynecologic Surgery, Pediatric and Adolescent Gynecology, and Reproductive Endocrinology and Infertility. This novel de-identified electronic survey consisted of 39 multiple choice, Likert scale, and free response questions covering demographics, strategies regarding number of programs targeted, perception of programs based on pre-review materials provided, self-perception of performance during the virtual interview day, preferences regarding the practical implementation of virtual interviews, confidence with decision-making in rank-list creation, and overall perception of virtual interviews  or Gynecologic Oncology . Among respondents, 64% completed the survey prior to the Match, while 36% completed it post-Match. No respondents reported past experience with virtual interviews. Though most respondents reported applying to the same number of programs as they presumed they would have for in-person interviews , most respondents reported interviewing at more programs than they anticipated they would have for in-person interviews , citing both minimal additional cost and anticipation of a more difficult year matching as reasons for doing so. Among respondents, 48.0% estimated saving at least $5000 U.S. dollars on travel expenses, and 95.9% reported anticipating the need for more coverage with in-person as compared to virtual interviews , program reputation , personalized interview schedule , interview day program overview , interview day interviews , and “question and answer (Q&A)” session with fellows  as very helpful, whereas pre-review materials , program brochure , program website , and informational video/virtual tour  were only somewhat helpful. The program “social” or “happy hour” was rated as somewhat helpful by the majority of applicants . Program social media was the only field to be rated largely neutral . All participants reported preference with the Zoom platform for interviewing, and most  preferred interviewing with individuals rather than a panel of interviewers (defined as a group comprised of two or more interviewers). When asked about the logistics of an interview day, most respondents preferred an average of five 20-min interviews  with a 5-min break in between each interview  , respondents mostly rated the program video , but were less confident in having the necessary information  or understanding the culture of various programs  to create an informed rank list, citing inability to get a true “feel” for the program in the majority of instances  , and inability to get a true “feel” for a program was identified as the biggest limitation of the virtual interview season  , or overall confidence in decision making to inform a rank list (p = 1.00)  Table .Table 5CIn our study of fourth-year OB/GYN residents in the United States applying into subspecialty fellowships via a virtual interview process during the COVID-19 pandemic in 2020, we found that although candidates applied to about as many programs as they anticipated they would have for in-person interviews, participants were more likely to interview at more programs due to minimal additional cost in the virtual format as well as perceived difficulty in matching during an unprecedented year. Respondents identified program interview components including a pre-recorded video of the program and Q&A with current fellows as very helpful to the process, whereas events such as the virtual “happy hour” and social media presence of the program were surprisingly less helpful. Additionally, despite interviewing at more programs, respondents reported feeling less confident in their decision-making when creating a rank list compared with perceived confidence had the interview season been conducted in person. Inability to get a true “feel” for each program was cited as the biggest limitation of the virtual interview process, with programs blending into each other and losing individuality on a virtual platform, while cost savings, convenience and minimizing burden of clinical coverage were identified as potential advantages. Importantly, a common theme among free response answers was the increased access to interviews, especially for applicants from smaller programs who would have had more difficulty arranging for clinical coverage given that less time overall was needed for interviews. Virtual interviews may therefore offer increased accessibility and equity as compared to in-person interviews. Nevertheless, more respondents would rather repeat an entirely virtual than in-person interview experience, although most would advocate for a hybrid process of virtual and in-person interactions, which may combine the advantages of virtual interviews without missing out on irreplaceable aspects of in-person interviews.Our results supplement scant existing though expanding knowledge regarding conceptualization of virtual interview processes within OB/GYN by both applicants and interviewers. A recent study by Lewkowitz et al. of Maternal-Fetal Medicine fellowship applicants suggested that OB/GYN residents are likely to accept more virtual interview offers than in-person interview offers given increased ease and convenience, which may have unintended downstream consequences on the ability of certain candidates to be offered interview opportunities . ResultsOur study was limited by an inability to compare groups in real-time, as respondents were asked to compare their virtual interview experience with a hypothetical comparison of an in-person interview experience. Respondent views relied on perceptions having interviewed in person for residency in the recent past. Also, despite our attempt to reach all applicants by disseminating the survey to all program directors in the ACOG program director database several times, we cannot be certain all OB/GYN subspecialty applicants had access to respond to our survey. Applicants who are not current U.S. or Canadian OB/GYN residents  were also not privy to participate in our study. Our response rate of ~ 15% is a substantial limitation of our study. Furthermore, although most OB/GYN residents in the United States identify as White and female (62.7 and 83.8% respectively according to the 2020 report on residents from the Association of American Medical Colleges [Our study poses a substantial question for subsequent interview cycles: how can hybrid application processes be implemented with maximal acceptance and trust by all stakeholders? Options previously proposed include having candidates choose between virtual or in-person interviews with programs offering both options versus an initial round of virtual interviews followed by a subsequent in-person visit to the program for interested parties. In-person visits can even be scheduled after programs have submitted their Rank list and prior to applicants finalizing their Rank list to help applicants evaluate intangible elements such as culture of programs without programs taking these in-person visits into account, as not all applicants will be able to afford traveling to every program. Future interview formats should take into consideration cost of any in-person component as these savings are a major advantage of the virtual process. Further study dedicated to creating and implementing best practices for virtual interview components and consideration of a hybrid interview model combining virtual and subsequent in person events are essential next steps. This information is crucial to both administrators and applicants in coming years as we plan for a new evaluation landscape that is both more streamlined and personal, one that minimizes unnecessary time commitment and cost while enhancing both stakeholders’ ability to evaluate the other party without bias.Our study demonstrated certain advantages and drawbacks of an exclusively virtual interview process, mainly the tradeoff of ease in scheduling interviews and cost savings for the uncertainty of ranking a program highly after only interacting in a virtual space. Respondents indicated the desire to consider a hybrid of virtual and in-person formats for future interview seasons.Additional file 1.


During the scaling-up of a national Norwegian take-home naloxone (THN) program, data collection methods shifted from paper-based to electronic. The aim of this study was to explore staff preferences towards the shift in data collection.n = 200). The survey included 17 questions, and covered staff demographics, experiences distributing THN, preferences towards data collection , and an open response section. Descriptive statistics were performed for the survey results. The open response section was recorded from each questionnaire and was coded into major themes by the authors.In January–February 2020, a survey was sent out via email to personnel involved with the THN program (In total, 122 staff completed the survey. Of these, 62% had experience with both electronic and paper-based forms, and there was a near unanimous preference towards electronic data collection over paper-based forms. From the free-text responses, staff found the electronic form to be a useful tool for conversation and overdose prevention education, and that the electronic form was easier to manage than the paper forms.The shift towards electronic data collection was necessary for the feasibility of the Norwegian national THN program. This study found that staff not only tolerated the shift, but in most cases preferred this organizational change. Overdose mortality in Norway is approximately 260 per year, with opioids being involved in most cases . In respScaling-up of THN programs to reach relevant and sizeable populations of high-risk users is possible , howeverIn the Norwegian context, the scaling-up of the THN-program led to a wider reach of clients, but it came with implementation and operational challenges. One of the main challenges related to data collection. Program evaluation was explicitly included in the program development, and as such, data collection was an integral component. As the program expanded, collecting paper-based questionnaires and registration forms from several distribution sites across the country became unfeasible. Consequently, project management needed to shift the program’s data collection from paper forms to an electronic form.Data collection was primarily conducted at low-threshold facilities, including drug consumption rooms and shelters . Low-thrFor the Norwegian THN program, it was necessary that the shift in data collection was acceptable to staff members implementing the program. The aim of this study was to evaluate staff preferences for electronic data collection for a large-scale THN program in a low threshold setting.This study was a cross-sectional survey study exploring distribution staff preferences towards the shift from paper to electronic data collection in the Norwegian THN program.At the time of the study, THN was available at over 100 distribution sites throughout the country. Distribution sites included low-threshold facilities, rehabilitation centers, street outreach, medical clinics, and prisons. All staff members distributing naloxone through the THN program are required to complete the train-the-trainer-course developed by the THN program . In addiThe paper forms were either stored at the distribution sites until a project coordinator picked them up physically or returned by mail. The time needed for data entry of the paper forms also presented a delay in processing data. As the program expanded, the workload related to paperwork escalated proportionally. In 2018 the data collection shifted from paper forms to an electronic form to improve the processes.The electronic form was developed in cooperation between peer advocacy groups, low-threshold facility staff members, and researchers. All of those that were involved with the forms  were involved in the process. Others have found collaboration among different levels improves buy-in , and thaThe electronic form were located online via the THN program’s website. It was a simple open-source html page accessible to most browsers. The questionnaire was programmed to eliminate illogical pathways and consisted of checkboxes. Participant consent was collected electronically. The form was automatically stored in the University of Oslo’s safe storage for research solution.During a two-week period in January–February 2020, a survey was sent via email to personnel involved with the Norwegian THN program. Approximately 200 staff members received the survey directly by email. Recipients were encouraged to forward the survey to other colleagues and relevant personnel. A reminder was sent out one week later, and participants had two weeks to respond before the survey closed. Being a trained staff member at one of the distribution facilities with experience with both questionnaires were the eligibility criteria. Those who reported to only have experience with one of the questionnaires were excluded from the study.The survey included 12 questions, and covered staff demographics, experiences distributing THN, preferences towards data collection , and a free-text response section where participants could elaborate on the use of the electronic form and the program in general.Frequency analyses were performed for the survey responses using SPSS statistics 27. The free-text response section was recorded from each questionnaire and was coded into major themes by the authors. The free-text response data provided insight and contextualization to each respondent’s preferences and experience relating to the Norwegian THN program and the data collection shift.This study was approved by Norwegian Centre for Research Data (624040).n = 48). The majority  of respondents were women. All participants had worked at their current workplace for more than 12 months (n = 71) and their mean age was 40.8 years old. Participants were asked about previous naloxone trainer course, and most had taken their trainer course more than a year ago . The remaining completed the course less than 6 months ago , 6–12 months ago , were not able to recall  or reported never taking the course . Participants were also asked about their distribution activity, and most respondents reported distributing one or more naloxone kits in the past 6 months .In total, 122 staff members completed the questionnaire. Of those, 74 reported to have experience with both forms of data collection. Those who reported experience with just one of the forms were excluded from the study (n = 70) preferring the electronic form. When asked if the electronic form affected the trainings, the majority  either found no difference or that it affected it in a positive way  elaborated on this topic. Responses included [translated from Norwegian]: “The electronic form are a more natural way of obtaining information on the client’s drug habits through conversation.” One respondent emphasized that, the electronic form had a positive impact on her relationship with the clients: “It is a good way to explore and address traumatic experiences. It is a way to empower the client to talk about, and reflect upon experiences, and drug habits, with a focus on overdose risks. It also has a positive impact on our relationship with the client.” This theme is further supported by the structured survey results in Table The first was that the electronic form was a useful tool for conversation and overdose prevention education. Half of those who responded in the free text  of the answers categorized under this theme: “With the paper forms I experienced that the client got a bit agitated seeing a stack of paper to fill out. This is not the case with the electronic form. Also, it is easier to use the THN instructional video as an educational tool when using the electronic form”. One respondent found the electronic form to offer clarity: “The questions are clear and specific, which encourages filling out the form.”The second theme was about how the electronic form was easier to manage than the paper forms. Just over one-third , is also associated with individual barriers . In thisWhen implementing organizational changes in healthcare, promotion of a “shared vision” and buy-in is important . This waFor researchers, electronic data collection has been found by others to be more accurate and cost-effective –22. FromThe study assessed staff members’ preferences towards the data collection shift, and did not collect feedback from the clients, which could have given a broader perspective. In addition, our sample was a convenience sample and not necessarily a representative sample of the entire population. However, given that most had distributed naloxone within the last six months, we believe that we have reached relevant respondents nonetheless.Overall, our findings indicate acceptance towards the shift from paper forms to an electronic form for a national THN program, without loss of quality from the services provided to the end-users. Staff preferred the electronic form, which may translate to improved client interactions and data collected for research. The transition to the electronic form was necessary as the program expanded, as the physical collection and data entry for a large, nationwide program was not feasible. Others who are in a position to improve data collection practices, particularly with large-scale or widespread programs should consider the use of electronic forms. It is however important to consider buy-in from those collecting the data on a site-level to ensure that the format fits the setting, and to include client experiences as well.


BackgroundThe U.S. residency recruitment process is expensive and time-consuming because of application inflation and over-invitation.ObjectiveUsing interview and match data, we quantify the predicted effects if anesthesiology residency programs excluded interviews for applicants who are very unlikely to match.MethodsWe previously published the validity and accuracy of the logistic regression model based on data from interview scheduling software used by 32 U.S. anesthesiology residency programs and 1300 applicants from 2015-18. Data used were program region, applicant address, numbers of interviews of the interviewee, medical school US News and World Report (USNWR) rank, the difference between United States Medical Licensing Exam (USMLE) Step 1 and 2 Clinical Knowledge (CK) scores, and the historical average of USMLE scores of program residents. In the current study completed in 2020, the predicted probabilities and their variances were summed among interviewees for 30 deidentified programs.ResultsFor anesthesiology, the median residency program could reduce their interviews by 16.9%  supposing they would not invite applicants if the 99% upper prediction limit for the probability of matching was less than 10.0%. The corresponding median savings would be 0.80 interviews per matched spot (0.34-1.33). In doing so, the median program would sustain a risk of 5.3%  of having at least one interviewee removed from their final rank-to-match list.ConclusionUsing novel interview data and analyses, we demonstrate that residency programs can substantively reduce interviews with less effect on rank-to-match lists. The data-driven approach to manage marginal interviews allows program leadership to better weigh costs and benefits when composing their annual list of interviewees. United States residency programs have seen an increased number of applications and interviews in the last five years (denoted “application inflation”), which imposes a large cost burden to applicants and programs alike -2. VideoOur prior work demonstrated the novel use of interview data from anesthesiology residencies to quantitate the effect of an applicant being from the same state on the probability of matching at a residency program . An applExpanding on our work , we soughttps://thalamusgme.com/) [The study was reviewed by the University of Iowa Institutional Review Board (IRB 201909708). Prior to submitting this for review, interviewee and residency program data were gathered from four software platforms  and de-identified before analysis (described below). The University of Iowa IRB determined that the study of this de-identified data does not meet the regulatory definition of human subjects’ research, and therefore did not require review of the IRB or written consent from interviewees or programs. Thalamus is a cloud-based, graduate medical education (GME) interview scheduling software and management platform  .The current study extends work completed in 2020 , whereinAnalysisIn order to assure program anonymity, two of the 32 programs in the original paper were removed because these programs each had fewer than 10 interviews. The two programs were removed after parameters and variances were estimated. The 30 programs studied in this work come from 19 unique states (40 US States and DC have at least one anesthesiology program). The median anesthesiology program has seven other programs in their state. The programs in our sample also had a median of seven other programs in their states.Our current study quantifies the potential reduction in surplus interviews per program, and the associated risk, through the implementation of a decision rule based on an estimated probability of matching. To achieve this, we needed a method to compute the number of interviews that would cease at a given specified certainty and threshold as well as a method of computing the associated risk of a change in the final match list. We computed intervals for each prediction at multiple levels of certainty . Whereasie to refer to the binary prediction from the logistic regression that interview i resulted in a match. The predictions and prediction intervals of the logistic regression are initially calculated on a log-odds (‘logit’) scale. That means that with ie being a prediction from the logistic regression, i = log (pi/1-pi) where pi is a probabilitye. The inverse logit function -1(ei) = exp(ei)/exp(ei)+1 = piL was used to recover the probabilities ip.We use is in the logit scale and added to ie to form an upper limit on the probability i = ei + si x l. The inverse logits of the interval were used to attain probabilities. The upper probability limits were then compared on the prediction intervals with a chosen probability threshold t; i.e., we removed interviews where p<t. Thresholds of 0.01 (1%), 0.05 (5%), and 0.1 (10%) were examined.Our objective was to estimate the proportion of interviews conducted per program and with a low probability of a match. To decide whether an interview was surplus, choices of a width of prediction interval as well as a threshold of minimum probability were made. Prediction intervals were constructed through a choice of certainty t f represented an indicator function corresponding to whether a prediction interval upper limit fell below the minimal threshold to be considered for an interview. The sum of t fover a program’s interviews yielded the total interviews to be removed for that program, at a choice of confidence level and threshold. Let c(P) denote the size of P . The number of interviews cut for each program are defined via a method for computing in Equation (1)  in the denominator of fhaccounts for the variation in the number of interviews among programs. Equation (1)  was computed. The computations follow the same pattern as above but sum the probabilities rather than interviews, shown in Equation (2)  and the denominator of Equation (2) to estimate the reduction in interviews counted per matched spot of the program.Estimating the characteristics of the median programt and gth. The table shows 97.5% two-sided non-parametric conservative confidence intervals for the median by reporting the eighth and twenty-first of the ranked values of th (sum of interviews) and tg (sum of probabilities) [The calculated quantities from the logistic regression model for each of the 30 programs are provided in the data workbook provided in the supplemental content. We calculated the reduction in interviews and associated cumulative risk of implementing a cutoff for each program. We chose a useful set of cutoffs to include in the workbook and summarize these findings in Table ilities) . Non-parIn this four-year cohort of anesthesiology match cycles, the median residency program would benefit in its ability to reduce the number of interviews completed by approximately 16.9% . As another example, an applicant may be overqualified for the program and have never even visited the state. However, the University has an investigator with which the applicant wants to do her postdoctoral fellowship. We find that the balance of risk and reward are best achieved through the highlighted row of Table The Coalition for Physician Accountability recently released recommendations to help guide the transition from undergraduate to graduate medical education that was admittedly lacking in an evidence base and formed primarily by consensus . The objWith an impending change in USMLE Step 1 reporting to Pass/Fail and increasing focus on diversity and equity, innovative metrics incorporating applicant-program fit are needed. This initiative requires that programs and applicants optimize interviews with the likelihood of match. We consider the "closeness" of interviewees to programs on dimensions of widely used measures of scholastic performance including board scores and membership to honor societies.LimitationsThe usefulness of our study is limited by the precision of the underlying probabilistic model . That model had a few drawbacks, chiefly that some of the data  were frequently missing.We do not directly consider race, ethnicity, or socioeconomic status. Therefore, we do not suggest that programs perform this analysis and then naively implement an interview threshold with no consideration of the nuances of candidates. Instead, we expect that these probabilities would be used as an aid to help programs feel more confident when determining who they should/not interview. If programs have particular recruiting criteria, such as attracting additional underrepresented minorities, programs can simultaneously consider our model's predictions as well as interviewees' demographics, and interview students who meet recruitment goals but are also relatively likely to match. Programs could also consider underrepresented minorities completely outside of this framework. The goal of this work is not to give programs a black box with which to blindly cull their interviewees but instead to provide an evidence-based tool to complement decision-making in recruitment. We then provide a summary of the potential reduction in interviews as a guide for programs to consider how to appropriately implement thresholds for their individual program.Another notable limitation was that the model used a commonly reported field, “Current Address,” to determine whether an interviewee and interviewer were located in the same state. This, of course, misses some of the nuances of the geographic relationships between interviewees and programs. However, this model has been validated . In the We aggregate the probabilities of select applicant and program characteristics in effecting a match to create a useful quantitative framework for programs to evaluate whether to invite applicants with a low probability of a match. We showed that a well-chosen decision rule nominally affects a program’s final match list when forgoing interviews with a low probability  of matching. Our results of a median 16.9% reduction in interviews (0.80 interviews per matched spot) are novel because they are calculated using data on interviews and not only of match lists.