Due to Covid-19, fellowship programs could not conduct in-person interviews during the 2020–2021 interview cycle and were forced to implement virtual interviews. We conducted two nationwide surveys of residency and fellowship Program Directors (PDs) involved in the Obstetrics and Gynecology (Ob/Gyn) Subspecialty Fellowship match cycle to gain a better understanding of virtual interviews from each of their perspectives. 1) Fellowship PDs’ confidence in using a virtual platform to holistically evaluate applicants during the 2020–2021 match cycle, 2) Residency PD’s perception of virtual interviews and impact on their program’s operations, and 3) to assess the desire of fellowship and residency PDs to continue virtual recruitment during forthcoming interview seasons. Two separate nationwide web-based surveys were administered to 1) Ob/Gyn fellowship PDs and 2) residency PDs through SurveyMonkey from July-September 2020 to assess the impact of virtual interviews form each parties’ perspective. Surveys solicited demographic information, four-point Likert scale questions, and free response questions Of programs meeting inclusion criteria, 75/111 (67.6%) fellowship PDs and 67/117 (57.3%) residency PDs responded to their respective surveys. Most fellowship PDs believed that they could confidently assess applicants’ professionalism (88%) during a virtual interview and (90.7%) felt confident in making a rank-order list. However, only 73.3% were just as confident in preparing a rank list after a virtual interview as they have been with in-person interviews. Most residency PDs (69.9%) believed that virtual interviews made it easier for their program to comply with duty hours, and 76.8% agreed that virtual interviews allowed their residents to accept more interviews than an in-person format. Most fellowship PDs found virtual interviews convenient. However, difficulty in observing social interaction and gauging applicant interest may be the biggest challenge moving forward. Obstetrics and gynecology residents applying for fellowship positions traditionally participate in face-to-face interviews with faculty members at each prospective program. This process is structured to allow applicants an opportunity to visit program sites, interact with program faculty, and socialize with the current fellows. Likewise, program directors and faculty use the interview process to assess an applicant’s academic motivations, professionalism, interpersonal skills, and overall fit for the program. However, there are several disadvantages to the traditional in-person interview format that impact residency programs, applicants, and fellowship programs. Due to required travel, in-person interviews are cumbersome to residency programs because of resident absenteeism, work-hour restrictions, disrupted program workflow, and loss of continuity of care . ResidenThe Covid-19 pandemic has had widespread impact upon the healthcare system, including academic medicine. The resultant travel restrictions and social distancing policies precluded applicant travel for fellowship interviews during the 2020–2021 match cycle. Interestingly, virtual conferencing interview platforms have been reported as early as 2000 for internal medicine residency interviews . Since tAlthough virtual interviews have not previously been utilized by any of the Obstetrics and Gynecology subspecialty fellowship programs, the 2020–2021 interview season presented an opportunity to evaluate its utility. We aimed to assess fellowship program directors’ perspectives surrounding the 2020–21 match season conducted virtually due to the Covid-19 pandemic. Many speculated that the virtual match process might lead to an overwhelming number of unmatched applicants and programs. However, the NRMP statistics for the 2020–2021 Ob/Gyn Fellowship match outcomes showed that the number of applicants per position, the percentage of matched applicants, and the percentage of filled programs were comparable to the previous three years . Yet sevThis prospective study was conducted at New York Medical College School of Medicine  and Richmond University Medical Center . The study protocol  was reviewed by the Institutional Review Board of New York Medical College and received an IRB exemption. We conducted a nationwide survey of residency and fellowship Program Directors (PDs) accredited by the Accreditation Council for Graduate Medical Education (ACGME) involved in the Ob/Gyn Subspecialty Fellowship 2020–2021 match cycle. The objectives of our study were to evaluate 1) Fellowship PDs’ confidence in using a virtual platform to holistically evaluate applicants during the 2020–2021 match cycle, 2) Residency PD’s perception of virtual interviews and impact on their program’s operations, and 3) assess the desire of fellowship and residency PDs to continue virtual recruitment during forthcoming interview seasons.TM. Surveys were administered from July 2020 – September 2020 to Ob/Gyn subspeciality Fellowship Program Directors and Ob/Gyn Residency Program Directors. For the 2020–2021 match cycle, the Match opened on 24 June 2020, the Rank Order List Certification deadline was September 30th, 2020 and the Match results were announced on October 14th, 2020.This study was conducted by email questionnaires administered through the survey tool, Survey Monkeyhttps://apps.acgme.org/ads/Public/Programs/Search). The generated list for each specialty was reviewed and programs that provided an email address of the Program Director were included. Exclusion criteria were programs providing email addresses for only the Program Coordinator, programs with no email address listed, and fellowship programs not participating in the 2020–2021 match cycle. During the 2020–2021 Match cycle, there were a total of 265 ACGME Accredited Fellowship Programs and 285 ACGME Accredited Obstetrical and Gynecological Residency Programs. Of the total number of programs, 115/265 (43.4%) Fellowship Programs and 122/285 (42.8%) met inclusion criteria and were sent surveys.The Fellowship Program Directors surveyed were those from the Female Pelvic Medicine and Reconstructive Surgery (FPMRS), Gynecologic Oncology (Gyn/Onc), Maternal Fetal Medicine (MFM), and Reproductive Endocrinology and Infertility (REI) subspecialties. Email addresses of Fellowship Program Directors and Ob/Gyn residency Program Directors were gathered from the national Ob/Gyn residency program list developed by Accreditation Council for Graduate Medical Education (ACGME) using the Institutional and Program Finder tool  were involved in the construction of the three survey tools. JMP has been an Obstetrics and Gynecology Residency Program Director for eleven years and has extensive experience in resident recruitment and the interview process; MK was a fellowship applicant and had personal experience with the virtual interviews, and NAL is a Clinical Research Fellowship Program Director with six years of experience in both resident and fellowship recruitment. The investigators developed survey tools based on existing literature, knowledge of current residency and fellowship interview practices, discussions with experts in the field, and their own experience in resident recruitment. In addition, the authors utilized data from the National Residency Match Program (NRMP) Specialties Matching Service Program Director Survey which contains information on what factors program directors consider important in ranking applicants . All thrThe Fellowship and Residency Program directors survey tools underwent a content validation process with faculty members that regularly participate in resident and fellowship recruitment. Based on these results, which showed an appropriate range of variance, we further refined the survey, including incorporating questions on the convenience of the virtual format. The survey tool was then piloted by a group of 5 program directors of varying specialties that closely resemble the targeted recipients. The results of the pilot resulted in no substantive changes in survey content, but several questions were edited for clarity based on this feedback.The final survey tools contained 11–13 four-point Likert scale questions evaluating demographic factors as well as several factors regarding the interview and matching process, and up to four free response questions . Likert Data were analyzed using the Statistical Package for the Social Sciences . Likert data were analyzed by computing means, and response distributions, for each question to compare self-reported participant agreement or disagreement with survey statements. For qualitative analysis of free response questions, JMP, MK, and NAL reviewed all qualitative data individually to identify salient themes. They pooled their initial analyses and reviewed the qualitative together until no new themes emerged. The final illustrative quotes were chosen to represent all salient themes by a majority consensus of investigators.A total of 265 ACGME accredited Ob/Gyn fellowship programs participated in the 2020–2021 Match cycle. A total of 115 questionnaires were sent to REI, MFM, Gyn/Onc, and FPMRS fellowship program directors meeting inclusion criteria . Overall, 2 (1.7%) opted-out, 4 (3.4%) ‘bounced back’ and 75 of the 111 (67.6%) fellowship program directors that received the survey completed it. Fellowship PDs that completed our survey included: MFM , Gyn/Onc  FPMRS , and REI . Based upon the total number of national Ob/Gyn fellowship programs (n = 265), MFM had the greatest representation in this survey n = 37/97, 38.1%), followed by FMPRS , GynOnc  and REI . Each respondent had varying amounts of experience in the position of Fellowship Program Director, ranging a minimum of 1 year to a maximum of 30 years (mean 10.1 years). Nearly half  of the fellowship program directors had experience using a virtual platform for education or business prior to the start of the Covid-19 pandemic. A virtual social event or virtual meet-and-greet was held by 48% (n = 36) of fellowship programs.Of the fellowship program directors surveyed, 90.7% (n = 68) strongly agreed or agreed that they could confidently assess the applicants’ interpersonal skills using a virtual platform, and 88% (n = 68) could confidently assess an applicant’s professionalism . HoweverThe fellowship program director responses to the free text questions revealed key themes for the first two free text questions. When asked ‘What did you like best about the virtual interview process?’ themes identified were: Cost-savings for applicants, Ability to reach more applicants, and Convenience. When asked ‘What was most challenging about the virtual interview process?’ themes identified included: Inability to assess fit for the program, Not being able to showcase facility, Inability to gauge applicants’ interest, and Adjusting to a new format. Key themes and selected quotes are shown in When asked ‘Were virtual interviews more convenient or less convenient to your fellowship program’, 72.0% (n = 52) of fellowship program directors responded that virtual interviews were more convenient for their program, 21.3% (n = 16) felt that they were no different than in-person interviews, and 6.7% (n = 5) believed that they were less convenient. Of the 52 program directors that found virtual interviews to be more convenient for their program, five further commented that the increased convenience of virtual interviews came with trade-offs, including being ‘less informative’, ‘less effective’, ‘less valuable’, ‘less useful’, as well as ‘less satisfying and more anxiety-provoking’. For those finding virtual interviews less convenient for their program, free text comments revealed that this sentiment was attributed to ‘adjusting to a new format’ and ‘additional [interview] days and more interviews’. One further elaborated on the factors associated with the novelty, stating, ‘It was less convenient, but I think that was because it was novel, and there were a lot of logistics … .for example, practicing with faculty, determining the questions, ensuring schedules, and making a program video that won’t be as necessary in coming years if we continue with a virtual interview platform.’ Regarding decreased convenience and novelty, another also commented, ‘Slightly less convenient because it was new, but less expensive and ultimately more efficient.’With respect to fellowship programs, most had a positive experience with using a virtual platform. A total of 25.3% (n = 19) programs reported experiencing a technical difficulty, but most were minor. Twelve programs experienced minor connectivity problems with individual applicant(s). In all cases, a follow-up telephone call allowed the interview to continue. A few programs experienced initial difficulty transitioning applicants between breakout Zoom rooms, but this was resolved with increasing experience on the platform. Three programs experienced a major problem, including, 1) a major storm that temporarily knocked out power on the day of the interview, 2) hospital Wi-Fi was not working on the first day of interview, and 3) obtaining an inadequate Zoom License contract that supported only 5 Zoom connections per user. However, these programs were able to overcome these challenges using ‘every other technology available to us, including but not limited to hotspots, FaceTime, [and] just good old telephone conversation!’A total of 285 ACGME accredited residency programs participated in the 2020–2021 Match cycle, of which 122/285 (42.8%) met inclusion criteria and were sent a survey. Of 122 surveys that were sent to Ob/Gyn residency program directors across the nation, 5 (4.1%) bounced-back, 1 (0.008%) opted-out, and a total of 67/117 received surveys (57.3%) were completed. Mean years of experience as a residency program director was 4.8 years (range 1–15 years). In the 2020–2021 interview cycle, 77.6% of program directors reported having residents applying for Ob/Gyn fellowships. The mean number of residents applying from these programs was 1.93 (range 1–9). When asked if they typically limit the number of in-person interviews a resident can accept, 94% (n = 63) did not establish a limit, while the remaining program directors set a limit that ranged from 6–28 interviews.All program directors were asked questions concerning the impact that traditional in-person fellowship interviews have on their residency programs. 53.7% (n = 35) of the Residency Program Directors agreed or strongly agreed that traditional in-person fellowship interviews make compliance with Accreditation Council for Graduate Medical Education (ACGME) duty hours challenging for their program. Most  agreed or strongly agreed that with traditional in-person interviews, residents who are not interviewing undertake extra workload to accommodate the travel arrangements of those who are . Due to The 52 program directors who stated they had residents applying for the 2020–21 Match season were then asked further questions about the impact that virtual interviews had on their program. Most (88.5%. n = 46) agreed or strongly agreed that compared to in-person interviews, the virtual format was less disruptive to workflow and continuity of patient care. Over two-third  believed that virtual interviews made it easier for their program to comply with the ACGME duty hours, and 76.9% (n = 40) stated that it allowed their residents to accept more interviews than an in-person format. However, 46.2% (n = 24) of program directors still felt that virtual interviews led to loss of surgical cases and clinical learning opportunities and 50.0% (n = 26) believed that there should be a maximum limit on the number of virtual interviews an applicant can accept. Finally, when asked ‘After COVID-19 restrictions are lifted, I would prefer that my residents participate in virtual fellowship interviews going forward’ 60.2% (n-36) of program directors agreed or strongly agreed with the statement. SatisfactionThe Fellowship PDs responses to our survey revealed several findings with regard to their satisfaction with virtual interviews. Key themes identified from free-text responses were convenience, ability to reach more applicants, and cost-savings. With regards to convenience, many Fellowship PDs appreciated that a virtual platform, such as Zoom break-out rooms, kept the schedule on track because ‘the time slot for interviewing was fixed and the sessions just ended’. Additionally, Fellowship PDs liked the flexibility of scheduling, not needing to ‘run from room to room’, as well as the fact that faculty and fellows were able to interview from remote sites throughout the day. Many Fellowship PDs also commented that they appreciated the convenience afforded to applicants with respect to not needing to travel and ease of intervening from a familiar environment. Another important finding of our study was that the virtual format allowed Fellowship PDs to reach more geographically diverse applicants, and that they were more likely to accept interview invitations since they did not have to travel. This may be beneficial for programs that are less competitive or more geographically inaccessible. As one PD said, ‘More applicants were likely to interview at our institution than might have if they had to pay to fly here. I might not have been on the top of their list to consider.’Additionally, based on free-text responses, many fellowship directors also recognized the financial benefit to applicants. Several studies have documented the financial burden associated with in-person interviews faced by fellowship applicants. A cohort of 269 applicants that participated in the Gynecologic Oncology fellowship match from 2008–2016, spent an average of $6,000 , using personal savings (54%), credit cards (51%), family support (11%) or personal loans 3%) to cover costs of traditional in-person interviews % to cove. With thChallengesHowever, several challenges also emerged. Fellowship PDs found it difficult to assess applicants’ interpersonal skills, showcase their programs virtually, and determine if applicants were genuinely interested in their program. Although the Likert survey responses indicated that 90% of fellowship directors believed they could confidently assess an applicant’s interpersonal skills using a virtual platform , free-teThe ease of scheduling virtual interviews also presented unique challenges to Fellowship Directors. Some expressed difficulty in assessing genuine interest in their program as applicants did not need to make a significant investment of time and/or money for travel to attend. The ability of a program director to gauge an applicant’s true interest when formulating their Rank Order List (ROL) is an important factor for several reasons. It is advantageous for PDs to rank highly those applicants who they deem are genuinely interested in their specific program because it increases the likelihood of a successful match. Overall, we found that almost 90% of Fellowship PDs claimed they could confidently prepare their rank order list at the conclusion of the virtual interviews. However, when asked if they were just as confident in preparing a rank list after virtual interviews as they had been with traditional interviews in the past, this number dropped to 73.3%. One Fellowship Director summarized their difficulty by saying: ‘Trying to gauge people’s interest … nothing says interest like spending some money to fly somewhere’. Another PD commented, ‘[I] would wager a program will be more apt to take a candidate from within the program … we definitely would have considered the insider first since the person would have been a known entity.’ Obviously gauging applicants’ interest is an important factor in the match process; the use of an allocation of signals to programs to indicate genuine interest in a program has been piloted and may prove of use in this regard .Another key challenge encountered by Fellowship PDs was the inability to showcase their program virtually. This included not being able to adequately represent their hospital’s facilities virtually and also to convey the culture of their program. Additionally, PDs were not able to properly convey the social life of the town where the program is located. Given that virtual interviews will persist, more tools could be developed to convey program facilities, neighborhood, and intangible aspects of the program to applicants. This may include the use of virtual tours of the neighborhood showcasing popular spots such as gyms, parks, and restaurants, videos featuring ‘A Day in the Life of a Fellow’, as well as links to the program’s social media accounts.Approximately half of the fellowship programs that responded to our survey hosted a virtual social event as part of the virtual interview schedule. These virtual ‘meet-and-greets’ were usually held in close proximity, either before or after the scheduled interview date. However, free text responses revealed that these virtual events did not replace typical social interactions. One program director stated, ‘We tried to do it [social event] virtually, but it doesn’t work as well for casual conversation’ and another declared, ‘Although we had a happy hour, it was not the same as happy hours we have in person’. . ImproviResidency Program Directors’ TakeThe majority of the interviews for the obstetrics and gynecology subspecialty fellowships occur from April through September. Furthermore, many interviews occur in July, when the new academic year for residency programs begin. Shortages in the workforce at this time can prove especially challenging due to promotion of current residents and onboarding of new interns. In a study by the Society of Gynecologic Oncology of the 2008–2016 match cycles, applicants spent a mean of 15 (range: 0–45) days away from residency activities and 37% reported difficulty arranging coverage . The mosVirtual interviews may eliminate many of these barriers, However, this may have unintended consequences for some applicants, as one fellowship program director commented, ‘I think the top candidates get to accept more interviews virtually as compared to if they had to travel. This reduces the number of opportunities for moderate/marginal candidates.’ Additionally, we found that 46.2% of Residency PDs we surveyed still felt that virtual interviews led to loss of surgical cases and clinical learning opportunities, and 50% believed that there should be a maximum limit on the number of virtual interviews an applicant can accept. Capping the number of virtual interviews has been suggested for residency interviews, and is a subject of controversy . AssessiGoing ForwardThe majority of Fellowship PDs surveyed (72%) in our study responded that, despite some of the challenges attributed to the virtual format, they would elect to participate in a virtual interview season in the future. Similarly, from the perspective of Residency PDs, 60.2% favored their residents participating in virtual interviews in the future. However, one limitation of our survey tool was that we did not specifically solicit Fellowship PDs opinion on a ‘hybrid’ interview option. It is unclear how a ‘hybrid’ option would look. One fellowship PD suggested that ‘ I would be willing to do a two step interview, first virtual and select fewer numbers for a second [in-person] interview’. However, it is quite possible a hybrid option necessitating multiple rounds of interviews may increase workload, especially for some smaller, lower resource programs, and thus would not be favored by all. Other hybrid suggestions included allowing applicants a choice of attending an in-person or virtual interview. However, this option may have unintended consequences, potentially disadvantaging applicants who may not be able to travel in-person.Strengths/WeaknessesStrengths of our study include the use of two survey tools that assess the different, but complementary perspectives of Fellowship Program Directors and Residency Program Directors. We included a diverse population of Residency Program Directors from all over the country, as well as Fellowship Program Directors representing four of the major Ob/Gyn subspecialties. Additionally, our survey included qualitative data in the form of free-text responses to gather a more detailed and subjective view of their experiences. However, our survey methodology enabled only PDs that provided an email address to be surveyed. Thus, our response rate may be misleading when considered in relation to the total number of ACGME accredited programs. Additionally, our survey may be influenced by response bias in that PDs not providing email addresses may have different opinions that those that do, or PDs with especially strong thoughts on virtual interviews may be more likely to respond to the survey. Nevertheless, our response rate compares favorably with that of other national program surveys such as the NRMP Program Directors Survey 2021, which ranged from 13.5% to 32.7% .Changes forced by the Covid-19 pandemic have created an opportunity to re-think how to best recruit prospective Ob/Gyn fellowship candidates during forthcoming interview seasons. Benefits to applicants using a virtual platform included financial savings, convenience, and ability to accept more interviews. Residency Program Directors found it to be less disruptive to their program. A majority of Fellowship Program Directors were confident in preparing a rank order list after their virtual interview season. Lack of social interaction and difficulties in assessing fit may be the biggest challenges for future implementation. Finally, virtual interviews may have unintended consequences. Due to lack of financial and travel constraints, highly qualified applicants may accept more interview offers, thereby disadvantaging the remaining applicant pool. This could potentially lead to inequities in the match process. Further investigation will be required as we continue to use virtual interviews in the fellowship match process.


Practice-based guidelines recommend patient education and exercise as first-line care for low back pain (LBP); however, these recommendations are not routinely delivered in practice. GLA:D® Back, developed in Denmark to assist clinicians to implement guideline recommendations, offers a structured education and supervised exercise program for people with LBP in addition to a clinical registry to evaluate patient outcomes. In this study we evaluated the feasibility of implementing the GLA:D® Back program in Australia. We considered clinician and patient recruitment and retention, program fidelity, exploring clinicians’ and patients’ experiences with the program, and participant outcome data collection.Clinicians (chiropractors and physiotherapists) were recruited and participated in a 2-day GLA:D® Back training course. Patients were eligible to participate if they had persistent or recurrent LBP. Feasibility domains included the ability to: (1) recruit clinicians to undergo training; (2) recruit and retain patients in the program; (3) observe program fidelity; and (4) perceive barriers and facilitators for GLA:D® Back implementation. We also collected data related to: (5) clinician confidence, attitudes, and behaviour; and (6) patient self-reported outcomes related to pain, disability, and performance tests.Twenty clinicians  participated in the training, with 55% (11/20) offering GLA:D® Back to their patients. Fifty-seven patients were enrolled in the program, with 67% (38/57) attending the final follow-up assessment. Loss to follow up was mainly due to the effects of the COVID-19 pandemic. We observed program fidelity, with clinicians generally delivering the program as intended. Interviews revealed two clinician themes related to: (i) intervention acceptability; and (ii) barriers and facilitators to implementation. Patient interviews revealed themes related to: (i) intervention acceptability; and (ii) program efficacy. At 3 months follow-up, clinicians demonstrated high treatment confidence and biomedical orientation. Patient outcomes trended towards improvement.GLA:D® Back implementation in Australia appears feasible based on clinician recruitment, program acceptability and potential benefits for patient outcomes from the small sample of participating clinicians and patients. However, COVID-19 impacted patient recruitment, retention, and data collection. To scale-up GLA:D® Back in private and public settings, further work is warranted to address associated barriers, and to leverage facilitators.The online version contains supplementary material available at 10.1186/s12998-022-00427-3. Low back pain (LBP) is the leading cause of years lived with disability worldwide , imposinGuideline recommendations are not routinely implemented, leading to treatment variability, wasted resources and potential harm . A recenTo improve the implementation of guidelines into practice for the management of persistent and recurrent LBP, GLA:D® Back (Good Life with osteoArthritis in Denmark) was developed. GLA:D® Back aims to translate guideline recommendations into a practitioner-delivered program to promote evidence-based self-management support . The GLAThe primary aim of this project was to investigate the feasibility and acceptability of implementing the GLA:D® Back program in Australia. We aimed to evaluate: (1) the ability to recruit chiropractors and physiotherapists to undergo GLA:D® Back training; (2) the ability to recruit and retain patients into the GLA:D® Back program; (3) the ability to directly observe program fidelity; and (4) barriers and facilitators for GLA:D® Back implementation for clinicians and patients. We also collected secondary data pertaining to: (5) clinician confidence, attitudes, and behaviour; and (6) patient self-reported outcomes related to pain, disability, and clinical performance tests.This study was approved by Macquarie University Ethics Board (Project ID 5655): 52019565510186.Chiropractors and physiotherapists were invited to participate from a convenience sample, formed from our clinical networks, assisted by the President of Sports Chiropractic Australia and one of the GLA:D® hip and knee Australia program leads, who both identified clinicians and clinics who had shown interest in, or had enquired about, GLA:D® Back. These clinicians were approached and invited via email and/or phone call by the GLA:D® Back Australia research team to participate in this feasibility study.Australian Health Practitioner Regulation Agency registered clinicians were eligible to participate if they provided treatment in a primary care setting for people with persistent or recurrent LBP. Clinicians were also required to have access to adequate floor space to conduct patient education groups and group exercise classes, and access to a computer connected to the Internet to enter data into the Research electronic data capture (REDCap) database system , 19.The Australian GLA:D® Back training course was delivered by the developers of the original program in Denmark  and locaClinicians were initially exposed to contemporary evidence regarding the burden of LBP to develop or enhance their ability to deliver patient education. Key messages relating to central education themes from the patient education material were introduced  reporting persistent or recurrent LBP were eligible to participate in the GLA:D® Back program. Patients currently under care at the clinic could participate in GLA:D® Back and if deemed relevant, continue other therapy, or care concurrently. The study was limited to participants with adequate English comprehension to complete the study questionnaires. Exclusion criteria included: first episode of acute LBP (less than 3 months duration); nerve root involvement/clear neurologic signs; suspected serious pathology; spinal trauma ; pregnancy; conditions that require a specialist consultation or could be aggravated or worsened if treated by the interventions in this study. Patients with planned spinal surgery or other major surgery within the study period were excluded, as were patients currently participating in a supervised exercise or rehabilitation program.Clinicians were directed to approach, screen, and recruit patients who they considered would benefit from improved self-management skills for recurrent or persistent LBP. Interested patients provided their email addresses and received study information and an online form in REDcap by which to provide consent. Patients who were interested in participating were required to sign informed consent at the clinic prior to commencing the program. Recruited patients completed outcome measures for the clinical registry at baseline, 8 weeks and 3 months follow up.Eligible patients performed four clinical performance tests, supervised by their clinician: the standing forward bending test , trunk fAssessing feasibility of implementation were based on the following 4 domains.We aimed to recruit 10-to-20 interested clinicians for the GLA:D® Back course within 2 weeks of contacting them. Of those recruited, we defined feasibility success if 50% of clinicians delivered the program.Clinicians aimed to recruit as many eligible patients as possible within a 3-month period. Additionally, we planned to record patient retention rates over the course of the program offered within each clinic and assess the feasibility of collecting patient outcomes at baseline, 8 weeks, and 3 months from the GLA:D® Back program commencement. Recruiting a minimum of 60 patients, with ≥ 80% retention rate  would meet our criterion for feasibility success.We aimed to observe and document elements of fidelity, i.e., adherence related to delivery of the program, including patient attendance, absences, and cancellations, as well as observing session length, evidence of individualised coaching and group discussions. We planned to observe and document the presence of materials used, such as visibility of the patient training manual, GLA:D® Back key messages poster, PowerPoint slides and clinician use of reflection exercises. Items of the fidelity checklist can be found at Additional file We aimed to explore barriers and facilitators for GLA:D® Back implementation through qualitative interviews with both clinician and patient participants. Interview questions were informed by the Theoretical Domains Framework (TDF) , to idenwww.msk.org.au) was invited to comment on the questions related to the patient’s LBP experience.One clinician from each participating clinic who delivered GLA:D® Back will interviewed with questions focussed on perspectives regarding the content of the clinical intervention and program implementation and feasibility. Patients who both completed and dropped out of the program were interviewed and asked to reflect on their experiences and what they gained by participating in the GLA:D® Back program. In developing the patient interview questions, a consumer who had experience with LBP from Musculoskeletal Australia  clinician recruitment rates; (2) patient recruitment, retention and follow up rates; and (3) directly observing the GLA:D® Back program. Qualitative data analysis occurred via codes generated and grouped into themes using thematic analysis from the qualitative semi-structured interviews .Researchers  conducted a qualitative analysis of each interview transcript by individually reading line by line to capture an overall impression of the data. Data codes, supported by Nvivo software , were collated into a broader level of potential themes. Subsequent ensuing themes were discussed until a consensus was reached by researchers . Themes related to barriers and facilitators were then mapped to the relevant domains from the TDF . Direct For clinician outcomes, group median change scores were calculated on the Practitioner Confidence, and Practitioner Attitudes and Beliefs Scales at baseline, post training and 3- month follow-up. Patient outcomes in terms of pain, disability, fear avoidance behaviour, prognosis, brief illness perception, quality of life and physical fitness were also described as median change scores from baseline to 3 months. As this was a feasibility study, no inferential analyses were performed.Eight chiropractors and 12 physiotherapists from 11 individual clinics participated in the 2-day workshop at Macquarie University, Sydney, Australia in November 2019. Clinicians were 35% female, with average age 36 years. Seventy percent of clinicians who delivered the GLA:D® Back intervention were very satisfied with the program. Clinician characteristics are described in Table Patients were 67% female, with an average age 57 years. Fifty-nine percent of patients reported experiencing LBP for more than 1 year, with 29% taking over the counter medication, and 14% prescription medication Table . Fifty-oThirteen clinicians were initially approached, with 9 recruited within 2 weeks of being contacted to undergo GLA:D® Back training. Twenty clinicians from 12 clinics were recruited within 4 weeks.Eleven of the 20 clinicians (55%) adopted the GLA:D® Back program, meeting our feasibility criteria (50%) with 8 of 12 (67%) clinics recruiting patients. Of the 9 clinicians who were unable to implement the program, two clinicians from one clinic had difficulty obtaining further ethics approval specific to their workplace , two clinicians from one clinic recruited patients but did not commence due to COVID-19, and two clinics, one physiotherapy (1 clinician) and one chiropractic (2 clinicians) had recently relocated to new premises and no longer had access to the patients they intended to recruit. Finally, in one chiropractic clinic, two of the three participating clinicians did not recruit patients. No reason was provided.Apart from one clinic that commenced recruitment immediately after the GLA:D® Back training, patient recruitment for most clinicians began within 3 months following program delivery. This was based on the upcoming holidays, considered likely to impact patient recruitment and retention over this period.A total of 57 patients were recruited by the 11 clinicians who implemented the GLA:D® Back program, just short of our feasibility criterion of 60 patients. Between 88 to 98% of patients provided baseline data from questionnaires, depending on the outcome measure. Of the participants recruited, 67% (38/57) attended the last treatment session and final assessment follow-up 8 weeks later. Similarly, 67% (38/57) completed the 3-month follow-up survey. Ninety-two percent of patients (35/38) participated in more than 80% of the GLA:D® Back program sessions. Of the enrolled participants, 33% (19/57) dropped out for various reasons, with the impact of COVID-19 being the most dominant reason. We therefore did not meet our feasibility criterion of ≥ 80% retention.We directly observed one exercise group session in person and three sessions online (due to COVID-19) at four different sites . From our fidelity checklist It’s an evidence-based program that works, I think in a nutshell. (Chiropractor)GLA:D® Back training was highly acceptable to these physiotherapists and chiropractors. All clinicians described a positive experience with the 2-day training program. The components of the program (education and exercise) were described as evidence-based and a structured package that would facilitate optimal care for their patients.The training prepared us well to deliver it. The minimal equipment is great, and … when COVID occurred was being able to do it via Zoom. (Physiotherapist)A strong referral base where doctors trust what you’re doing…. What made it easy was the really good information…. That we could send out to doctors when we were looking for people to be part of the pilot program. (Physiotherapist)Several clinicians reported finding it easy to implement GLA:D® Back. They described the following facilitators: having a referral network ; having demand ; having a short list of patients to invite to participate; keeping the cost low to patients; having the resources ready to go, including an online platform to deliver the program during the COVID-19 pandemic.We had one that had lost his job, so couldn’t afford it… a couple that probably weren’t interested in in doing it virtually (Physiotherapist)I think the biggest thing comes down to the way we marketed it, and probably not having the time to explore different forms of marketing to get a better response (Chiropractor)For the clinicians who had difficulties implementing GLA:D® Back, a common barrier was difficulty with patient recruitment. Clinicians described being slow to start or recruit, not having a rolling recruitment, and not having an adequate marketing strategy. A lack of administrative support made it challenging to start the program, including the time required to prepare the equipment needed, patient logbooks and educational resources. Other barriers included: scheduling challenges (out of clinic hours/peak clinic hours), the upfront cost of the program to patients, hospital ethics and the COVID-19 pandemic.Two dominant themes emerged from the patient interviews that related to feasibility of implementing GLA:D® Back: (1) acceptability of GLA:D® Back as an intervention; and (2) effectiveness of the program.It’s absolutely fantastic…. You don’t need any special equipment… it’s something you can take you know in your normal life.Getting that concept across that pain has got nothing to do with damage and it was really enlightening actually. And also, there was a bit of emphasis on you need to be able to do all movements you can’t be stopping movements. Which was true for me where you’ve been told not to do years ago.Most of the patients thought GLA:D® Back was practical and great, describing the program using descriptors such as ‘fantastic’, ‘simple’ and ‘manageable’ ‘essential and ‘enlightening’.It was good great to do it as a group…to motivate me to do it… you can ask a question and if you’re not sure about it I think it helps other people.Most of the patients spoke positively about the group aspect fostering some healthy competition/encouragement and said that being supervised by a clinician made them feel safe, exploring new movements.Definitely a few classes in front of them (clinicians) and get some confidence and then go back to virtual.People are starting to feel comfortable, but I think the timing of the transition ruined that… we just haven’t gone on long enough to sort of recapture it via zoom.Patients compared their experience of face-to-face to the online mode of delivery, and the transition to online worked well for most patient, but not for everyone.It’s definitely given me the confidence to try new things. And do things and push myself further then I would have before.I feel much more confident doing things. That if I do get a little flare-up it’s not going to hold me back like it did before.Most patients reported a variety of benefits such as an increase in confidence in their ability to manage their back, increased strength, a reduction in pain (except for soreness after some of the sessions), and a reduction in the number of flare-ups.Clinicians were very satisfied with the GLA:D® Back program (70%) and satisfied/very satisfied with the patient education materials (90%) Table . PractitFrom baseline measures, patients reported improved median change of 7 points on the FABQ, 8.4 points on the ODI, 1.5 points for back pain and 1 point for leg pain  adopted the GLA:D® Back program. This met our pre-determined success criteria (50%), and is in line with the GLA:D® Back 2019 Danish Annual Report of clinicians conducting the program (54%) . HoweverWe successfully recruited 57 patients, just short of our minimum criterion of 60. However, retention was impacted by a 33% dropout rate, thus not meeting our predefined criterion of ≤ 20% of dropouts. This contrasted with the Canadian program, which reported a 12% drop out rate . The COVOf our 57 participants, 35 of 38 (92%) who remained in the program took part in more than 80% of the GLA:D® Back program sessions available, suggesting good program adherence. This is comparable to the Canadian program, where 84% of their participants attended both education sessions and 74% attended the majority  16-session within the program [The patient attrition rate impacted completeness of patient-reported items, with 67% of patients completing the 3-month follow-up survey. This rate was identical to the Canadian program , but shoWe did not observe the key education messages poster during all four clinic fidelity observations, despite one or more key messages from these posters delivered during the classes by the clinician. The GLA:D® Back key messages poster is recommended for the education sessions and integrated into the exercise sessions . PostersMost clinicians were confident, motivated, and freely volunteered for the course. GLA:D® Back implementation was smooth, particularly for those who previously implemented systems associated with the GLA:D® hip and knee program, i.e., had dedicated floor space and established referral networks. Clinicians observed several patient benefits, including clinical performance test improvements at follow-up, new knowledge gained and learning to move ‘more freely’. Patients felt more confident handling their back problems despite the ongoing presence of pain and future flare ups, underscoring one of the desired effects of the GLA:D® Back program. These apparent patient beliefs observed quantitatively via the FABQ and behavioural changes observed by clinicians should be further investigated for maintenance over the longer-term. Patients also felt reassured, with clinicians nurturing a safe environment, promoting a friendly, competitive atmosphere in a group setting . As suchSome clinicians encountered difficulties implementing the GLA:D® program, including patient recruitment challenges. This underscores the importance of skilled staff and other resources dedicated to patient recruitment . Clinicians undertaking such roles were likely discouraged to implement the program, being drawn away from their established clinical responsibilities. Greater support provided by the GLA:D® Back team (beyond the training course) may facilitate greater widespread adoption of GLA:D® Back among clinicians.Virtual platforms were not adopted by all patient in our GLA:D® Back study, with some favouring the initial face-to-face approach of the program. Emerging evidence , indicatClinicians’ overall evaluation of the course was positive, reflected by successful adoption of the program. Clinician treatment orientation (beliefs and attitudes) favoured a more biomedical orientation at 3 months follow up, contrasting the GLA:D® Back studies in Denmark , Canada A key limitation of our study implementation was the impact of the COVID-19 pandemic. As a feasibility study, only preliminary short-term evaluation of clinician and patient outcomes took place, hence the maintenance of long-term outcomes is unknown. We also conveniently sampled and recruited clinicians we felt would be interested in and likely adopt the program. Although evidence and guidelines support the approach and intervention taken with GLA:D® Back, no clinical trial has evaluated efficacy of the program compared to usual care. The small sample in our feasibility study signifies a need to evaluate implementation with a larger sample of clinicians and patients prior to a national roll out of the program. Implementation of our study was limited by the in-person training of clinicians, i.e., limited by cost and location, and there is a need to train local trainers to deliver the program. The feasibility of online training was also not assessed. Further, inclusion of other stakeholders in qualitative research is needed, including potential referrers , and funders . We only assessed some aspects of fidelity, with future fidelity assessments targeting focused behavioural change techniques and clinician communication styles. While our secondary aims were pertaining to patient outcomes, it should be noted that improvement trends in these outcomes cannot be taken as evidence for effectiveness. Our feasibility study in Australia builds on the original English implementation of the GLA:D® Back program in Canada , therefoImplementation of the GLA:D® Back program in Australia appears feasible. The program was acceptable to the small sample of invited chiropractors and physiotherapists and with potential improvement in patient outcomes. However, the COVID-19 pandemic impacted patient recruitment, retention, and data collection. To scale up GLA:D® Back implementation in private and public settings, further work is warranted to address associated barriers, and to leverage facilitators. Also, clinical benefits of the program when compared to current models of care needs to be determined.Additional file 1. Appendix Table S1. Key messages in GLA:D® Back used during patient education and repeated during the exercise sessions to encourage self-management.Additional file 2. Appendix Table S2. Overview of clinician and patient outcome measures.Additional file 3. Appendix Table S3. Fidelity elements of the clinical intervention to be observed.


Obesity underlies much chronic disease. Digitalization of obesity management provides an opportunity to innovate our traditional model of health care delivery within this setting, and to transform its scalability potentially to the population level.The objective was to assess the feasibility and effectiveness of the Low Carb Program app for weight loss, applied within our hospital-based (tier 3) obesity service. Due to the disrupting effects of the COVID-19 pandemic on our obesity service, we compared the clinical outcomes from the Low Carb Program app applied in the context of remote patient appointments over the telephone with the prepandemic traditional standard of care.1c as a marker of glycemic control. We compared data from the Low Carb Program group with a retrospective control group (n=126) that had received traditional face-to-face obesity management from our team without concomitant use of the Low Carb Program app in the pre–COVID-19 era. T test comparisons were employed, with P<.05 considered significant.We invited patients who attended our hospital-based obesity service to engage with the Low Carb Program smartphone app. We combined this approach with remote delivery (over the telephone) of obesity management from medical and psychology members of our obesity team during the COVID-19 pandemic. Outcome variables included changes in body weight and changes in HbA1c. Paired sample t test analysis revealed a statistically significant mean loss of body weight of 2.7 kg (P=.001) and improvement in HbA1c of 3.3 mmol/mol (P=.01). The mean weight of control group patients (n=126) was 137.1 kg, with 74% (93/126) females and a mean age of 44.4 years. The mean follow-up for this group was 6 months. Data comparisons between the app user group and the pre–COVID-19 retrospective control group revealed equivalence for loss of body weight and change in HbA1c between the two groups.The mean weight of participants (n=105) was 130.2 kg, with 59% (n=62) females and a mean age of 48.8 years. Most participants  completed the Low Carb Program app registration process and engaged with the Low Carb Program app program; at follow-up, most participants  had actively engaged with the Low Carb Program app within the prior 30 days. The majority of participants  self-reported outcomes within the app. Mean duration of clinical follow-up for recruited participants who received the app was 7.4 months. Paired data were available for 48 participants for body weight and 41 participants for HbAWe provide evidence to support the feasibility of implementing the Low Carb Program app combined with remote management; this is the first proof of concept for digitalized management within a hospital-based (tier 3) obesity service. We demonstrate the potential clinical efficacy of the approach in terms of improvements in body weight and glycemic control. In the United Kingdom, 27% of men and 30% of women live with obesity  and have2 or BMI>35 kg/m2 with obesity-related comorbidities), there is hospital-based obesity management (tier 3), which usually consists of a multidisciplinary team, with specialist dieticians, medical staff, and psychological support. Bariatric surgery (tier 4) sits at the top of the structure, and as such represents a restricted resource (despite representing an excellent treatment choice for obesity) available to a very small proportion of the obese population who are potentially eligible for this procedure [Within the UK-based National Health Service (NHS), a tiered framework underlies the pyramidal structure of obesity management, in which progressively restricted resources and numbers of patients typifies upward tier progression. Obesity management within a primary care setting (tier 1) consists of promoting healthy lifestyles, education, and preventive strategies. Community-based services (tier 2) usually comprise dietary advice and optimized physical activity. For people with more severe obesity  is restricted, and as such does not represent a scalable model applicable to a population level . Further2 -9. Thus,2) in a retrospective observational analysis [1c reduction of 0.3%-0.5%) [The NHS Long Term Plan has set a goal of developing digital interventions to improve patient outcomes and effectiveness of care . Howeveranalysis . The proanalysis . Unfortuanalysis . Apps foanalysis  and the analysis . Edson aanalysis  evaluate3%-0.5%) .1c reduction of 13 mmol/mol) and remission of T2D in 26% of the participants, sustained at 1 year [The Low Carb Program is a multiplatform, NHS-approved digital health intervention that supports weight loss through the provision of therapeutic carbohydrate restriction, intended for people living with obesity, prediabetes, or T2D . The appt 1 year . HoweverThe aim of our study was to assess the feasibility and effectiveness of the Low Carb Program app for weight loss, applied within our hospital-based (tier 3) obesity service. Due to the disrupting effects of the COVID-19 pandemic on our obesity service, we also compared the clinical outcomes from the Low Carb Program app applied in the context of remote patient appointments over the telephone with the prepandemic traditional standard of care.Access to the Low Carb Program app was offered to all new patients at their initial medical consultation after being referred to our hospital-based (tier 3) obesity service at UHCW, United Kingdom, over a 9-month period between January 2020 and September 2020. The initial contact was with a medical doctor. Patients were informed about the app and those who wanted to use the app were given instructions on how to download it. We recruited those patients who were interested in using the app (n=105), and provided each of them with a unique code that enabled activation of the app free of charge when downloaded from the NHS App Library. The only exclusion criterion included inability to understand English. Due to the observational nature of our study, and its innovation as a strategy to improve clinical care, no formal approvals from research ethics or institutional review board committees were required. We registered our study as a service evaluation with the Research and Development team at UHCW. We integrated provision of the Low Carb Program app to patients as part of their obesity management. As such, it was not possible to randomize this intervention. Aligned with their clinical care, participants did not receive any payment for their participation in our study. Participants provided their consent for use of their anonymized self-reported data from the Low Carb Program app for research purposes. Inclusion in the study and overall engagement with the Low Carb Program app did not affect patients’ clinical management.The Low Carb Program app is an NHS-approved medical device  that provides a 12-week digital health intervention with personalized and structured content for adults diagnosed with T2D, prediabetes, and obesity . The LowFor the purposes of our study, we used a streamlined version of the Low Carb Program app, specifically tailored to people living with obesity. In the first 2 weeks of the Low Carb Program app program, participant users receive education on the physiology of obesity and the role of dietary modification in its management. This includes a description of how a low-carbohydrate diet can improve the management of postprandial blood glucose levels (in T2D) and body weight. The subsequent Low Carb Program app modules explore strategies to reduce dietary sources of sugar, including foods with a high starch content, such as bread, pasta, potatoes, and rice, and foods that contain refined sugar. Participants are encouraged to make portion control and carbohydrate restriction decisions based on visual plate representations. In place of carbohydrate-rich foods, the Low Carb Program app modules advocate an increased intake of green vegetables, low glycemic index fruits  and healthy fats . Participants could access the tools supplied within the Low Carb Program app, including educational modules, and were encouraged to track their body weight and other measures of overall health, and access support from the community discussion board. Participants also had access to a searchable library of recipes tailored to dietary preferences and allergies. Digitally excluded patients are supported with a physical “starter pack” containing a welcome booklet, educational modules and resources, and cookbook composed of recipes from the program. Digitally excluded patients are encouraged to join weekly coaching and meetup sessions that can be attended by telephone. We provide a list of the weekly educational topics for the Low Carb Program app in Each recruited participant had ongoing clinical input and follow-up with members of our hospital-based (tier 3) Obesity management team as part of usual care throughout the study period. Although conceived and commenced within the pre–COVID-19 era (January 2020), there was substantial overlap of much of our study with the COVID-19 pandemic . As a result, no patient in the tier 3 weight management service received specialist dietary input from March 2020 onward. The clinical follow-up varied between patients but most received telephone review by a doctor 6 months after the previous appointment. However, the Low Carb Program app supported each participant with invited virtual meetups every Monday to provide an opportunity for social connection with other users during the “lockdown” period in the United Kingdom. We conducted the virtual meetups through coach-led video conferencing sessions that provided an informal space for the sharing of personal experiences and establishment of peer support networks.α and power 1–β is given by the following expression [The total sample size required to detect an expected standardized difference Δ at two-sided significance level pression :α/21– and Zβ1– are percentage points of the normal distribution, which for 5% significance and 80% power are given by Zα/21–=1.96 and Zβ1–=0.84 with standardized difference Δ=(dt–d0)/sd, whereby sd is the standard deviation of the difference, and dt and d0 are a group mean at time t and baseline, respectively. An approximate expression is given by N ≈ 2+8/Δ2 for 80% power at the 5% level.Where Zt. A large cohort study on outcomes of a specialist weight management program in the United Kingdom found that the mean weight loss of 1838 patients was just under 1 kg at 3 months [1c was not done.The study was planned based on a clinically significant expected change in mean weight (weight loss) of 1 kg at time point 3 months . The sta1c  and user engagement with the Low Carb Program app. Quantile-quantile (Q-Q) plots were used to test the assumption that data on body weight were normally distributed. Data on HbA1c and age were assumed to be normally distributed. We compared baseline and follow-up data from the Low Carb Program app users through the paired Student t test, with a P value of <.05 considered significant. For comparison of the efficacy of the Low Carb Program app to the pre–COVID-19 pandemic standard of care, we obtained retrospective control data collected between 2016 and 2019 from a random selection of patients (n=126) attending our UHCW-based obesity service during that time. Comparison of changes in body weight and HbA1c between the retrospective control group and the Low Carb Program app user group were compared using the independent Student t test, with a P value of <.05 considered significant. We obtained data regarding user engagement with the Low Carb Program app (including Low Carb Program app activation and duration of use) from Diabetes Digital Media. Chi-square test was used to compare baseline data (gender and percentage of patients with diabetes mellitus) from the intervention group to the retrospective control group.Statistical analyses were performed using SPSS . The primary outcome variables were change in body weight and HbA1c of 48.0 (SD 15.5) mmol/mol. The majority of participants  were female. Overall, a minority of participants  had diabetes mellitus . Baseline weight and HbA1c data were missing from 2 Low Carb Program app users. The baseline phenotype of Low Carb Program app users was broadly similar to the retrospective control group in terms of baseline weight, proportion of patients with diabetes, and baseline HbA1c, but was significantly different in terms of age and gender (We provide data on baseline characteristics of the Low Carb Program app users (n=105) in d gender .1c and body weight was 5 months. Half of all participants engaged with the Low Carb Program app between 3 and 7 months (range 1-12 months), as shown in Of the recruited participants (n=105), all enrolled onto the Low Carb Program app. Overall, 90 of the 105 participants (86%) completed the Low Carb Program app registration process and engaged with the Low Carb Program app program. A total of 88 participants (84%) actively engaged with the Low Carb Program app within the previous 30 days. Only a minority of participants  completed the entire Low Carb Program app program (defined as completing ≥9 of the 12 education modules available). A total of 58 of the 105 recruited participants (55%) self-reported outcomes from the Low Carb Program app. Of the 47 participants who did not self-report outcomes, 29 participants (62%) were actively using the Low Carb Program app at follow-up assessments. The mean duration between baseline (registration) and follow-up check-in to self-report HbA1c. Paired sample t test analysis revealed a statistically significant mean loss of body weight of 2.7 kg (P=.001) and improvement in HbA1c of 3.3 mmol/mol (P=.01), summarized in Paired data were available from 48 Low Carb Program app users for body weight and 41 Low Carb Program app users for HbANo adverse events were reported throughout the duration of this study.1c between the two groups  revealed similar loss of body weight and change in HbAo groups . The meaThis study demonstrated the feasibility of using the Low Carb Program app as part of a remotely delivered obesity multidisciplinary team during the COVID-19 pandemic era, as well as efficacy regarding both loss of body weight and improvement in glycemic control over a 7-month period.Our data confirm equivalence of body weight reduction resulting from remote obesity management complemented by digitally enabled support through the Low Carb Program app versus traditional face-to-face hospital-based obesity management implemented during the pre–COVID-19 era. This is a novel insight that has important implications for the future delivery and hybridized digitalization of hospital-based obesity services. Our data corroborate a recent meta-analysis on mobile app interventions, showing similar reductions in body weight and improvements in glycemic control . The cli1c by between 5 and 10 mmol/mol. In our study, we observed an average reduction of 3 mmol/mol, which is almost on par with some oral therapies for T2D management. The importance of good glycemia control was highlighted in the UK Prospective Diabetes Study, as it reduces the risk of developing diabetes complications [1c level.Most oral therapies for glycemic control in patients with T2D only lower HbAications . The imp1c observed with the Low Carb Program app and remote management to any dietetic input. Conversely, these clinical outcomes stemmed from remote medical and psychological input combined with patient engagement with the Low Carb Program app .It is important to highlight the serendipitous nature of our study, with its conception and initiation prior to the emergence of COVID-19 onto the world stage. Although originally designed as a means to complement the traditional standard of face-to-face, multidisciplinary management of obesity delivered within a hospital-based setting, use of the Low Carb Program app instead complemented the remote delivery of obesity management delivered over the telephone by a diminished team (without focused dietetic support), due to the obstructive effects of the COVID-19 pandemic. Thus, the execution of our study morphed out of a necessity to adapt and align our clinical practice in response to a global pandemic, in combination with a study design in which COVID-19 did not feature. In retrospect, the Low Carb Program app seems more apt to complement a remote care model than one of a traditional (pre–COVID-19) standard of care. Furthermore, during the COVID-19 pandemic, none of our patients received any dietetic support due to staff redeployment at UHCW. Therefore, we cannot attribute the changes in body weight and HbAThe Low Carb Program app had a high sign-up rate, with all participants signing up , and a high engagement rate at follow-up . Overall, 15 of the 105 participants (14.3%) did not complete the registration process and 47 of the 105 participants (44.8%) did not report outcomes once they had activated their registration. However, much of this group  were actively using the app at follow-up. These results suggest that the intervention requires adaptations to fully engage patients diagnosed with obesity and that other features of the app may be more engaging than health tracking.The emergence of digital health care interventions that complement conventional health care delivery  provides an opportunity to personalize the treatment and care provided to patients ,25. The Furthermore, the integrated use of digital health care technology within established clinical pathways aligns with NHS priorities, such as supporting the prevention of obesity-related disease and the transition toward digitally enabled health care . The curOur study had several limitations. Due to the observational nature of our study, and its design as a clinical innovation, there was no randomization and no inclusion of a control (placebo) group for direct comparison with those participants who engaged with the Low Carb Program app in combination with remotely delivered obesity management. Although it is possible that the clinical benefits of this combination stemmed solely from the remote interactions with members of the obesity team, this seems unlikely given the notable absence of any focused support from specialist dieticians. A much more likely scenario is that engagement with the Low Carb Program app helped to complement remote management in patients’ achievements of clinical outcomes. Furthermore, although we did not include a control group, we did make comparisons retrospectively with a group of patients who had received a traditional standard of care pre–COVID-19.As this study assessed the feasibility of the app, we did not collect information on medication changes that future studies should capture. Change in glycemic therapy could be a confounder, given the effects of SGLT2 inhibitors and GLP1 analogues on body weight. Additionally, data on BMI was not available for all participants and therefore we did not include it as part of this pilot study.A further limitation included a lack of data collection on all the patients originally invited to use the Low Carb Program app. These data would have been useful to understand why some patients declined to register with the Low Carb Program app. Due to the impact of the COVID-19 pandemic and the requisite remote management paradigm, we were not able to measure participants’ body weight using the weighing scales in our clinical setting. Rather, all participants self-measured and self-reported their body weight measurements throughout. This may have introduced some inaccuracy, and we were not able to verify self-reported body weights.Another potential confounder includes the unusual scenario of the national “lockdown” within the United Kingdom due to the COVID-19 pandemic that coincided with much of the study period. However, given the documented propensity for weight gain during the lockdown and self-quarantine period -34, thisPatient engagement with the app at follow-up was high. This may be because patients received the app at no cost to them. Although this may impact engagement with the app, in the United Kingdom, costs of therapy are borne by the state health care provider and given to patients for free, which reflects the real-world nature of this study. Future studies should evaluate the impact of the app based on the referral mechanism to compare whether paying for access to the app or receiving it for free affects app usage and outcomes.We provide the first evidence to support the feasibility of implementing the Low Carb Program app combined with remote obesity management, providing the first proof of concept for digitalized management within a hospital-based (tier 3) obesity service. We demonstrate the clinical efficacy of such an approach, both regarding loss of body weight and improvement in glycemic control. Future studies should explore how to adapt the Low Carb Program app to populations seen within the clinical obesity setting to improve user engagement and long-term outcomes. Ultimately, the Low Carb Program app represents a management option that is potentially both accessible and scalable at the population level. Furthermore, from a preventive perspective, the Low Carb Program app has relevance for the general population, regardless of obesity status. A healthy lifestyle is important for all of us. After all, an ounce of prevention is worth a pound of cure.


Public health professionals are expected to use the best available research and contextual evidence to inform decision-making. The National Collaborating Centre for Methods and Tools developed, implemented, and evaluated a Knowledge Broker mentoring program aimed at facilitating organization-wide evidence-informed decision-making in ten public health units in Ontario, Canada. The purpose of this study was to pragmatically assess the impact of the program.A convergent mixed methods design was used to interpret quantitative results in the context of the qualitative findings. A goal-setting exercise was conducted with senior leadership in each organization prior to implementing the program. Achievement of goals was quantified through deductive coding of post-program interviews with participants and management. Interviews analyzed inductively to qualitatively explain progress toward identified goals and identify key factors related to implementation of EIDM within the organization.Organizations met their goals for evidence use to varying degrees. The key themes identified that support an organizational shift to EIDM include definitive plans for participants to share knowledge during and after program completion, embedding evidence into decision-making processes, and supportive leadership with organizational investment of time and resources. The location, setting, or size of health units was not associated with attainment of EIDM goals; small, rural health units were not at a disadvantage compared to larger, urban health units.The Knowledge Broker mentoring program allowed participants to share their learning and support change at their health units. When paired with organizational supports such as supportive leadership and resource investment, this program holds promise as an innovative knowledge translation strategy for organization wide EIDM among public health organizations.The online version contains supplementary material available at 10.1186/s43058-022-00267-5. Contributions to the literatureThis study implemented and evaluated an intensive Knowledge Broker mentoring program to facilitate evidence informed decision-making, using an innovative mixed methods design for deeper analysis than is possible with quantitative methods alone.The findings highlight that the key factors for program success are integration of knowledge brokers into activities across the organization, embedding evidence into decision-making processes, and supportive leadership with organizational investment of time and resources.These findings contribute to the broader literature for successful implementation of knowledge broker roles to facilitate evidence-informed decision-making, and notably explore successful implementation in smaller, rural organizations, where limited success has been achieved.Evidence-informed decision-making (EIDM) in public health integrates the best available evidence from research and the local context to optimize decision-making. Achieving EIDM requires, in part, finding, synthesizing, and applying the best available evidence from research sources, community data and local contexts, societal and political preferences, and available resources –3. EIDM Despite these expectations, barriers to achieving EIDM persist. Public health practitioners do not universally have the knowledge and skills for finding, appraising, and using evidence in practice; some public health organizations are unable or unwilling to support evidence-informed practice –9. StratReviews of interventions to increase research use in public health have found that strategies should be adapted to the needs and priorities of the organization and individuals –19. KnowThis pragmatic study sought to determine the effect of an intensive KB mentoring program delivered by the National Collaborating Centre for Methods and Tools (NCCMT) to facilitate organizational change for EIDM including organizational level use of evidence, perceptions of the value of the KB mentoring program, and public health units’ success or challenges in integrating EIDM.A convergent mixed methods design was used to evaluate two sequential cohorts of the KB mentoring program, with individuals from five public health units participating in each cohort. In the quantitative phase of the study, senior leadership at each health unit participated in a focus group to set goals for research use. Following program completion interviews with participants and their managers were deductively coded to quantify the extent to which the goals were achieved. In the next phase, a descriptive qualitative study  investigAny Canadian public health organization involved in front-line service delivery was eligible to participate in the KB mentoring program. The NCCMT issued an open call through its newsletter and contacted organizations with previously expressed interest in the program. As a result of these efforts, ten public health organizations confirmed their participation in the program over two sequential cohorts. The first cohort ran from January 2015 to December 2016, and the second cohort ran from January 2017 to June 2018.Organizations identified five-to-six staff members to participate in the program by seeking volunteers or strategic selection. Diverse staff positions were eligible to participate including public health nurses, health promoters, dietitians, public health inspectors, librarians, managers, or others.The program’s objectives were to assess and assist public health organizations to develop organizational capacity for EIDM, as well as to build capacity among selected staff to function as “internal” knowledge brokers in evidence-informed practice. The program is described according to the TIDieR (Template for Intervention Description and Replication) Checklist; a completed TIDieR Checklist is included in Additional file The program began with an organizational assessment with the senior leadership team of each health unit. Assessments identified the leadership team’s EIDM goals for the program specifically and the organization generally.Participants took part in a 5-day in-person workshop, followed by a 3-day workshop at 6 months, and a 2-day workshop at 12 months held at McMaster University in Hamilton, ON. A printed course pack containing the course syllabus and copies of the required readings were given to participants prior to the start of the Program. Workshops were held daily from 9 AM to 4 PM, consisting of a small number of didactic lectures, with most time spent in small-group problem-based sessions; content focused on a systematic approach to EIDM . ParticiCharacteristics of each participating health unit and the populations they serve were collected from government websites . ParticiIs Research Working for You?  within their teams” . Some participants were formally assigned with providing orientation and guidance for EIDM. Other participants became informal sources of knowledge and guidance for colleagues, “…[participant] walked our staff through how to apply this approach and so as a result, a brief review was done of the intervention and literature related to it.” .At four health units, participants provided formal workshops to the broader organization. One health unit had participants deliver a series of presentations on EIDM, while three others implemented monthly journal clubs where participants led appraisals of journal articles with colleagues.Overall, management and peers recognized program participants as resources for EIDM. Various strategies allowed participants to share their expertise, reinforcing participants’ learning and developing other staff’s skills.Some participants felt the program curriculum did not directly or easily transfer to their work. A participant noted little room to apply EIDM, “our programs are very clearly scripted about what we have to do, and how and why.” .“When you look at the model about [EIDM] we focused a lot on the research bubble. And so things like the political climate and resources, those other bubbles that are in that model weren’t really touched on but yet I know from my own job that they’re extremely important.” Others noted that the program’s focus on research evidence did not equip them for applying other types of evidence,These criticisms of the program curriculum related to its applicability to participants’ work and whether the program adequately prepared participants to apply EIDM in different contexts.Most health units encountered staff turnover during the program. At one health unit, half of the participants had transitioned to new roles by the time the interviews were conducted. It was noted that participants were typically high achievers, which led to their promotion to new roles, “what happened was probably partly because the individuals we selected were high performers and were leaders in and of their own right, but over time they went on to other positions” . Other participants left roles for personal reasons. Loss of these participants hindered implementation of EIDM, especially as they had become champions for EIDM.Health units prioritized using evidence more consistently in decision-making. Most sought to assess and adapt research evidence more often while fewer prioritized considering quality of evidence, systematically integrating evidence use and holding management accountable for research use.Thematic analysis uncovered that some health units concretely integrated EIDM into decision-making processes by developing structures, processes or templates to support EIDM, creating or adapting staff positions for EIDM and pursuing additional learning opportunities.Six health units described embedding EIDM into reporting templates or processes to prompt providing an evidence base for program planning recommendations. One health unit revised its briefing note template, “We’ve got a template and a process for doing briefing notes [that includes] what level of literature research are you using” . Some health units integrated resources supporting EIDM, such as guidelines and tools, into various processes. Others described guides or resource hubs to support all staff at the organization to engage in EIDM. A resource library was shared through a health unit’s internal network to help staff apply EIDM in program planning. These various strategies helped reinforce and remind staff to use EIDM in all processes.Three health units noted that roles had changed to integrate EIDM work, “because with the training we were given dedicated time, dedicated FTE [full-time equivalent], all of us, and I think the dedicated FTE really helped us” .At two health units, additional staff were hired in specialist roles for EIDM support. On the growing number of research analysts at a health unit, “when I first started, I was one of the first [research analysts] and there were only about three at the time, and now I think there's four or five of them” . A dedicated specialist role had been created and filled at another health unit, to further educate frontline staff and guide them through rapid review processes.Staff noted that dedicated time and roles to EIDM not only facilitated implementation, but it reinforced the prioritization of EIDM at the organization.“We really had to … remind people that we have this training. It wasn’t necessarily, okay, you guys have this training now, we’re going to get you to do this, this and this or this is, kind of, the plan to have it sustainable. It’s, kind of, been us that’s been advocating for the sustainability of it.” Five health units reported specific goals that had not been achieved. Issues included lack of direction for participants,Plans for participants at these health units were absent. Newly acquired knowledge and skills were not applied or shared more broadly within the health unit, which led to frustration from participants.“I think the challenge is always to sort of protect the time of those people so that they’re able to act as sort of a knowledge broker for the team and that they’re not struggling to sort of meet that need while continuing to do 100 percent of their normal job.” Interviewees at six public health units expressed frustration at not dedicating enough time for EIDM practices. Noting that it was difficult to balance staff time to meet requirements while also using evidence,Interviewees noted that decision makers needed to allocate time dedicated to EIDM work to ensure that EIDM remained a priority and the work sustainable. Similar to lacking a concrete plan for participants to apply their EIDM skills, participants were frustrated at not having the opportunity to apply the knowledge and skills gained from the program.“We released our rapid review just as the merger was happening and, you know, we were without phones and fax machines and computers, and the knowledge broker piece was very much lost in that shuffle, and it's just picked up momentum again.” .Some health units faced other factors that limited their focus on EIDM. Three had unexpected large-scale changes, such as in leadership, organizational structure or a merger with another health unit. Changes significantly overshadowed EIDM as a priority, “We’ve had a lot of change in upper leadership… and just because of all that flux and change we haven't really had a chance to really dig in [to integrating EIDM]” . Similarly, merging with another health unit delayed progress with EIDM,While the changes described by interviewees were varied, the effect was consistently a lack of focus on EIDM and lower prioritization of EIDM implementation.Fewer goals were set for staff attitudes and organizational culture. Many health units prioritized expanding work with external partners. Health units commented on varying extents of culture shift. Culture changes included leadership support for EIDM, expectations that decisions would use EIDM, acceptance of time for learning and doing EIDM, and peer learning.“She’s always wanting, you know, to know like where does this come from, you know, and it’s not to kind of, you know, be testing you or whatnot, but it’s really wanting to be grounded in the evidence and that like all the – everything that we do, our practice is to be grounded in evidence.” Six public heath units’ leadership set expectations for the transition to an EIDM approach. Actions of Medical Officers of Health were specifically cited, “EIDM and the use of the knowledge brokers within the office of the Medical Officer of Health [supported] ongoing involvement and leadership from the MOH office in creating a culture of [EIDM]” . Noted of another Medical Officer of Health,Leadership was described as especially impactful for helping generate organization-wide interest in EIDM and an appreciation of its value, beyond participants and their teams.Some health units described a profound shift in expectations for EIDM. Compared to before the program, “I find that staff are more apt to say ‘well, we should look at the research.” . At another public health unit, “ and, supervisors and, managers are providing the time. They're very supportive of giving employees time to learn about this, which is a shift in itself.” Three public health units allocated time specifically for staff to develop skills for EIDM, and managers were accepting and encouraging of this practice.While a lack of protected time to apply EIDM hindered implementation, the provision of time for staff to develop EIDM skills was described as valuable. Acceptance of this time allowed participants and other staff to develop their skills without time pressures.“… there probably still is the tendency for some of the areas to want to find the quick answer … they would be more likely to just search for something that supports [their opinion] as opposed to taking the time to ask it in a more objective way.” Four public health units found staff on teams that did not have a participant from the KB mentoring program embedded in the team were reluctant to change their processes. For some, this was attributed to the time required for EIDM.“But my team – I had people on my team going ‘I’m not listening to her. She’s only been here for like three years!’ … she does have all this great knowledge, but it just doesn’t work that way.” Reluctance to adopt EIDM in some departments was due to biases, specifically to participants’ perceived lack of career experience,In some cases, staff reluctance was directly related to EIDM processes, or to external factors, such as interpersonal relationships.“… engaging a few more at the management level would have resulted in perhaps a few of those glitches that we experienced, like mitigating those a little bit in terms of how we supported more of our frontline staff … and how to best support the team in terms of sharing their learning and building on it more systematically than we were.” Some participants’ immediate supervisors did not appreciate how EIDM would affect day-to-day work and did not set appropriate expectations. Training of management would be beneficial,There was frustration that workloads and time were not managed effectively. While EIDM work was added, there were no other projects removed from portfolios, leading to participants becoming overwhelmed.Interviews provided insight into the successes and challenges faced by program participants in implementing EIDM. Thematic analysis uncovered several major factors for the implementation of EIDM on an organizational level, including building capacity for EIDM, integrating EIDM into processes, and organizational culture.Over two cohorts, ten public health units in Ontario participated in an intensive KB mentoring program. Public health units reported varying degrees in meeting their identified EIDM goals. These results show the real-world attainment of organizational EIDM goals through such a program and the factors that influenced EIDM implementation.Achievement of EIDM goals varied across organizations. Interviewees at some organizations reported culture shifts to widespread adoption of EIDM; others saw limited implementation. Organizations that demonstrated readiness through strong leadership supportive of resource investment and a culture accepting and enthusiastic for change achieved greater success in realizing EIDM goals. Similar readiness factors have been shown to be important factors for EIDM , 28, 44.Smaller, rural public health units were successful in implementing EIDM, demonstrating that size and location of organizations do not limit EIDM practices. Several of the smallest and most remote participating organizations were successful in achieving their EIDM goals, despite their own expectations of limited success due to relatively fewer available resources. As noted by participants, the impact of program participants was perceived to be greater in smaller organizations than in larger ones. Previous EIDM implementation studies in health-related rural and remote settings identified perceived barriers to EIDM, such as gaps in knowledge, unfavorable attitudes, and lack of resources –51. ThisParticipants’ professional roles did not limit contributions to EIDM practices, consistent with findings reported by others , 45. ThiOne limitation of the KB mentoring program was the focus on using research evidence, rather than data from other sources such as local context. Local evidence has been previously found to be more influential than external evidence . FurtherThe study did not include a control group and as such it is not possible to distinguish the program’s effects from changes that may have occurred without program participation. In future iterations, a time series or stepped wedge design, where organizations are observed prior to intervention, may help further determine the program’s specific effects .KB roles can pose challenges, both for the individuals acting as KBs and the organizations they serve. The KB role is not well-defined, so contributions vary greatly in different contexts , 54. SinWhile this study offers new insights into a KB mentoring program to advance EIDM in public health, there are aspects to explore further. Enhanced focus on non-research evidence and implementation should be included in future program iterations. Participation by smaller, rural, or remote organizations should be considered to further explore EIDM implementation in settings typically perceived as lacking capacity. Exploration of EIDM implementation for diverse public health roles will optimize approaches for organization wide EIDM.This KB mentoring program was promising at an individual level, indicated by reported practice changes by program participants. Shifting organizational culture to EIDM is complex, but this program shows potential for effective integration of EIDM within organizations. A KB mentoring program can support organizations to initiate and progress toward identified EIDM goals. Important organizational factors that are key to progress toward EIDM goals include supportive leadership and invested resources.Additional file 1: Appendix 1. The TIDieR (Template for Intervention Description and Replication) checklist.Additional file 2: Appendix 2. Interview structure.