Purpose This study aimed to perform a process evaluation of a participatory supportive return to work program for workers without a (permanent) employment contract, sick-listed due to a common mental disorder. The program consisted of a participatory approach, integrated care and direct placement in a competitive job. Our main questions were: were these components realized in practice and in accordance with the protocol? The evaluation took place alongside a randomized controlled trial. Methods The study population consisted of workers who filed a sickness benefit claim at the Dutch Social Security Agency, professionals of this agency and of vocational rehabilitation agencies. We focused on sick-listed workers and professionals who had actually participated in the intervention. Data was collected mainly by questionnaires. Results Only 36 out of 94 intervention group participants started with the program. In half of these cases application of integrated care was reported. Most other steps in the program were completed. However, fidelity to the protocol was low to reasonable. Much delay was observed in the execution of the program and only two sick-listed workers were placed in a competitive job. Still, satisfaction with the participatory approach was good. Conclusions Despite the positive evaluation of the participatory approach, the full program was executed less successfully compared to similar programs evaluated in earlier studies. This will probably affect the outcomes of our trial. Findings from this study will help to interpret these outcomes. Nevertheless, more knowledge is needed about experiences of stakeholders who participated in the program. Trial Registration NTR3563.The online version of this article (doi:10.1007/s10926-016-9625-6) contains supplementary material, which is available to authorized users. Sick-listed workers without a (permanent) employment contract, such as sick-listed unemployed workers, temporary agency workers and workers with an expired fixed-term employment contract, often face more obstacles for return to work (RTW) compared to sick-listed employees, especially when there is no (longer a) workplace to return to , 2. MentThe participatory supportive RTW program is a complex intervention, consisting of various components and involving different stakeholders. The program combines a participatory approach, in which the sick-listed worker is encouraged to develop an action plan for RTW, direct placement in a competitive job and integrated care. In the absence of an employer, the Dutch Social Security Agency (SSA) is responsible for RTW guidance of sick-listed workers who have no (longer an) employment contract. Different OHC professionals of the SSA were involved in the program. Vocational rehabilitation agencies were contracted in order to support the sick-listed workers in searching for a suitable (competitive) workplace.Because of the complexity of the participatory supportive RTW program, it was important to get insight into the extent to which the program was executed as planned . A proceThe aim of the present study was to evaluate the process of the participatory supportive RTW program. Despite the fact that process evaluations of RTW programs have become more common –14, thisOur main research questions were: which components of the participatory supportive RTW program were realized in practice and to which extent were these components executed according to the protocol? We also evaluated the procedures used to attract sick-listed workers and professionals for participation in the RCT and their reach, perceived barriers and facilitators for RTW and for implementation of the participatory RTW program and satisfaction of the sick-listed workers and professionals who participated in the program.This process evaluation was conducted alongside a RCT on the (cost-)effectiveness of a participatory supportive RTW program for workers without a (permanent) employment contract who were sick-listed due to a CMD, ‘The Co-WORK’ (in Dutch: ‘SamenWERK’) study. This study was approved by the Medical Ethics Committee of the VU University Medical Center and was registered at the Dutch Trial Register  on August 7, 2012 NTR3563). All participants in the Co-WORK study signed informed consent. The study design has been described in detail elsewhere [3. All paBased on the components of a process evaluation defined by Linnan and Steckler , we asseThe study population consisted of workers without a (permanent) employment contract who were sick-listed due to a CMD, OHC professionals of the Dutch SSA and case managers of vocational rehabilitation agencies.Eligible for participation were unemployed workers, temporary agency workers and workers with an expired fixed-term employment contract, who had applied for a sickness benefit at the Dutch SSA. They had been sick-listed between 2 and 14 weeks, with mental health problems as the main reason for their sickness benefit claim. Sick-listed workers could not participate when one or more of the following exclusion criteria was present: (1) not being able to complete questionnaires written in the Dutch language; (2) a conflict with the SSA regarding a sickness benefit claim or a long-term disability claim; (3) the presence of a legal conflict, e.g. an ongoing injury compensation claim; (4) a sickness absence episode due to a CMD within 1 month before the current sickness benefit claim; (5) already having received usual OHC since the start of the current sickness absence period; (6) pregnancy, up until 3 months after delivery; (7) no signed informed consent form and; (8) no intention to RTW before recovery from symptoms. The latter exclusion criterion was based on findings of two earlier studies, which had revealed that sick-listed workers who believe they should be fully recovered before they RTW, require another RTW intervention , 16.All participating OHC professionals were working at an SSA front office and participated in the study within an intervention team. These intervention teams consisted of at least one insurance physician, one labor expert and one RTW coordinator. All teams were trained in the participatory supportive RTW program by the researchers. They also received a syllabus with the intervention protocol and practical schemes.The participating vocational rehabilitation agencies were all certified commercially operating agencies. At each agency one case manager was appointed. These case managers received a detailed instruction for the placement of intervention group participants in a competitive job.In the participatory supportive RTW program, the insurance physician, labor expert and RTW coordinator of the SSA together with the case manager of the vocational rehabilitation agency supported the sick-listed worker in the development of a consensus-based RTW action plan and in his or her search for a suitable job. Active participation by the sick-listed worker in the program was stimulated. The labor expert monitored the development of the RTW action plan and was responsible for a safe environment in which the sick-listed worker should feel free to come up with suggestions for achieving RTW. A summary of the consecutive steps of the program is presented in Table Three months after randomization and allocation to the intervention group, the intervention group participant, the assigned OHC professionals and the case manager of the contracted vocational rehabilitation agency, all received a questionnaire. Participating professionals were asked to indicate which steps of the participatory supportive RTW program had been realized and when. All stakeholders were asked about barriers and facilitators for RTW and for implementation of the program, using a predefined list of possible complicating and facilitating factors, and about their satisfaction with the different components of the program. In addition, participants were asked to evaluate the extent to which they felt that they had been taken seriously by the participating professionals, based on the Patient Satisfaction with Occupational Health Services Questionnaire (PSOHSQ) . ParticiIn addition, written reports were examined, such as the RTW action plans and reports by the vocational rehabilitation agencies. Furthermore, we used data of the baseline questionnaire of the Co-WORK study to give an overview of the characteristics of the intervention group participants at entry into the study . For theWe defined recruitment as the procedures used to attract sick-listed workers, teams of OHC professionals, and vocational rehabilitation agencies for participation in the Co-WORK study. We described these procedures and illustrated the flow of sick-listed workers in the recruitment process.At the level of sick-listed workers, reach was defined as the proportion of the target population that had actually participated in the Co-WORK study, including both intervention and control group participants. The target population consisted of all sick-listed workers who had been approached for participation in the study and had been eligible for participation, based on the in- and exclusion criteria. Reach was also investigated at the level of the OHC teams. Information was registered about the front offices of the Dutch SSA that had been approached for participation in the study and the front offices and teams of OHC professionals that actually had participated in the study.We combined the dose delivered and the dose received in one evaluation component, the dosage. This component was defined as the extent to which the steps of the participatory supportive RTW program had been completed in practice. We determined for each step in the program in how many cases this step had been completed. Only participants who had actually started with the program were included in these analyses.S1 (Online Resource 1), it was possible to calculate an overall fidelity score per participant. In case no information was available about the completion of a certain step in the program, no score was given for this step and also no point was subtracted. We defined a score of 0–9 as low fidelity, a score of 9–15 as reasonable fidelity, and a score of 15 as the highest fidelity. A score of 9 could mean that all steps of the program were realized in practice, but not according to the protocol. Therefore, this score was used to differentiate between low and reasonable fidelity. We counted the number of participants in each of the three fidelity categories. In addition, we calculated a mean overall fidelity score, by adding up all overall scores and by dividing this by the number of participants. Only participants who had actually started with the program, were included in these analyses.At a general level, fidelity was defined as the extent to which the participatory supportive RTW program had been implemented according to the protocol. We registered for each participant, which steps of the program had been completed (two points per step). One point was given for fulfillment of the first two steps in the program, as these steps consisted of usual OHC. One point was subtracted in case a step had been completed, but not according to the protocol. By using this scoring system, illustrated in table To get more insight into the timing of the program in practice, we assessed the duration between the steps of the program in the study and compared this to the maximum duration between these steps according to the protocol.In addition, we assessed the quality of the three basic intervention components in practice, i.e. integrated care, a participatory approach and direct placement in a competitive job. To assess the quality of the integrated care performed (step 2), we registered the number of cases in which the insurance physicians had contacted the healthcare provider(s) of the participant according to the protocol, which was by telephone.To assess the quality of the participatory approach , we evaluated the content of the written RTW action plans. The International Classification of Functioning, Disability and Health (ICF) was used to classify the identified obstacles for RTW described in the RTW action plans. The ICF is a classification system for (problems in) human functioning . It distThe quality of the support by the vocational rehabilitation agencies (step 6) was assessed by determining the mean number of suitable jobs offered to each participant. Moreover, for each participant who had been placed in a workplace, we investigated whether this placement met the prescribed criteria for placement in a suitable competitive job, i.e. an employment contract of at least 3 months resulting in at least 50 percent of the salary of the participant’s last job.We made an overview of frequently reported barriers and facilitators for realization of RTW. We also described how the investments by the different stakeholders had influenced the execution of the program, according to these stakeholders.For each of the three basic intervention components, the most frequently reported experiences by the different stakeholders were described. In addition, it was investigated how satisfied the participants had been with the guidance of the professionals who had participated in the program.Descriptive statistics  and Excel 2010) were used to analyze the data. For the evaluation of obstacles for RTW, we developed a coding system. Each component of the ICF model was given a different color. These colors were used to code the obstacles for RTW that were written in the RTW action plans. The coding of obstacles was done by the first author and repeated by a research assistant. Disagreements were discussed in order to achieve consensus.Table The boards of nine front offices of the Dutch SSA, were approached by the researchers for participation in the Co-WORK study. Each office was asked to form two intervention teams, of which one could serve as a back-up in the situation that the other team was (temporarily) not able to participate in the program. In most cases, the manager invited two existing teams of OHC professionals to participate in the study. In case one of these teams was not willing to participate, another team was approached.Based on performance indicators, the SSA contracted three commercially operating vocational rehabilitation agencies.Figure Seven out of nine SSA front offices were willing to participate, corresponding to a reach of 78 %. The (perceived) time investment was the main reason for the other offices not to participate. At two offices, only one intervention team was formed. Each team consisted of at least one insurance physician, one labor expert and one RTW coordinator. At the start of Co-WORK, 13 insurance physicians, 12 labor experts and 16 RTW coordinators participated in the study. During the study, one insurance physician, one labor expert and one RTW coordinator were (temporary) replaced by a new professional, because they found a new job/were not willing to participate anymore because of the time investment/were on sickness benefit.Of the total group of 186 participants in the Co-WORK study, 94 participants had been allocated to the intervention group based on randomization. The flow of sick-listed workers in the participatory supportive RTW program is illustrated in Fig. Most steps of the program were completed in many cases, which corresponds to a high dosage. However, the application of an integrated care approach was reported in slightly more than half of the cases. In some cases, information was missing about the execution of a certain step. Information about the application of integrated care was missing in eight cases (step 2), about the inventory of obstacles for RTW between the labor expert and the participant in three cases and between the labor expert and the RTW coordinator in five cases (step 3), about the brainstorm session in four cases and about the creation of a RTW action plan in two cases (step 4) and about the number of workplaces offered in three cases (step 6).In 14 of the 36 cases (39 %) in which the participatory supportive program had been implemented, the fidelity of the application of the program by the intervention providers was low . In the remaining 22 cases (61 %), the fidelity was reasonable . The mean overall fidelity score was 8.9 (SD = 2.2).Table In 13 of the 19 cases (68 %) in which the insurance physician reported that he or she had contacted the participant’s healthcare provider(s), the insurance physician had contacted the healthcare provider(s) by telephone.Eight out of 33 written RTW action plans (24 %) contained at least one description of an activity limitation or participation restriction, such as the inability to cope with high workload, deadlines or complex issues or a restriction in the available working hours. Most of the RTW action plans (n = 27) contained a description of a personal characteristic, without explaining how this characteristic formed a barrier for RTW. Likewise, in some RTW action plans mental health problems were described, without linking this to RTW. Sometimes only a few words were given instead of a description of an obstacle for RTW. In a few cases a solution was described, instead of an obstacle. The most frequently reported obstacles for RTW were “uncertainty or low self-esteem” (n = 12), “trouble concentrating” (n = 8), “mental health problems” (n = 6), “restriction in available working hours” (n = 3) and “worry” (n = 3).Almost all RTW action plans (n = 32) contained at least one solution related to the perceived obstacle(s). In all action plans was described who was responsible for the fulfillment of at least one solution. A timetable was present for at least one of the solutions in 28 action plans (85 %). In 25 action plans (76 %), at least one of the solutions was described clearly.In nine RTW action plans (27 %), both descriptions of suitable work and job examples were given. In 12 RTW action plans (36 %) only descriptions of suitable work were given, such as less demanding work, and in ten action plans (30 %) only examples of a suitable job were listed, e.g. ‘postman’ or ‘mechanic’. In two action plans (6 %) suitable work was not described. Preconditions for work resumption were mentioned in 26 action plans (79 %), e.g. step-wise work resumption and support of a colleague or supervisor at the workplace.On average, each of the participants had been offered three workplaces by the first agency the participant had been referred to. Of the nine workplaces in which participants were placed, only two met the criteria for placement in a suitable workplace.Of the 36 participants who had actually started with the participatory supportive RTW program, 31 had filled out the three-month follow-up questionnaire (86 %). A questionnaire had been filled out by the RTW coordinator in 30 out of 36 cases (83 %), by the insurance physicians in 28 cases (78 %), the labor experts in 27 cases (75 %) and the case manager of the vocational rehabilitation agency in 21 cases (58 %). Sometimes questions could not be answered (yet) at the time of the process evaluation, because execution of the program had been delayed or postponed.The participating professionals often indicated that they did not know whether a certain factor had hampered or facilitated realization of RTW. However, the content of the program was mostly seen as facilitating. To illustrate, in most cases the insurance physician (75 % of the cases), labor expert (93 %), RTW coordinator (63 %) and case manager (57 %), indicated that the development of a RTW action plan had facilitated RTW. Of the participants 55 % indicated that this had facilitated RTW. Also many of them could not tell whether this had been facilitating. This was also true for the job search by the vocational rehabilitation agencies and by themselves.Many times the insurance physician (43 % of the cases), the labor expert (70 %), the RTW coordinator (57 %) and the case manager (76 %) indicated that their time investment in the program had facilitated a successful execution of the program. This item was also often evaluated as ‘neutral’.In more than half of the cases (53 %) in which the insurance physicians reported that they had contacted the participant’s healthcare provider(s), the insurance physicians evaluated the attitude of the healthcare provider as active and cooperative. Often they were also positive about the communication with the healthcare provider(s) (63 % of the cases), and with the degree of agreement that had been reached (53 %). Twenty-one participants reported that they had consulted the insurance physician. Of them about one-third had indicated that their insurance physician was sufficiently aware of the treatment by the general practitioner (GP) or psychologist. Also many of these items were evaluated as ‘neutral’ or ‘not applicable’.In many cases the labor expert was positive about the contribution of the participant to the identification of obstacles for RTW (96 % of the cases), the development of solutions to overcome these obstacles (74 %) and the discussion of suitable workplaces (78 %). Often the labor expert also thought that the RTW coordinator had contributed largely to the identification of obstacles for RTW (93 % of the cases), the development of solutions to overcome these obstacles (85 %), and the discussion of suitable workplaces (82 %). Moreover, the labor experts very frequently reported that the participant and the RTW coordinator had reached consensus about solutions (96 % of the cases) and suitable work (93 %). Twenty-three participants indicated that they had visited a labor expert, and the majority (74 %) reported that the labor expert had contributed largely to a sense of security or support and to the perceived equality between the participant and the RTW coordinator (78 %).The case managers of the vocational rehabilitation agencies were more often dissatisfied (24 % of the cases) than satisfied (19 % of the cases) with placement of the sick-listed worker in a suitable job. Also participants were more frequently dissatisfied (36 % of the participants) than satisfied (10 %) with the job offer by the vocational rehabilitation agency. The number of cases in which the RTW coordinator positively evaluated the offering of a suitable job by the agency was equal to the number of cases in which dissatisfaction was expressed (about 30 % of the cases). In the remaining cases these items were evaluated as neutral or not applicable.S2 (Online Resource 2) shows how the participants had evaluated the guidance of the OHC professionals who had participated in the participatory supportive RTW program. In table S3 (Online Resource 3) is presented how the participants generally had appreciated the guidance by all professionals who had participated in the program. Overall, satisfaction was good. However, also many items were evaluated as ‘neutral’ or ‘not applicable’.Table The aim of this study was to conduct a process evaluation of a participatory supportive RTW program for workers without a (permanent) employment contract who were sick-listed due to a CMD, alongside the Co-WORK study. The process evaluation revealed that only a small part of all intervention group participants had actually participated in the program. In these cases, the dosage of the program was high. However, the application of an integrated care approach had been reported in only half of the cases. Moreover, fidelity to the program was low to reasonable. This poor fidelity was mainly the result of a delay in the execution of the program and a low number of placements in a suitable competitive job. Nevertheless, most of the stakeholders were satisfied with the use of the participatory approach, which was the core of the participatory supportive RTW program.Earlier studies have demonstrated good feasibility of similar participatory RTW programs for sick-listed employees with low back pain, employees with distress and sick-listed unemployed and temporary agency workers with musculoskeletal disorders –14. Our Although the program was aimed at a large group of sick-listed workers, in our trial the program seemed to be suitable for only a small group, i.e. those whose sickness benefit was not likely to end in the near future and who had no contra-indication for participation in the program. The percentage of participants with a medical contra-indication in our study (28 %) was much higher compared to the percentage in an earlier study by Van Beurden et al.  on a simTo our knowledge, this was the first time that direct placement in a competitive job was added to a participatory approach in order to improve RTW of sick-listed workers. Unfortunately, only two sick-listed workers were actually placed in a suitable competitive job by the contracted vocational rehabilitation agencies. Although the support of the vocational rehabilitation agencies was possibly still ongoing at the time of the process evaluation, the number of placements in a competitive job was very low. Moreover, very few sick-listed workers were satisfied with the support by these agencies. This could be a result of a lack of support, but also external factors could have played a role. In the Netherlands, between 2013 and 2014 there was an economic recession, and employment opportunities were limited , 19. ThiIntegrated care was another intervention component that was added to the original participatory RTW program. Despite the fact that this was part of the protocol, only in half of the cases the insurance physician reported that he or she had contacted the participants’ healthcare provider(s). This is in line with an earlier study by Anema et al. , reportiCompliance to the main intervention component, the participatory approach, was also lower compared to the application of such an approach in earlier studies –14. In mDespite the often unclear descriptions of obstacles for RTW, most action plans did contain at least one practical solution to overcome these obstacles and clear descriptions or examples of suitable work were given. Moreover, in most cases both the participant and the professionals involved were positive about the way the RTW action plan had been developed, and they all thought this plan would facilitate RTW. The majority of the participants were also satisfied with the coordination by their labor expert, which is in accordance with the high satisfaction with process guidance found in the study of Van Beurden et al. . PossiblAn important strength of this process evaluation is that all stakeholders were consulted. This made it possible to integrate experiences of stakeholders with various interests in the OHC field. Consequently, the evaluation of a process evaluation component was seldom based on perceptions of only one stakeholder.Another strength of our study is that we used a well-known framework to structure our evaluation. The framework of Linnan and Steckler  helped uIn this evaluation we distinguished between the three basic components of the participatory supportive RTW program, i.e. integrated care, a participatory approach and direct placement in a competitive job. This enabled us to differentiate between those components of the program that can successfully be implemented in daily practice and those components that still need some improvements. However, by making this distinction we ignored the fact that a complex intervention is more than only a sum of the parts . Also thBecause of the study design, we were not able to disentangle the reach of the participatory supportive RTW program from the study’s reach, which is a second limitation of our study. Furthermore, it was not possible to determine whether those who did not respond to the invitation for the study would have been eligible to participate, as they were not screened. Possibly, they were not  belonging to the target population as was assumed in the calculation of the reach.Also the recruitment procedures were related to the design of the RCT. Because allocation to the intervention program was based on randomization, it was important that the sick-listed worker was willing to participate in both the intervention and the control group. This process evaluation does not reveal how sick-listed workers can be encouraged to participate in the intervention program.Another limitation of our study is that mainly questionnaires were used for our data collection. This quantitative research method seemed insufficient to gather data about experiences and satisfaction with the program and about barriers and facilitators for realization of RTW and for implementation of the program in practice. Many of the items to measure these constructs were evaluated as ‘neutral’ or ‘not applicable’.A last limitation of our study is that probably only sick-listed workers and professionals interested in the Co-WORK study, participated in the participatory supportive RTW program. This may have resulted in selection bias. For this reason, generalizing the results of this study to another context could be difficult.Despite the positive evaluation of the participatory approach, it is likely that the low compliance measured in this evaluation will affect the outcomes of our trial. The results of this process evaluation will assist us in the interpretation of the effectiveness evaluation of the participatory supportive RTW program. Nevertheless, new research questions have emerged. Further research could investigate the function of the participatory approach according to the stakeholders who participated in the program; perceived barriers for a successful application of integrated care and direct placement in a competitive job; reasons behind the high number of cases in which there was a contra-indication for participation in the program and reasons for delay in the execution of the program. In this way, more in-depth insight will be obtained about the execution of the full program in our trial. This will be helpful in both the interpretation of the trial results and the decision for future implementation of the program. The use of qualitative research methods seem to be most appropriate to address these topics for further research and to unravel processes of implementation and change .Supplementary material 1 (DOCX 16 kb)Supplementary material 2 (DOCX 17 kb)Supplementary material 3 (DOCX 16 kb)Below is the link to the electronic supplementary material.


In Canada, measles was eliminated in 1998 and rubella in 2000. Effective measles and rubella surveillance is vital in elimination settings, hinging on reliable laboratory methods. However, low-prevalence settings affect the predictive value of laboratory tests. We conducted an analysis to determine the performance of measles and rubella IgM testing in a jurisdiction where both infections are eliminated.21,299 test results were extracted from the Public Health Ontario Laboratories database and 1,239 reports were extracted from the Ontario Integrated Public Health Information System (iPHIS) from 2008 and 2010 for measles and rubella, respectively, to 2014. Deterministic linkage resulted in 658 linked measles records (2009–2014) and 189 linked rubella records (2010–2014). Sixty-six iPHIS measles entries were classified as confirmed cases, of which 53 linked to laboratory data. Five iPHIS rubella entries were classified as confirmed, all linked to IgM results. The positive predictive value was 17.4% for measles and 3.6% for rubella. Sensitivity was 79.2% for measles and 100.0% for rubella. Specificity was 65.7% for measles and 25.8% for rubella.Our study confirms that a positive IgM alone does not confirm a measles case in elimination settings. This has important implications for countries that are working towards measles and rubella elimination. Measles has been eliminated in Canada since 1998 , with spIn Ontario, a single dose of the combined measles-mumps-rubella (MMR) vaccine was implemented in 1975, followed by a two-dose program in 1996 , and is Surveillance of measles and rubella is based on a combination of clinical symptoms and laboratory diagnosis ,11, and In high incidence settings, measles and rubella diagnoses can be made based on clinical data ,13. HoweThe low incidence of measles and rubella in elimination settings presents a challenge for diagnostic serological IgM testing, because the positive predictive value (PPV) of a test depends on the prevalence of disease . Few jurIn Ontario, the Public Health Ontario (PHO) Laboratories perform all IgM serological testing for both measles and rubella. Since October 2009, the PHO Laboratories has been using measles and rubella IgM immunoassays manufactured by Euroimmun AG, Luebeck, Germany. Prior to this, Dade Behring Enzygnost anti-measles virus and anti-rubella virus IgM immunoassays  were used. Evaluation of the Euroimmun platform prior to implementation showed that it performed similarly to the Enzygnost assay. Results for specimens that were submitted for measles and rubella testing were extracted from the PHO Laboratory Information System. Prenatal tests, PCR or viral culture tests, IgG results and specimens that were also vaccine-strain PCR positive  were excluded. Measles and rubella IgM serology results were available as of December 2008 and March 2010, respectively; the latest result available for analysis was November 2014. Test-level results were summarised into patient-level results using a combination of patient name and date of birth. Multiple test results per patient were collapsed using the following hierarchy: reactive > non-reactive > indeterminate. Multiple test results were considered as belonging to the same episode if they occurred within four months.In Ontario, reporting of measles and rubella cases by laboratories and/or physicians to public health authorities is mandatory . EntriesWe conducted deterministic linkage of iPHIS entries with patient-level serology results by first collapsing test-level laboratory results into patient-level results, then linking patient-level laboratory results with iPHIS entries on the basis of name and date of birth. Positive and negative predictive values as well as sensitivity and specificity of a reactive serology result were determined by using a confirmed case of measles or rubella as reported in iPHIS as the gold standard for disease. We chose confirmed cases reported in iPHIS as the gold-standard because all cases entered in the system are subjected to rigorous case investigations that account for clinical, epidemiological and laboratory factors. Cases must meet a stringent case definition to be entered as ‘confirmed’, such as positive laboratory confirmation including virus isolation, PCR, paired IgG serology, or IgM serology with an epidemiological link to a confirmed case or appropriate travel history, or alternatively, clinical symptoms and an epidemiological link to a laboratory confirmed case ,11. All This study received approval from the Public Health Ontario Research Ethics Board. It used already existing diagnostic laboratory testing results from the PHO Laboratories as well as public health information in iPHIS, and therefore participants did not provide written or verbal informed consent to be included. The research relied exclusively on the secondary use of information rendered non-identifiable for all practical purposes. Consent was not required as the conditions of Canada’s Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans 2, Article 5.5B were satisfied.A total of 21,299 measles and rubella laboratory test results were extracted from the PHO laboratory information system . After rA total of 1,239 entries were extracted from iPHIS, each representing a patient investigated for measles or rubella by public health authorities. After removal of duplicates and erroneous entries (n = 13), 1,226 entries remained. Of these, 988 were measles-related and 238 were rubella-related.Laboratory results were linked to iPHIS entries by name and date of birth. For measles, we were able to link a total of 662 entries. These comprised 67.0% (662/988) of iPHIS measles entries and 11.0% (662/6019) of PHOL measles entries. For rubella, we were able to link a total of 189 entries. These comprised 79.4% (189/238) of all iPHIS rubella entries and 1.8%  of laboratory rubella results. Linked measles results from December 2008 were associated with an outbreak and are not representative of the usual setting in Ontario [Linked measles records as of January 2009 (n = 658) had a median age of 5.6 years, older than iPHIS-only entries (median age 5.4 years) and younger than laboratory-only results  . Linked The number and proportion of measles IgM reactive results varied by year depending on the occurrence of known local outbreaks. Linked records had a higher proportion of reactive IgM results, with 243/658 (36.9%) of the linked records having a reactive IgM compared to only 76/5267 (1.4%) of laboratory-only results (p<0.0001). A total of 66 measles entries classified as ‘confirmed’ were entered in iPHIS from 2009 to 2014. Of these, 53/66 (80.3%) were linked to laboratory results, of which 42 were IgM reactive. The remaining 13 iPHIS-only entries (19.7%) between were likely symptomatic cases that were epidemiologically linked, or diagnosed by viral culture only .We saw a similar pattern for rubella. The number and proportion of IgM reactive results differed from year to year, although there was less variation than for measles. Linked records had a higher proportion of reactive IgM results, with 140/189 (74.1%) of the linked entries having a reactive IgM compared to only 142/10,031 (1.4%) laboratory-only results (p<0.0001). There were only five entries classified as ‘confirmed’ cases in iPHIS between 2010 and 2014 (one per year), all of which were linked to IgM reactive laboratory results.The PPV of a reactive IgM for measles during the study period was 17.4% . This inFor rubella, the PPV of a reactive IgM test was 3.6%, increasing to 15.0% with reported fever and 15.4% with reported rash . The tesMeasles and rubella IgM serology is key for both clinical diagnosis and for surveillance purposes, and is a recommended diagnostic by the WHO . HoweverOur analyses of laboratory and public health investigation data, using thoroughly investigated iPHIS data as the gold-standard, showed that the PPV for both measles and rubella IgM testing is low in Ontario. Although this would be expected in a region with a low burden of measles and rubella disease, the degree to which we observed this was striking. The PPV was lower for rubella than for measles, at only 3.6%. This is likely because there were only five confirmed cases reported in iPHIS from 2010–2014, but 10,220 laboratory tests performed. With such high volumes of testing and such low disease prevalence, even a laboratory test with superb performance will falter. Interestingly, the PPV did not increase substantially when clinical symptoms such as fever or rash were taken into consideration, perhaps due to the non-specific nature of these symptoms.Like the PPV, the test sensitivities for both measles and rubella were also mediocre but higher than the specificities, since in jurisdictions with very little disease there is also little opportunity for false negatives. Once again, this was particularly apparent with rubella, which had a test specificity of 25.8%, suggesting that three-quarters of all specimens that tested positive for IgM tests are false-positives.Positive IgM tests for measles or rubella represent one of four scenarios . The firOthers have described the occurrence of false-positive measles and rubella IgM test results, including a case report of a patient who was diagnosed with parvovirus infection after initially testing positive for measles IgM , as wellThere are several limitations to this study. Some characteristics of our setting may limit generalisability, particularly the high numbers of measles and rubella IgM tests are being performed in an elimination setting. We do not have further details on why testing was performed beyond being aware that large numbers of students and healthcare workers are screened and that some physicians may also test patients post-immunisation, despite the fact that IgM serology should never be used to screen otherwise healthy individuals. Physicians sometimes order measles and rubella IgM for any patient with rash. Cross reactivity and false positives with measles, rubella and other viral IgM tests also result in additional repeat testing. Since we excluded other laboratory tests from our analysis, such as IgG serology and PCR results for measles, rubella and other differential tests, and instead used confirmed iPHIS cases as the gold-standard, we are unable to directly validate our reported IgM sensitivity and specificity against other laboratory diagnostic methods. However, since the ‘confirmed’ case definition in iPHIS must meet strict criteria including other laboratory tests performed, we do not think that this affected our results significantly. Our findings underscore the need for technological solutions to improve the information provided by clinicians on their reasons for testing to help laboratories perform the appropriate tests and provide correct interpretation. Of the IgM test results, 319 were measles IgM positive and 282 were rubella IgM positive. However, there were only 53 linked, confirmed cases of measles (of which 42 were IgM reactive) and five linked, confirmed cases of rubella during the study period. It is very unlikely that the IgM positive patients represent true cases. Without further information such as the reason for testing, other laboratory tests performed (i.e.–PCR apart from vaccine-strain positive or IgM testing for other viruses) epidemiological or clinical data we do not know why they tested positive or why they were not reported to public health authorities. The fact that they were not reported supports testing being conducted as part of routine screening of healthy individuals, which should not be occurring but has been described by others . AlternaThe finding that the PPV for measles and rubella IgM is so low is of global importance for other countries and regions that are working towards elimination. Although the concept is familiar ,35, few


Computerized cognitive behavioral therapy for insomnia (CCBT-I) has a growing evidence base as a stand-alone intervention, but it is less clear what factors may limit its acceptability and feasibility when combined with clinical care.The purpose of this study was to explore barriers and facilitators to use of an adjunctive CCBT-I program among depressed patients in a psychiatric clinic by using both quantitative and qualitative approaches.We conducted the qualitative component of the study using face-to-face or telephone interviews with participants who had enrolled in a clinical trial of a CCBT-I program as an adjunctive treatment in a psychiatric clinical setting. In line with the grounded theory approach, we used a semistructured interview guide with new thematic questions being formulated during the transcription and data analysis, as well as being added to the interview schedule. A range of open and closed questions addressing user experience were asked of all study participants who completed the 12-week trial in an online survey.Three themes emerged from the interviews and open questions, consistent with nonadjunctive CCBT-I implementation. Identification with the adjunctive intervention’s target symptom of insomnia and the clinical setting were seen as key reasons to engage initially. Persistence was related to factors to do with the program, its structure, and its content, rather than any nonclinical factors. The survey results showed that only the key active behavioral intervention, sleep restriction, was rated as a major problem by more than 15% of the sample. In this clinical setting, the support of the clinician in completing the unsupported program was highlighted, as was the need for the program and clinical treatment to be coordinated.The use of a normally unsupported CCBT-I program as an adjunctive treatment can be aided by the clinician’s approach. A key behavioral component of the intervention, specific to insomnia treatment, was identified as a major problem for persistence. As such, clinicians need to be aware of when such components are delivered in the program and coordinate their care accordingly, if the use of the program is to be optimized.Australian and New Zealand Clinical Trials Registry ACTRN12612000985886; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=362875&isReview=true (Archived by WebCite at http://www.webcitation.org/6njjhl42X) Depression is one of the most common psychiatric disorders in older adulthood and accounts for much of today’s health care burden placed on medical professionals .The recent development of eHealth technology has led to many efficacious interventions for depression being provided consistently to large numbers of people . ComputeWith rising health care costs, such CCBT programs have been proposed in clinical guidelines to help reduce the cost burden in primary and secondary health care. In 2006, the National Institute for Clinical Excellence   interventions , with evWith the implementation of these adjunctive eHealth treatments into routine clinical practice, emphasis must be placed on process and other logistical concerns. It is less clear what factors may limit acceptability and feasibility of the effects seen in the Internet-only CCBT-I trials when used in clinical practice. Of particular interest are the ways in which patients are able to adhere to intervention techniques provided by the program with respect to timing, dose, and frequency (adherence), and the duration of time in which they are able to actively engage with therapy from initiation until discontinuation (persistence) . QualitaDiagnostic and Statistical Manual of Mental Disorders (DSM-5) criteria for either a major depressive disorder or dysthymia, the Structured Clinical Interview for DSM-IV  SHUTi program?”On completion of the interviews, an external agent transcribed the audio files, and the interviewer cross-checked the transcripts against the audio recordings. In conjunction with observational field notes, the chief investigator (NG) and coinvestigator (CC) then collaboratively coded the interview transcripts for recurrent response features. This was followed by focused thematic analysis, where we defined and pursued a central set of themes based on the prevalence and frequency of codes identified by the primary researcher. Conceptual memos were written from the focused themes to help develop an understanding of the themes and how these related to the data. We used no coding or analysis software for this analysis. We used method triangulation to check consistency.We obtained written informed consent from all participants engaged with the SOMNA trial. Ethics approval for the SOMNA research trial and this qualitative substudy was obtained from the University of Sydney Human Research Ethics Committee, Sydney, Australia. The trial was registered on the Australian and New Zealand Clinical Trial Registry (ACTRN12612000985886).A total of 39 of the 40 (98%) eligible participants who commenced the SHUTi program and remained in the trial (regardless of their level of adherence to the modules) completed the survey. Participants were male and had an average age of 59 years. Of the 39 completers, 29 (74%) were married, 36 (92%) had completed post high school qualifications, 33 (85%) met DSM-5 criteria for a major depressive disorder, and 13 (33%) had self-reported cognitive complaints. Due to the anonymous nature of the interview protocol, we did not ascertain demographic data for the qualitative study participants.Only one aspect of the program was reported as a major problem, and thus potentially a barrier to adherence, by more than 20% of the participants . DifficuWe conducted a total of 4 telephone and 6 face-to-face interviews between September 2014 and February 2015.Three main themes emerged from the qualitative interviews: (1) reasons to commence, and expectations prior to commencing the program related to intrinsic motivators, (2) motivators and barriers related to the program structure and content to continue with the program, and (3) clinical support. The content extracted from the face-to-face interviews regarding the motivators and barriers to continue with the program was triangulated with that collected from the self-report.Most participants reported they could identify themselves on the advertisement, which displayed the study inclusion criteria . The advertisement was presented in various media forms  and was viewed by the participant prior to any involvement or enrollment with the SOMNA clinic. Participants reported being led by advertising material to approach the clinic in the first place, despite many already being in treatment for depression. Some stated they had ignored their insomnia for some time and the advertisement acted as a prompt to do something about it. Linked expectation included the hope that the program would improve their sleep. Some added they had hoped the program could also improve their mood and energy level (interviews 3 and 5). Surprisingly, some participants stated they had no real expectation of the program. Participant 7 said he had tried a few different methods but had not really found much success. He said he participated to “give it a go.” Most participants reported an intrinsic motivation to improve their health holistically. They understood that there was no “quick fix” for their insomnia, and they wanted to understand what the maintaining factors for chronic insomnia were.A total of 2 participants stated that they were interested in being involved in clinical research.I try to be evidence based in my professional work, so I thought, well I will probably get a higher standard of care if I am on a trial than if I just go to some clinic or private practice.Interview 6This was echoed in perceptions that this setting might help where previous approaches had failed—for example, reporting having tried various types of remedies with little benefit. In some cases, this had led to some fears of entering the program, that “nothing works,” and even “anticipating failure” (interview 6).Most participants reported they found the program easy to use and the contents easy to follow. Participant 1 saidI enjoy the visual representation…it feels good looking at the graphs…I look forward to putting them in to see how I went with getting my sleep efficiency and then felt good when I achieved that efficiency.Participant 2 agreed that the program fit well with the “visual person.” Participant 5 said that it suited his goal-directed personality and he found it rewarding to see the results.Most participants reported benefits from psychoeducation on sleep patterns, including understanding different factors that can influence sleep, developing good habits, and challenging cognitive misconceptions about sleep behaviors. Participants reported making “cognitive changes” and letting go of rigid reinforcing beliefs.I learned from the program that it is okay to have minor setbacks…I was able to move on and try again the following night.Participants reported that reinforcement of concepts throughout the modules made it easier to remember them when faced with challenges and insomnia. Most participants found completing a daily sleep diary helpful in making them aware of their sleeping pattern, and also allowing them to go back and review the patterns. This actual behavior and the feedback from the program became self-reinforcing during the program.It becomes a habit for you so it was easier and that became the motivation, it was just something that you got up and did.Interview 5Some identified the behavioral focus early on in creating good habits.The only way that it is going to be of any benefit to me is if I am regimented with, and do exactly, what they have asked me to do and just have faith in what was said, even though I am thinking how do I go to bed at one o’clock?Interview 1I think if in one area of your life you can become disciplined and regimented with it, that can help you to get disciplined in other areas of your life so it can have a flow-on effect.Some of the participants preferred the impersonal nature of using the Web over going through the therapy with a psychologist.The impersonal nature of using the Web was helpful for me. If I had to sit with a psychologist and go through the cores, the experience would be vastly different.The convenience and constant availability were also identified as the practical benefits of the Internet program in the postintervention feedback.On the self-reports, 2 participants suggested using newer graphics and videos. One participant, who worked in a graphic design background, stated “It could look more modern, as it is very conservative, but perhaps the current look is targeted to my demographic or older.” It was also suggested that the program could provide videos produced in Australia (the current videos are produced in the United States).One participant found 1 of the modules to be too long. He said that “there was too much readings and going to next pages and back again. This would have been more beneficial in shorter modules.”Some participants reported difficulties in completing the sleep diaries. One of them stated “the methodology to manually log sleep is crude and unreliable. The watch or other devices are more accurate method and should be integrated with the log.” In the self-reports, 3 participants commented that there should be more flexibility with making entries in the sleep diaries. One of them suggested a daily sleep chart question on daily health and whether it impacted on sleep. Some also reported a sense of failure if they did not complete the modules and diaries.Several recommendations were made regarding the technical aspects of the program. There were 2 participants who commented that the program should have increased compatibility with Apple iPad devices. One of them statedSome sections within each of the cores would not appear on my iPad. The page would have the border etc but would be blank inside, next page and previous page would be fine. This technical issue needs fixing.Other suggestions includeA facility to retrieve or change one’s password is imperative. In addition to using dropdowns you should allow people to enter the information directly, such as typing minutes. When the user logs out the webpage is dimmed, which is not a good way to...A way to jump to any particular part/page directly or with a minimum key strokes.More printouts and more written information.Provide examples or interpretations of the intent behind various questions: eg, “how long did it take you to fall asleep?”—Is this during the sleep window (as the context suggests) or generally?Graphs only depicted in 1-week lots—it would be great to be able to nominate a time period or range of dates and see it all on one graph . More personal stories in video format please, need to see in time.Participant 10 had no access to a computer for a period during the treatment program so he could not respond to reminders, surveys, and emails.A key behavioral component of CBT for insomnia is sleep consolidation. This process involves using the data derived from people’s sleep diaries to identify the optimum period and duration for their individual sleep. Counterintuitively, this usually leads to a period of sleep restriction. Most participants found it challenging to stick with the sleep window, in particular finding it difficult to stay up until the designated sleep time. For example: “I would sit there and I was in no-man’s-land.” Some were concerned about the effects of the fatigue due to the initial small sleep window on their ability to function and work the following day. Some suggested users should be advised to identify a window of time for their period of sleep deprivation. It was also difficult to stay awake well after the bedtime of their partners, something that integration with the clinical aspects may have enhanced.Due to the nature of the double-blind randomized trial, clinicians were unaware whether a participant was allocated to the active computer program intervention or the generic psychoeducation program. It was therefore not possible to integrate the clinical care fully into the computer program intervention in this study.Nevertheless, most participants found the comprehensive psychiatric assessment and management helpful both for themselves and in persisting with the program: “I would not have completed the program without the clinical support” (interview 3). Conversely, 1 participant perceived the clinical setting as a potential barrier because of either the stigma associated with psychiatry (interview 5) or the logistical difficulties of attending a clinic during working hours.Two participants (2 and 10) commented that the clinical and computer program interventions and advice seemed disjointed at times. One commented that there may have been a disconnect between the computer results and the clinical assessment, and the clinicians could have been more in tune to the participants’ input in the Web-based program in regard to sleeping patterns (interview 2).This study aimed to identify the factors influencing adherence to adjunctive CCBT-I in a clinical setting delivering treatment for depression and insomnia. Three themes emerged from the interviews and self-report questionnaires: (1) reasons to commence, and expectations prior to commencing the program, which related to intrinsic motivators, (2) motivators and barriers to continue with the program in relation to the program structure and content, and (3) the interaction of the program with clinical management.Most of the participants who underwent qualitative interviews about the SOMNA trial stated they could identify themselves in the advertisement. Some had previously been unsuccessfully treated for insomnia and depression. Some reported an intrinsic motivation to better their health holistically, which helped this group of participants persist with the program. The belief in recovery and helpfulness of the program as suggested by Barazzone et al  appearedSome participants were attracted to joining the program, as it was conducted within a clinical trial setting, despite seemingly having little expectation or incentive. The reasons, of perceived better quality of care in a university trial setting, were slightly different from the more altruistic reasons for engaging in a trial, that it would benefit others, reported by Donkin and Glozier .It appeared that the program structure of SHUTi had facilitated the participants’ adhering to the Web-based intervention. A majority of participants reported they derived clinical improvements from SHUTi in having a structure to follow. Evidence showed that improving therapeutic relationships would foster a working alliance, which might be associated with proactive information seeking and adherence to eHealth programs . Our parIn our study, psychoeducation (on insomnia) appeared to be regarded well by the participants. Many of the participants reported gaining more knowledge during the program. This was particularly true for those who had ignored their insomnia as a problem for a long time and had little or no previous interventions for their insomnia. Psychoeducation corrected the participants’ misconceptions about sleep behaviors. The results from the survey showed that more than half of the participants found the website improved their insomnia and helped them feel prepared to handle symptoms of insomnia in the future.The visual presentation in SHUTi, such as the graphic interpretations of sleep efficiency, was cited as particularly beneficial for those who had a goal-directed personality. The videos appeared to help participants to relate to others with similar sleep problems. Key concepts were reinforced throughout the modules, and this helped the participants in forming habits, making it easier to persist with the program.The data entry and self-evaluation of sleep efficiency enabled the participants to review their sleep patterns and receive tailored feedback on their progress. The Web-based program also allowed the participants to learn and practice at their own pace. In this way, these appear to provide feedback, responsiveness, and flexibility, as identified by Barazzone et al  as imporIn terms of barriers identified in this trial, aspects of CBT for insomnia in general and program-specific technical issues in particular could have hindered the participants’ adherence to the Web-based program. For instance, many participants highlighted the intrinsic difficulty of sleep restriction as challenging. These challenges are present in face-to-face CBT for insomnia too. In their exploratory study on patients engaging in sleep restriction therapy for insomnia, Kyle et al  reportedSome participants found the modules were too lengthy. The modules in SHUTi typically involve up to an hour of engagement, over a 9-week period . PreviouEvidence from eHealth literature suggested that Web-based intervention might reduce stigma and allow more people seek help for mental health conditions . In our In the qualitative data, the participants did not think the website helped reduce their contact with health professionals. Most participants reported that the regular clinical reviews facilitated their adherence to the Web-based program. This group said the clinical reviews provided them with a sense of accountability to complete the online tasks. They found that the reviews promoted their intrinsic motivation, and some added that the recommended clinical interventions, such as medications and mindfulness techniques, helped them to persist with the Web-based program. This may be why we did not identify some factors that emerged in nonclinical settings that were linked to persistence, such as a sense of “getting things done” and satisfaction at completing the program, as these may have been subsumed by the clinic.The clinical management appeared to aid the CCBT-I by setting goals, planning treatment, and selecting intervention; that is, providing individualized treatment to the individual. This is essential in CBT, and indeed in any treatment. These elements would likely be more effectively achieved by incorporating a clinician into the Web-based intervention. On the other hand, the impersonal nature and constant availability of the Web-based program made attending the program more convenient for participants.As Barazzone et al  suggesteFurther, while most participants found the clinical intervention helpful, others found the clinical and computer program disjointed. This lack of coordination could be explained by the nature of the double-blind randomized trial. However, it highlights that in future commercially available CCBT-I programs need to have some facility to provide treating clinicians with information on where the participant is in the program, what interventions and messages are being delivered at that time, and any clinical progress. Otherwise, clinicians will be unable to integrate clinical intervention with the specific contents in the Web-based programs and potentially disengage from the programs. Clinicians in this study were therefore not aware of which intervention the participants received and were encouraged not to ask specific questions about the Web-based interventions in order to avoid unblinding.There are several limitations to our study. Our results are unique to the targeted population  and the analysis of this research team. Consideration should be given to the variance in results that may have been produced by focusing on a less-specific cohort. For example, studying women or a younger adult population could conceivably yield different findings. However, for the purposes of this study, this is of little consequence to the anticipated outcomes. The data analysis was also influenced by the perspective of the researchers. In this study, the primary author worked as a clinical psychiatrist. This position could have resulted in the development of clinically biased views of what influences adherence behavior in a clinical setting. This, however, may be different from the influences of adherence in a research or online forum. Given this, we attempted to be as objective as possible, but the analysis is likely to have been couched within these experiences.The participants interviewed were people who volunteered to participate in a qualitative interview from a group that volunteered to participate in a trial. This group was therefore probably highly motivated to participate in research and may be different from those who do not wish to share their views. This is reflected in the reported emphasis of research as an important motivator by some interview participants.Due to trial blinding constraints, we contacted participants over a period of months, some participants not until approximately 6 months after they had completed the intervention. This may have affected the ability of some participants to recall their intrinsic motivations for continuing to engage with the trial, and their recall is likely to have been influenced by their present situation. While some of the themes that emerged appear to be values based, and therefore quite static, it is likely that state variables such as time factors and frustration may not be as accurately reflected in these findings. Also, in the self-report data, the participants were not asked about what factors motivated their ongoing participation.We conducted 4 of the 10 interviews over the telephone. While phone interviewing may have allowed respondents to feel relaxed and better able to disclose sensitive information, the absence of visual cues may have resulted in loss of contextual and nonverbal impressions on the interviewer, and may have inadvertently compromised rapport, probing, and interpretation of responses. By using a semistructured, standardized interview schedule, we attempted to consistently present interview questions in the same arrangement and verbal manner with all participants, while simultaneously encouraging elaboration of responses through directive provocation and unlimited time for responses. Further to this, the interviewer adhered to neutral responses  to prevent leading the participant. Any environmental bias was minimized by the implementation of standard interview practice requiring a quiet, comfortable room in which to conduct the interview, and was moderated by any specific requests of the interviewee .Finally, we were only able to recruit participants who had completed the trial, although many had not necessarily completed all the modules; that is, they persisted with the trial but were not completely adherent to the intervention, or were withdrawn, preventing us from exploring the barriers and motivators in people who did not persist with the intervention. We were also not able to recruit anyone who completed the Web-based program but not the 6-month follow-up assessment. Of those who did not persist and did provide additional contact with the researchers, their reasons for dropping out of the trial tended to comprise of a lack of time to complete the program at present, and therefore an unwillingness to participate in a further interview. This, together with other limitations discussed above, might have contributed to an early saturation in our data collection after 10 interviews. This is a relatively small sample size in similar qualitative research. Future research would benefit from interviewing participants who have dropped out of Web-based interventions to determine their reasons for doing so.In terms of maintaining participants in Web-based programs, it may be useful to consider ways to maximize engagement and overcome the barriers that participants mentioned. The findings from this study indicated that the Web-based intervention benefitted those who were motivated, had a goal-directed personality, liked graphic interpretations, and were interested in improving their health holistically. The participants reported a benefit from the psychoeducation and behavioral reinforcement throughout the program in terms of engagement. Clinicians may capitalize on the motivation of this group of participants by providing regular feedback on the data provided and working on goals; for example, designing sleep restriction windows may enhance engagement and increase the likelihood of completion.Other findings from this study suggested that care needs to be taken to ensure that messages and interventions given by the program and the clinician are coordinated if the use of the program is to be optimized. Future studies could further explore ways of integrating clinical input to enhance adherence and optimize treatment. For instance, Cavanagh  suggesteClinicians may need to provide more additional support for those participants who are less motivated and goal directed. It would give these individuals realistic expectations and minimize anticipatory failure. Some participants mentioned online forums to promote social support and share knowledge. This may enhance a sense of community, and thus increase the likelihood of completing the program. Such forums have been successfully used in other programs . ComparaIntegrating automated eHealth interventions into routine clinical practice may improve outcomes; however, the programs and clinicians need to work in a synergistic fashion, sharing information and supporting the patient in their engagement with both aspects of treatment, if we are to realize the full potential of such approaches.