Pulmonary rehabilitation (PR) is an effective intervention for the management of people with chronic respiratory diseases, but the uptake of and adherence to PR programs is low. There is potential for mobile health  to provide an alternative modality for the delivery of PR, overcoming many of the barriers contributing to poor attendance to current services.The objective of this study was to understand the needs, preferences, and priorities of end users for the development of an adaptive mobile PR (mPR) support program.A mixed methods  approach was used to assess the needs, preferences, and priorities of the end users  and key stakeholders . The formative studies included the following: (1) a survey to understand the preferences and priorities of patients for PR and how mobile technology could be used to provide PR support, (2) ethnographic semistructured interviews with patients with chronic respiratory disorders to gain perspectives on their understanding of their health and potential features that could be included in an mPR program, and (3) key informant interviews with health care providers to understand the needs, preferences, and priorities for the development of an mPR support program.Across all formative studies , the participants were positive about the idea of an mPR program but raised concerns related to digital literacy and confidence in using technology, access to technology, and loss of social support currently gained from traditional programs. Key stakeholders highlighted the need for patient safety to be maintained and ensuring appropriate programs for different groups within the population. Finding a balance between ensuring safety and maximizing access was seen to be essential in the success of an mPR program.These formative studies found high interest in mHealth-based PR intervention and detailed the potential for an mPR program to overcome current barriers to accessing traditional PR programs. Key considerations and features were identified, including the importance of technology access and digital literacy being considered in utilizing technology with this population. Chronic obstructive pulmonary disorder (COPD), an umbrella term for a range of debilitating respiratory diseases , is the One of the most effective interventions for COPD is pulmonary rehabilitation (PR), an evidence-based, interdisciplinary intervention that is a key component in the management of people with respiratory diseases . PR is aThere is potential for mobile health  to provide an alternative modality for the delivery of PR, overcoming many of the barriers contributing to poor attendance to current services. By utilizing mobile technology, PR can be made available to people within their everyday lives at times and places most suitable to the patient, removing the barriers associated with transport, timing, and location. There is a wealth of evidence for the use of mobile technology to deliver health interventions to people with chronic conditions including the delivery of rehabilitation interventions, self-management support programs, behavior change interventions, and supportive care -18. Not When developing new mHealth tools, engagement with end users in the design is essential. By incorporating the perspectives of end users, it ensures the intervention will meet the population need and enables it to be tailored to specific cultural needs, contexts, and levels of technology access . When enThis study aimed to understand the needs, preferences, and priorities of end users for the development of a mobile PR (mPR) support program. Specifically, it aimed to understand (1) the preferences and priorities of patients for PR and how mobile technology could be used to provide PR support and (2) the needs, preferences, and priorities of health care providers including physiotherapists, respiratory physicians, primary care nurses, and general practitioners (GPs).This cross-sectional study utilized a mixed method  approach incorporating surveys and interviews to assess the needs, preferences, and priorities of end users and key stakeholders. The study was split into 3 parts—(1) patient survey, (2) patient interviews, and (3) key stakeholder interviews—and was conducted between July and December 2018.Ethics approval was obtained from the NZ Health and Disability Ethics Committee (18/NTA/105). Research approval from each District Health Board (DHB) from which patients were recruited was also obtained. Written informed consent was obtained from survey and interview participants before their participation in the studies.Inclusion criteria were adults with chronic respiratory disease who would be eligible for PR, able to read and understand English, and provide informed consent.Potential participants were identified by clinicians at 2 DHBs in the Auckland region, NZ, through respiratory outpatient clinics and inpatient services. Eligible participants were given a letter about the study, the information sheet, and consent form. By selecting patients from this defined subgroup, we were more likely to define patients who had either been invited to attend or had attended a rehabilitation program. Those interested in participating provided informed consent before completing the survey via 1 of 3 methods: (1) on the web via the study website, (2) over the phone with a member of the research team, or (3) on paper. Surveys completed by phone or on paper were entered into the Web-based version of the survey by the researcher.The survey comprised 4 parts and contained both closed- and open-ended questions to allow participants to elaborate on their answers: (1) your health including diagnosis, attendance at PR, and barriers to attendance and completion of PR; (2) technology access, including current use and access to technology and devices; (3) technology and pulmonary support, including perceptions of mPR, preferences for technology-based support, and perceived barriers and benefits to technology-based pulmonary support; and (4) demographics, specifically age group, gender, and ethnicity.The survey was designed in paper format and then uploaded into REDCap software (v8.5.0). It was pretested by the research team and a selection of patients before finalization.Survey participants who had consented to be contacted for future research following completion of the survey were eligible for inclusion in an interview.Patients who completed part 1 of this study and identified that they were happy to be contacted about further research were contacted to invite them to take part in an interview. Participants provided informed consent to participate as well as consent to access medical records related to their eligibility for PR. Interviews were conducted in person, or over the phone, by a trained interviewer (PH) at a time and location convenient to the participant. If needed, interviews were split over multiple sessions to reduce burden for participants. Notes were taken by the researcher during the interviews, and the interviews were recorded to supplement the notes. At the end of the interview process, the participants were offered a NZ $20.00 (approximately US $13.00 or €11.50) voucher for their time.Ethnographic, semistructured interviews were undertaken. The interviews were designed to explore in-depth beliefs and perceptions of chronic respiratory disorders and their treatments, perspectives on potential features of an mPR intervention, and patient understanding of their health.Inclusion criteria were clinicians managing patients with COPD and other chronic pulmonary disorders , able to read and understand English, and able to provide informed consent.Potential participants were identified through respiratory and PR services by coinvestigators RW, PH, SC, and JR and invited to take part via email. Interviews were conducted in person or via phone at the participant’s preference by a trained interviewer (RD) and were between 30 min and 1 hour in duration. Notes were taken by the interviewer, and the interviews were recorded to supplement the notes.Interviews were semistructured and designed to cover the current use of technology in care of people with chronic respiratory conditions, experiences with PR, and perceptions of mPR including enablers and barriers.Quantitative data from the survey were analyzed and summarized using descriptive quantitative analyses including means, standard deviations, and proportions. Qualitative comments were analyzed using a simple, general inductive thematic approach to identify common themes and meanings from the data. Only completed surveys were included in the analysis. Prioritized ethnicity was used as recommended by the NZ Ministry of Health for the reporting of ethnicity data; only 1 of the ethnic categories nominated by the participant was used according to a predetermined hierarchy .There were 34 entries to the survey site from which 30 people consented to take part and completed the survey. Slightly more than half of the sample were male , and the majority were aged >65 years  reported that they had completed PR or were intending to complete it in the future. Nearly a quarter of participants  had started a PR program but had not completed it. The reasons for not completing the program included issues related to location and transport (n=2), timing of the sessions (n=2), being hospitalized (n=2), and other commitments (n=1). A total of 3 participants  were offered PR but did not attend because of transport (n=2) or other commitments . The remaining 6  participants reported that they had not been offered PR or did not remember being offered PR.Participants were asked about their access to digital devices  for personal use, with all but 4  participants reporting having access to a mobile phone. Of those who had a mobile phone, the majority had a smartphone  and the remainder  a mobile phone without internet capability. Of those that had a smartphone, only 60% (12/20) reported having access to the internet on the smartphone all the time. A total of 5  had access to the internet on the phone sometimes, 2  not at all, and 1  did not answer.One-third of the sample  reported having access to a tablet for personal use, 14  a computer , 2  a Fitbit or other fitness tracking device, and only 1 participant reported access to no devices for personal use. A total of 23  participants reported access to internet at home, and an additional 3  reported they sometimes had access to internet at home. There were 4  that had no access to the internet at home.In relation to the use of technology-based devices and tools to manage health, there was only 1 participant  that reported using an app, 1  who reported using a smart watch, 3  used a fitness tracking device, and 4  a peak flow meter. A total of 20  participants reported no use of these tools to manage their health.Participants were asked about their perceptions of a technology/mobile phone program that would allow people to receive PR support at home. The majority  reported that they liked this idea, and 7  did not like the idea. The proportion of those that liked the idea was highest in Maori participants  compared with NZ European  or Pacific . Those that liked the idea (n=23) were asked what they liked about the idea. The most common theme was around the access at home meaning no need to travel (n=10), and others reported the cost reduction related to parking and travel (n=3), being able to access the program wherever and whenever (n=1), less embarrassment within the home (n=1), and that family would be able to help and be involved (n=1):Privacy—don’t have to be embarrassed.45-54-year-old male, ID number 27Would make it easier as would not have to travel in bad weather.75-84-year-old male, ID number 8The convenience. At the moment, travel outside the home is quite onerous for me.65-74-year-old male, ID number 9Family would be able to help Dad to do this at home.75-84-year-old male, ID number 25Would have saved me some trips... Transport is hard for people, don't have a car.45-54-year-old male, ID number 27There were 5 participants whose responses were moderately supportive of it being a good idea and 3 who did not answer the question. In addition, 3 participants identified concerns around the proposed idea potentially resulting in less social contact and less access to health care professionals:Sounds good although I do like meeting other people.65-74-year-old female, ID number 33But it is good to meet other people and learn from others.65-74-year-old female, ID number 22Those that did not like the idea (n=7) were asked the reason, with the most common theme being that they found technology too difficult/hard (n=3). Other themes included not having internet (n=1), wanting the company/social aspect of an in-person group program (n=1), or preferring to go to the hospital :Too hard to use computer.75-84-year-old male, ID number 18No internet at home.55-64-year-old male, ID number 19I would feel isolated, when you are older you need company.75-84-year-old female, ID number 21Participants were asked about the features they thought would be helpful to include in the proposed program. Of those that liked the proposed idea and who responded to the question (n=22), 19  selected tips and suggestions for managing their breathing, 17  information and education about their condition, 12  access to their health information, 11  tracking of their health data, and 9  motivational and support messages. A total of 23  reported that they would consider wearing a sensor  as part of an mPR program, and 22  reported that they would want the program to be linked into their existing health record and information so that the program could be tailored to their individual health condition and treatment.Participants were asked to identify the barriers to using the proposed mPR program . The mosWhen asked about the perceived benefits of using technology to support people with health conditions in the community, the majority  identified the convenience of having access at any time or anywhere as a benefit. There were 15  participants who identified the benefit of being able to involve/include family/whanau, and 4  identified feeling more comfortable not being in the group or clinical environment.A total of 8 patient interviews were conducted. Participants included 2 women  and 6 men , with 2 of the 8  individuals identifying as Maori and the remaining  as NZ European. The majority were aged 75 to 84 years , 2  aged 65 to 74 years, and the remaining  aged 45 to 54 years. Participants were registered across 2 DHBs and interviewed at home , in the hospital , or via phone . Analysis of the interview data is summarized in 4 major themes.Participants identified themselves as having COPD, emphysema, and asbestosis, but only 6 participants were able to clearly describe their diagnosis and treatment. Diagnosis was reported as being a shock to receive; however, this shock did not necessarily change behavior:Well I didn’t know much about it, so it didn’t make much of a difference to me. I continued smoking, didn’t I?75-84-year-old male, ID number 4I knew from the day he said that, that things were not so good for me.75-84-year-old male, ID number 6Participants described COPD affecting all facets of their lives negatively. Despite this, all expressed some form of resilience and stoicism during the interviews:It affects everything: social, mental, physical.75-84-year-old female, ID number 5It can't be cured you just have to live with it and get on with life and do what you've got to do.75-84-year-old male, ID number 2I’m pretty humble with it. I’ve lived a pretty good life.45-54-year-old male, ID number 3Only 2 participants showed some degree of understanding of their test results, and the remaining participants did not understand the results of their tests; they knew what their diagnosis was but found the presentation of the test results difficult to comprehend. All participants struggled to remember which tests had been conducted and at which point in their illness trajectory:Honestly, I just go when I am told to go and have it done, and that’s it.75-84-year-old female, ID number 5Of the 8 participants, 7 were ex-smokers and acknowledged this as a contributing factor to their current respiratory condition; however, 5 offered additional environmental factors as contributors including asbestos and work environments.All patients interviewed had some experience of hospital-based PR programs. A total of 2 had completed programs within the past 3 years, 2 had started but stopped indefinitely because of hospitalization, 3 were in the process of completing an 8-week program, and 1 had tried a program over a decade previously. All but 1 reported enjoying the programs:I wish I had done it earlier.75-84-year-old male, ID number 2I love it. I wish they had it 12 months of the year.65-74-year-old female, ID number 7Yeah, I enjoy the exercise. It's just a matter of getting your mind in the right place. I get up and I do it and I feel a lot better afterwards.65-74-year-old male, ID number 1Reasons for this included socializing during the training, being able to compare progress and daily lived experiences with others with a similar condition, compassionate and dedicated staff who were known to the participants, and both a measurable and perceived improvement in physical ability. However, a participant reported anxiety and an increased sense of vulnerability after he was transferred to community support, following completion of the hospital-based PR program, arising from inconsistency in times and scheduling, different staff members at each visit, and a decreased sense of support.All but 1 of the participants  described that they preferred to be offered a PR program in person at the moment of diagnosis by a medical professional they trusted and had an existing relationship with, rather than when in hospital with an exacerbation:When you’re really sick and they come at you, like the physio comes, and this one comes, and that one comes, and you just feel like being left alone, I felt, just leave me alone sort of thing... maybe after being first diagnosed.75-84-year-old female, ID number 5The main barrier to mPR was a perceived incompetence with technology and a fear that this would be difficult to work with; however, 6 participants appeared to be confident texting during interviews, and 4 talked about using Facebook to communicate with family. A participant enjoyed using health apps to measure aspects of his health.Feature suggestions for mPR included knowing which part of the body exercises were targeting, with the most ambitious proposals suggesting:I think you should give them every feature that you can and maybe give them the option to choose... incorporating videos of people doing it and how they started and then how they finished... a motivational start, and then a talk and showing “look at me, now I can run two kilometers.45-54-year-old male, ID number 3truth of diagnosis, with patients reporting that they struggled to accept their diagnosis until they saw an image and that seemed more real than paper, which they saw as a tool for the medical professional:During the interview, participants were shown a model of the lung ,24 and aI didn’t get how bad my lungs were until I saw it.45-54-year-old male, ID number 3She said “have you ever seen your lung?” and showed me this x-ray and I thought, “whoa.”75-84-year-old male, ID number 2A participant said she had never been shown an x-ray or image of her lungs and insisted that this would have caused her to stop smoking earlier.A total of 8 key stakeholder interviews were completed. Key stakeholders included 2 doctors , 2 nurses, 3 physiotherapists, and 1 health psychologist. Participants worked across either predominantly urban  or predominantly rural  populations. A total of 5  participants were directly involved in the delivery of PR, whereas the remaining 3  were involved in referring patients to PR services.When asked about their perceptions of patients’ understanding of COPD, all acknowledged that it was generally poor. A total of 3 participants acknowledged that understanding varied, and those that had attended PR or sought health information through the internet had a better understanding. Common tools for explaining COPD to patients included handouts and pamphlets (n=3), Web resources , drawings and models (n=2), and patient scans (n=1).All participants thought that a technology- or mobile-based PR program was a good idea, particularly for overcoming the barriers their patients currently experienced to attending PR. However, the majority of the participants also raised concerns that an mPR program would lack the social element and not be suitable for some groups such as older patients or those with limited confidence and access to technology.Participants were asked how they felt an mPR program would fit into current models of care. Responses could be grouped into 3 main categories: (1) an alternative service to increase the access to PR for those that were not able to attend current services, (2) a maintenance program following traditional in-person PR programs, and (3) a combination/mixed model of care where patients used an mPR program to complement the in-person program. All participants reported that an mPR program should include the following components: (1) education, (2) exercise information, (3) motivation and support, (4) the ability to view personal health information/data, and (4) monitoring of health behaviors.Other components identified by the participants for inclusion in an mPR program included health psychology content, medication reminders, personalized action plans, step-by-step videos, social features, sharing of information between patient and health care professional, and general self-care information. Participants also identified that it would be essential for a program to be both individually and culturally tailored.When asked about the different technologies the potential program should utilize, many different technologies were identified including sensors, smart inhalers, apps, and text messages. But consistent across all participants was that the technologies used needed to consider differing access to technology, devices, and data, as well as confidence with using technology:Important to use mobile phone as a lot of people won't have anything else.ID number 1An app would be a good way of doing it... Sensors are useful for helping patients to see progress to goals.ID number 2Text messages are easy for patients to get, older patients to get, minority populations do not tend to have a lot of money on their phones.ID number 3A lot of people don’t have internet at all. We text or phone them, they aren’t able to text, call as they have no money on their phones. Some people don’t even have mobile coverage...ID number 6We already use SMS, people are used to getting text messages.ID number 7There were conflicting views from participants around how an mPR program should be accessed. A participant felt that referrals should be through the same avenues as current services to ensure safety was prioritized, and only people who were appropriate were accessing the program. Others felt that in addition to clinician referrals, patients should be able to access it directly to reduce barriers to access. There was only 1 participant who felt that clinicians should not be involved in referring to the program at all because of concerns that the clinician would need to then provide technical support:Referred [by a clinician] is best rather than self-referral, ensures that the info will be right for that patient. Same criteria as current groups.ID number 1Shouldn’t put up a barrier of the clinician referring them. Especially if they can’t afford to see their GP.ID number 2Anyone should be able to have access to it but there needs to be some sort of way of knowing people are safe with it, that they are safe to exercise.ID number 4Patients should be able to access it themselves. Health care professionals could promote it but don’t want to become IT support.ID number 5Finally, participants were asked about the potential barriers/downsides to an mPR program. All but 1 participant  reported concerns relating to the digital divide, including access to technology and the confidence to use it. A full list of barriers identified can be seen in This study aimed to identify the needs, preferences, and priorities of end users in the development of an adaptive mPR support program. A survey of patients, together with interviews of patients and key stakeholders, found a common interest in an mPR program. The potential for mPR overcoming barriers to accessing traditional PR programs was highlighted. These findings are consistent with previous research reporting high acceptability of digital health tools in patients with chronic respiratory conditions -28.This formative study has identified important aspects of our target audience and their diversity of needs. Some have low access to digital technology as well as low digital literacy and confidence, although most would like the option of a technology-delivered program. Needs include support during PR programs (between group sessions), support after a PR program to maintain behavior change, and a complete program for those unable to attend a traditional inpatient PR program. The study also identified a difference in clinicians’ perceptions of patients’ understanding of their condition (poor) versus patients’ actual comprehension and management of their condition.second class program for those with lower technology access is essential. Providing a program that not only considers individual technology access and literacy but also considers personal preferences and characteristics is also needed to ensure that it will be positively received by users. As indigenous populations, ethnic minority groups, and those from socioeconomically deprived groups suffer worse outcomes from COPD, ensuring an mPR program that strives for equity is essential. A strength of the survey sample was that it comprised over one-third Maori and Pacific participants. Key stakeholders highlighted the importance of considering culture and of culturally tailoring the mHealth tool.From these findings, our key considerations in designing a user-centered mPR support program will be in replicating the benefits of social support provided by the in-person group sessions, ensuring options are available for differing levels of digital literacy and confidence with technology while not providing a Key considerations in terms of satisfying referring clinicians are ensuring safety can be maintained and providing appropriate programs for different groups within the population. Finding a balance between ensuring safety and maximizing access is vital to ensure that an mPR program overcomes the barriers and increases access to PR support. Although the majority of the participants in the survey and interviews had attended PR or intended to, this study has identified a range of barriers to traditional in-person PR services that align with the previous studies -11. ThesIt is important that these findings are interpreted in light of the study’s limitations, including the small number of participants surveyed and interviewed, the potential sampling bias in those who agreed to participate, and the patients who participated having a higher proportion accessing PR services than the general population categories. It is likely that those patients who did participate had more interest in mHealth; therefore, engagement with this type of tool may be lower in the wider population. Pretesting of an mPR program with a wider patient sample will be essential to understand the acceptability of this type of intervention.Although there is a proliferation of technology in health care and increasingly innovative technologies being embraced for patient self-management, our study, consistent with previous studies, has shown that a digital divide exists, contributed to by differing access to data and tools as well as confidence and digital literacy to effectively use them -31. AlthWe have developed a prototype mHealth-based PR program based on the results of this mixed methods research. This includes different components suggested in the paper that will be pretested with people with chronic respiratory disorders. Feedback will be provided, and funding will be sought for the development of a full mPR program. The findings support the need for involving patients in the initial design and development of an mHealth intervention and the need to conduct a feasibility pilot study, once the intervention is developed, to try and better understand the degree of support required by health professionals and the degree of technical support required.


Family-based drug prevention programs that use group-based formats with trained facilitators, such as the Strengthening Families Program (SFP), are effective in preventing underage drinking and youth drug use. However, these programs are resource-intensive and have high costs and logistical demands. Tailoring them for Web-based delivery is more cost-effective and makes it easier to scale these programs for widespread dissemination. This requires the active involvement of all key stakeholders to determine content and delivery format.The aim was to obtain consumer, agency stakeholder, and expert input into the design of a Web-based parenting skills training and youth drug prevention program.We conducted 10 focus groups with 85 adults , 20 stakeholder interviews with family services agency staff, and discussed critical design considerations with 10 prevention scientists and e-learning experts to determine the optimal program content and technology features for SFP Online. Focus group participants also answered survey questions on perceived barriers to use, desired navigational features, preferred course format, desired content, preferred reward structures, course length, interactive components, computer efficacy, and technology use. Descriptive statistics were used to examine consumer characteristics; linear regression was used to examine relations between SFP exposure and four continuous outcome measures, including desired program content, interactive technology, and concerns that may inhibit future use of SFP Online. Logistic regression was used as a binary measure of whether consumers desired fun games in the SFP Online program.P=.03) and also wanted more interactivity , parents with less interest in a mixed delivery format (class and online) reported fewer reasons to use the online program , comfort using computers was negatively associated with concerns about the program , having mobile phone access was related to fewer concerns about online programs , willingness to view an online program using a mobile phone was positively associated with wanting more online components , and parents who wanted fun games wanted more interactivity .Three broad thematic categories emerged from the qualitative interviews enumerating the importance of (1) lesson content, (2) logistics for program delivery, and (3) multimedia interactivity. Among the many significant relations, parents who viewed more SFP lessons reported more reasons to use an online program (beta=1.48, Formative evaluation based on user-centered approaches can provide rich information that fuels development of an online program. The user-centered strategies in this study lay the foundation for improving SFP Online and provide a means to accommodate user interests and ensure the product serves as an effective prevention tool that is attractive to consumers, engaging, and can overcome some of the barriers to recruitment and retention that have previously affected program outcomes in family-based prevention. Youth alcohol and drug use continue to be top public health priorities. This emphasis stems from recognition that experimental drug use that often coincides with adolescent rebelliousness can easily transform into addiction if left unabated. Numerous trials have now shown that family-based drug prevention programs are an effective first line of defense preventing underage drinking and youth drug use -3. The eThe Strengthening Families Program (SFP) is one example of a parenting skills training program that combines youth drug prevention in a group-based, facilitator-led format for children aged 3 to 17 years. The program was first tested with 14 sessions designed to improve parenting skills among drug-addicted adults in treatment settings ,12. SincIn keeping with well-established research traditions, the program’s core components draw from family systems theory , relatioSimilar to many other family-based programs, parents and children receive separate instruction in the first hour and then join together to practice and receive feedback on newly learned skills during a second hour. This portion includes supportive role play and games that the parent and child play together, which encourages parents to implement nonjudgmental dialog and increase parent-child attachment. Behavioral reinforcement techniques, such as situational role play and behavioral rehearsal with positive feedback given to families by highly trained facilitators, are a hallmark feature of SFP. The program uses curriculum guides, homework, and workbooks that can be practiced in the home to reinforce lesson material. Providing family meals before each weekly session, offering childcare for young siblings, and assisting with transportation are implemented to address attendance barriers that potentially inhibit participation and retention over time.Notwithstanding evidence of their overall effectiveness, family-based programs remain resource-intensive with high costs and logistical demands. Any attempt to scale them for widespread dissemination has to address these implementation demands. In recent years, internet-based behavioral interventions have been making tremendous strides as alternatives to programs that customarily use face-to-face delivery, presenting a promising avenue for delivery ,29. Web-Repackaging group-based, facilitator-led programs for Web delivery faces several formidable challenges. To begin with, a program developer must consider how to preserve the active ingredients of a program as it is transitioned to a Web-based platform. This entails finding ways to transform behavioral practice and positive reinforcement techniques into digital content that preserves the integrity of the intervention strategies. Then developers must consider finding the proper dosage and session length, particularly because internet-based programs are traditionally shorter than in-person, group-based formats. Developers must also find ways to assess exposure, including engagement, and use this information to establish a metric for fidelity of implementation and adherence—two important components related to program efficacy -38.Engagement can be defined as the adoption of the program by consumers, and then continued use can be measured through session exposure. In both cases, recruitment to participate and retention throughout delivery in family-based programs have faced several hurdles -41. Inde. Participatory action research strategies in the form of consumer preference studies represent one way to address the concerns that arise from the deployment of Web-based programs [Transparency in the process of meeting these challenges would provide a template to guide future research examining the design and implementation of Web-based parenting skills training and youth drug prevention programs . Participrograms ,46. Consprograms . The strprograms  and Tripprograms -33 when programs . The goaprograms . Motivat. In choosing to develop an online version, several pressing questions needed to be addressed, including whether consumers felt the group-based format could be transformed in a way to maintain the active ingredients with no loss of fidelity in a digital environment. We also wanted to know what levels of interactivity and what type of program content are needed to offset factors that contribute to noncompliance. We applied both formative and summative strategies to obtain this information before developing and testing a prototype. We also posed specific questions to consumers regarding the feasibility, perceived utility, technology acceptance, desired instructional format , and other utilization factors that could influence engagement. This type of in-depth feedback is essential during the development process because it can be a foundation for the design of intervention modules and help structure their delivery. Although we solicited and received input from both parents and youth, we only focus on the adult consumer responses in this study.This study uses a mixed methods formative evaluation design to explore consumer preferences in the prototype build stage for SFP OnlineWe also extended prior work by including agency staff (stakeholders) and subject matter experts because they often have tangible insight that can shed light on and give shape to the product’s functionality. Agency staff who routinely work with families offering services may have their finger on the pulse of what families want and need and may also recognize some of the barriers to attendance that affect program outcomes. Experts can weigh in on how to maintain close alignment with the developmental and family-based theories that drive most parenting skills training interventions. As detailed subsequently, a second angle that e-learning experts can weigh in on includes integrating various instructional design principles into a serious educational game to increase the program’s overall attractiveness. In addition to the formative strategies, we used a consumer preference survey administered to focus group participants, which is explained in detail.We recruited 85 adults from nine geographically dispersed agencies in the US covering the western portion , the middle portion , the eastern seaboard (NC and NY), and the south (TX). These agencies were representative of the target agencies where SFP Online could be distributed, and they were willing to participate in the formative evaluation. The agencies conducted the focus groups between May and early June of 2018. Each agency had participated in prior SFP implementation training using the traditional group format or had implemented the DVD curriculum used in conjunction with group classes. An email was sent to all the agencies outlining the scope of the project and the requirements for participation. After the initial contact, we sent a memorandum of agreement to each agency outlining the participant recruitment requirements  and what family participation would entail, including a description of incentives that would be provided to both the individuals and the agency.We gauged the number of groups (approximately 4 to 10 members in each group) based on recent studies that confirmed thematic sufficiency with nonprobability sampling strategies can be achieved with three to six focus groups  and alsoEach focus group lasted between 60 and 90 minutes and involved a moderator; refreshments were provided at the midpoint. Agency staff assisted in obtaining written informed consent, conducting the groups, and collecting survey materials. Moderators were given a seven-page instructional guide outlining how to conduct the group, preserve confidentiality, and solicit balanced input from the group members. Moderator probes queried the parents’ computer technology and mobile phone usage, their video game experience, desired program content, instructional materials, navigational features, interactive components, and whether parents wanted asynchronous social media connections . Members of the research team were present during one of the focus groups and listened to others via GoToMeeting Web conferencing.We also conducted 20 key informant semistructured interviews by teleconference with agency staff primarily drawn from the same agencies that participated in the focus group. One exception was a new agency in MA that participated in the interviews. One agency staff from each site participated, except for NY, UT, and KS, where more than one staff member was interviewed. Agency staff had been previously trained to deliver SFP as part of activities distinct from this project; they also provide, as part of their regular duties, additional psychoeducational and support services to the families. Their input was intended to address whether they felt the intended Web-based program could retain the core instructional strategies, how to structure the instructional modalities so they would be engaging, and what technology design features would be appropriate for the target families. Before the phone call, each stakeholder was provided a 20-item questionnaire with specific probes intended to elucidate their involvement with SFP, their organizational role, the demographic profile of the agency clientele , their perceived feasibility and utility of implementing SFP Online, and the essential program components and skills training they felt should be included.We recruited 10 subject matter experts to weigh in on prototype development. Five were prevention scientists who we asked to address the procedures for translating core active ingredients into online programming and ways to ensure engagement. An additional five e-learning experts shared ideas on different ways to capitalize on current trends in multimedia interactive programming and whether this should influence the design of SFP Online. A brief survey was emailed to the experts before the teleconference interview, and their written responses helped guide the interview. The first part included 10 brief questions intended to elucidate their experience designing interactive programs. This was followed by 19 interview probes addressing the feasibility of SFP Online, implementation strategies, use of live family coaches, social media, interactive activities to promote family problem-solving skills, dashboard and navigation features that might be attractive, methods to stimulate engagement and increase session exposure, and perceived barriers or problems when implementing eHealth programs .Parents in the focus groups also filled out a 45-item consumer preference survey. Following completion, the anonymous surveys were collected by agency staff; placed in sealed, preaddressed envelopes; and mailed to the research team. Digital audiotapes from the focus groups and interviews were professionally transcribed verbatim, and the content of this material was then examined using thematic analysis as a research tool. This approach breaks down the open-ended answers into smaller units so the researcher can actively identify consistent and meaningful patterns in the text . We usedThe survey included additional items assessing issues that might arise in using SFP Online, which were modeled as outcomes. Seven items assessed technology barriers that might present concerns when using SFP Online, including data plans, visual content, screen size, and navigation, with responses coded on a five-point Likert scale ranging from not at all concerned to extremely concerned. An additional 20 items assessed what subject material parents felt was important to include in the online version  with responses coded on a five-point Likert scale ranging from very important to not at all important. An additional 16 items assessed the importance of different program components for learning parenting skills  with responses coded on a five-point Likert scale ranging from very important to not at all important. A 10-item scale assessed reasons for using SFP Online  with responses coded on a five-point Likert scale ranging from very important to not at all important. A single item asked parents if they were interested in a game-like version of SFP, with responses recoded as yes or no.P<.05 with two-sided tests.Both factor analysis and estimates of internal consistency indicated low reliability and lack of conceptual purity in the outcome scales. This is likely because parents treated each subset of items as a checklist, and this prevented us from obtaining homogenous unidimensional scales. As a result, we created unit-weighted indexes to represent the different technological, logistic, programmatic, and motivational reasons for using SFP Online. We dichotomously recoded the Likert response scales to 1 or 0 by collapsing “not at all concerned” and “slightly concerned” to 1 and coded the remaining response categories to 0. Likewise, we collapsed “very important” and “important” to 1 and all other responses to 0. For all analyses, the significance level was set at Eighty-five parents attended the 10 focus groups . Their age breakdown included 11 parents (13%) between 18 and 31 years, 43 parents (51%) between 32 and 45 years, 19 parents (22%) between 46 and 52 years, 10 parents (12%) between 53 and 66 years, and 2 grandparents (2%) 67 years and older. In all, 56 (66%) were white, 8 (9%) were Native American or American Indian, 5 (6%) were Hispanic or Latino, 4 (5%) were African American, 9 (9%) were mixed race, 2 (2%) were Asian, and 1 (1%) indicated “other.” The family service agencies served a fairly homogeneous clientele, who were relatively poor, socially marginalized, experiencing family distress, and characteristically low in education. Therefore, we did not ask about their socioeconomic status using income or education.Of the 85 parents, 79 (93%) said they had computer access with internet connection, and 69 (82%) said they were comfortable using computers, 11 (13%) said they were somewhat comfortable, and 5 (6%) said they were not comfortable. As anticipated, all the parents had some exposure to the SFP traditional group-based classes. Twenty-nine (34%) took a class with their family. Twenty (24%) viewed the SFP DVD video clips in a class setting, 21 (25%) viewed the DVD at home, and 11 (13%) viewed the SFP DVD with a coach. Thirty-three (39%) parents attended between zero and three lessons, 11 (13%) attended between four and six lessons, and 40 (48%) attended more than six lessons. Twenty-two (26%) parents said they preferred an online class, 35 (41%) said they wanted to attend an in-person class, and the remaining 23 (27%) were undecided between the two. Seventy-four (87%) parents said they would view SFP Online if it was available on a smartphone. When asked about including games in the new online version, 4 parents (4%) said they were not at all interested in games, 10 (12%) said they were slightly interested, 41 (48%) said they might be interested, and 30 (35%) said they would be very interested.A total of 241 keywords were culled from the thematic content analysis of the focus groups and interviews. Two coders conducted the content analysis and reduced the pool of open-ended answers to 86 distinct keywords. An example of a word that parents provided that was highly prevalent was “activities,” and the corresponding commentary included “parent-and-child activities to do together.” Another example was “challenges,” and the commentary included “ability to customize challenges” and “challenges that spark our interest.” We then tallied the keywords by frequency of their use with the most popular terms, which included “games” (n=55), “unique” (n=40), “fun” (n=15), “challenges” (n=14), “points” (n=11), and “rewards” (n=9). The remaining terms had frequencies in the single digits. Using an inductive procedure, we then summarized these terms into three major categories: program content, logistics, and interactivity/engagement. Program content encompassed teaching points and lesson plans , and referenced specific SFP core activities .Logistics encompassed program delivery features, such as what will make SFP Online engaging (technology and navigation control) and encourage future continued use . Interactivity and engagement encompassed multimedia features of the program . ContentExplain importance of sincere compliment (brain triggers); show Emotional Bank AccountExplain five steps of a reinforcing complimentKeep a 4:1 ratio of compliments to correctionsClick to create compliments you can give family membersShort, realistic video clipsDownload using SFP handoutsLogisticsClear, simple navigation tools; effective search barBright colors; animation; lots of graphics; no clutterCustomizable dashboardApp for tracking complimentsInterface linking parent and child responsesPoints for completed lessonsInteractivity/EngagementVirtual, customizable family coach who can ask questions, give advice, invite family to practiceCustomizable Cheering SquadRewards for learning and practicingAllow optional competitionSend chat messages to a family member (parent discussion board)Use Scoreboard to track and see if achieving personal SFP goalsEarn “practice coins” to “buy” accessories for avatar and coachContentAllow families to choose lessons; explain prerequisitesAbility to download and use SFP handouts“Yes, but...” page to resolve concerns with a virtual family coachPage with additional info: “To learn more, click here...”LogisticsSimple, short, text languageAudio for nonreadersInterface between parents and children to send messagesFrequently asked parenting questions with answersInteractivity/EngagementText message remindersReward family completion of a lesson visually on dashboard Enable parent discussion board ContentTeach core essential SFP Skills that target key risk and protective factorsInclude on-going reviewEarn points for skill practiceReward extra points and give congratulations on completionProvide family and individuals Certificate of CompletionLogisticsAbility to practice and get quick feedback.Low-stakes failure with immediate correcting featureOn sign up, receive a Family Page plus subpage for each childSite autoreports results completed and skills practicedMobile phone accessibleInteractivity/EngagementCustomizable Family page with name, banner, motto, DashboardCustomize learning modules to fit the age of childrenAllow parents and youth to choose lesson topics; explain needed lesson prerequisites for each skillReward family goal setting21=4.0, P=.04, φ=−0.22), were more likely to want to track their skills practice on a computer , and were more likely to want a coach . White parents  viewed more SFP lessons  and preferred doing the classes online . There were no significant race/ethnicity, age, or gender differences for desiring a gamified version of SFP Online. There were also no significant demographic differences in the mean scores for the four outcome measures .There were no gender differences in the categorical measures assessing parents’ SFP course exposure, technology readiness, or course content preferences. Younger parents were more likely to want to view SPF Online using their mobile phone . Parents who reported being very comfortable using computers expressed fewer concerns  compared to parents reporting they were somewhat comfortable  or not very comfortable , albeit these differences were not significantly different from each other by the Tukey-Kramer post hoc multiple comparison test.F1=10.94, P=.001; those owning a phone: mean 2.23, SE 0.63; those not owning one: mean 5.34, SE 1.07). Those wanting to view SFP on their mobile phone reported fewer concerns  for parents who could view SFP on a mobile phone compared with those who could not view SFP on a mobile phone . The third block included a measure asking parents how many lessons should be included in SFP Online, the proposed length of the individual lessons, whether fun games would help their family learn new skills, and whether family members should earn reward points when engaging with the online program. None of the individual predictors were significantly associated with technological concerns using SFP Online.The second block included whether parents had access to a computer with internet service, whether they would use SFP Online even if they had a class, if they had a mobile or smartphone to view SFP Online, would they view SFP Online if they had a smartphone, and whether they would record their home practice assignments on a computer or smartphone. Parents with access to a mobile phone reported significantly fewer concerns . The post hoc multiple comparison showed that parents expressing some comfort using computers wanted significantly fewer lesson components  compared with parents feeling very comfortable . Parents wanting to view SFP Online using a mobile phone wanted more online components  compared with parents who did not want to access the online program using a mobile phone . No predictors in the third block were individually significant.The next model examined the associations between the same three blocks and an index assessing important programmatic components to include in SFP Online. For the first block, ease of using computers was the only significant measure in the model . Parents who completed an intermediate number of lessons (4 to 6) wanted more interactive components  compared with parents who watched the most lessons  and parents who watched the fewest lessons . The second block included one significant measure; whether parents desired to take a class even if they took SFP Online was significant . Parents desiring to use SFP Online even with a class wanted more interactive components  compared with parents stating “no” , although these post hoc comparisons were not significant.The third index captured desired interactive components to include in the new online program. In the first block, the number of SFP lessons completed on the DVD was significant , wanting SFP Online to include fun games , and wanting rewards built into the program . Parents viewing between 1 and 5 lessons wanted fewer interactive components  compared with parents viewing between 6 and 10 lessons . Parents stating “yes” to wanting fun games also reported wanting more interactivity  compared with those stating “maybe” .The final block included three significant predictors assessing the number of proposed lessons for the new online program . Parents not wanting to track their skills practice reported significantly fewer reasons to use SFP Online  compared with those saying “yes”  or “maybe” . In the second block, only wanting to take SFP Online in conjunction with a class was a significant predictor . Parents stating they would probably not do the online class in addition to a group class reported significantly fewer reasons to use SFP Online  compared with those stating “maybe”  and “yes” . In the third block, wanting rewards while using the new online program was significant . Parents who said they probably did not want rewards through the online program reported significantly fewer reasons to use SFP Online  compared with parents who did want rewards .The final model included the same three blocks individually predicting reasons (motivations) to use SFP Online. In the first block, only tracking SFP skills practice on the computer (versus handouts) was significant  to want a game-like experience. Those parents who were undecided were five times as likely  to want a game-like experience compared with the reference class. Unadjusted, the C-statistic (a variant of Somers’ D) was 0.719, and the rescaled R2 (coefficient of determination) accounted for 16.6% of the variance.We then repeated the block entry procedure using logistic regression with a binary measure asking parents whether they were interested in using a game-like version of SFP . To avoid convergence problems (maximum likelihood estimates that were not trustworthy), we examined each predictor one by one and collapsed response categories for the measures, reducing the number of cells in the estimation process. We then combined all the significant predictor measures culled from the individual models into a single model, controlling for race and age. Based on the individual models, parents who preferred taking SFP online (attend class was designated the reference or comparison class) were almost nine times as likely to want a game-like experience than those who were undecided  compared with those reporting they were not at all comfortable using computers. Parents with mobile phones on which they could view SFP Online were 11 times as likely to want a game-like online version than parents lacking mobile phone access . Parents who had a smartphone were six times as likely to want fun games than parents who did not have smartphones . Parents who stated they would be likely or somewhat likely to record home practice assignments on their computer were 78% more likely to want a game-like version compared with parents who reported they would be somewhat unlikely and unlikely .Parents reporting they were very comfortable using computers were eight times as likely to be interested in a game-like online version of SFP . When all six predictors were modeled at once, only the measure of preferring the online version to a class was significant ; adjusting for race and age did not improve the model fit.A model with all the individually significant predictors combined fit well  desired content of the program, (2) logistics affecting program delivery, and (3) interactive features that can stimulate engagement. Interestingly, these themes reflected commonality in how consumers, agency stakeholders, and experts view the essential components of an online parenting skills training program that also involves youth drug prevention. Importantly, experts emphasized features consistent with cognitive behavioral principles, including skills practice, positive feedback, low-stakes failure exercises, and using reward structures  that help families move toward various programmatic benchmarks. They also outlined several implementation challenges, including program length and finding ways to use multimedia and interactive features to reinforce the focus and core active ingredients of the program.Experts also felt the online program content should be realistic, simple, and use video examples to teach problem-focused coping skills, reflect the body of literature on risk and protective factors, and provide parents with ready solutions to their problems. The parents, in particular, stated that all this should be crafted using animation that can exemplify role-playing skills and reinforce lesson content. Experts also felt that SFP Online should stimulate “authentic” collaboration between parent and child and offer coaching, discussion boards, and virtual interactions—all with the goal of increasing engagement to maintain high exposure over time., including shorter sessions, graphics emphasized over text, dashboards for the entire family, text message reminders, home practice materials, printable worksheets, and a means for agency personnel to track family progress. They also mentioned that parents may still want to meet in person with a facilitator, even if they had online access, and that boosters might be essential to reinforce program content over time.Agency staff agreed in principle with the experts and felt that the online version should maintain a skills-oriented approach and be configured around real family experiences. Given their familiarity delivering the group format of SFP, they emphasized preserving many of the current fun activities that work well in the group setting. From a logistical standpoint, they listed numerous features that would enhance SFP OnlineParents felt very enthusiastic about helping to design the program content and delivery format for the online version. They wanted shorter sessions and state-of-the-art navigation features, with pop-up notifications, rewind capabilities, drop-down screens and scrollable information, graphic display with virtual animation, and realistic role plays. The parents who had the most prior exposure to SFP provided the most feedback on program components that they believed could work online. They felt the program should include bulletin boards, quizzes, pause-and-practice features, proficiency rewards, weekly blogs from experts, and virtual role plays with instruction. They also felt the online version should appeal to a culturally diverse population.The consumer preference survey augmented the inductive procedures by adding more fine-grained information that helped to clarify what factors are associated with technological barriers, course content, interactivity, and motivations to use the online version. Interestingly, technology-savvy parents expressed fewer concerns about using an online program and showed a clear preference for interactivity and designing SFP Online like a serious educational game. The parents reluctant to track their performance online and those not wanting performance rewards were less inclined to want a Web-based platform to learn parenting skills. Mobile phone access was ubiquitous in this sample, underscoring the narrowing of the digital divide in terms of access to mobile technology and the potential for eventually creating a mobile app to complement SFP Online . In all Even though the qualitative portion of the study reinforced tremendous overlap between the different groups, there were also unique contributions expressed by each group. For instance, agency stakeholders desired to find ways of connecting parent and child in the Web platform as this could be a strength of SFP Online. In addition, agency stakeholders were less inclined to address animation as crucial to the program, and more inclined to address interactive components that reinforce behavior change  and the use of technology to improve compliance . In this case, the focus of stakeholders appeared vested more in creating teachable moments and less focused on using technology to increase consumer involvement.Several limitations should be mentioned. The data are cross-sectional, and we do not know, for example, whether any of the computer and mobile phone use questions actually predict subsequent use of SFP Online. Moreover, the purposive sampling intentionally recruited parents with prior exposure to SFP, including having used the DVD or participating in the group format. This can introduce bias because many of the parents who attended classes but desired an online version wanted the option to attend a class based on their prior exposure. Having the opportunity to receive immediate in-person feedback on their performance may be an attraction that is hard to duplicate on the Web. In addition, the small sample size altered the model testing sequence to avoid overfitting models. As a result, we could not test incremental variance attributed to each block of predictors. Related to this, some of the statistical relations may not have achieved significance merely because the study was underpowered. This problem underscores the trade-off between obtaining qualitative data from a small but sufficient number of focus groups and wanting a large enough and appropriately powered sample to produce robust parameter estimates. Notwithstanding, there is a wealth of information gathered using a mixed methods approach, and we certainly benefited from employing these different strategies.One strength of this study is that we solicited collaborative input from prospective consumers as part of an active partnership preceding the development of the online program. This made participants feel valued and that they could contribute to program development in a meaningful way.Curtailing the rising tide of addiction in America requires taking evidence-based family prevention programs to scale, which can best be achieved by providing high-quality, low-cost, online delivery. Creating an effective, online family-based prevention tool that parents and youth will enjoy and use requires knowledge of what parents and youth want to learn, how they want to acquire this knowledge, and what factors will keep them engaged so they can develop the appropriate skills and complete the course.