The sciatic nerve block in the popliteal fossa is a popular lower extremity block for below-knee surgeries. Here the sciatic nerve is targated at or just above the point of its divergence into the tibial and common peroneal nerves. Amongst the described techniques, the supine approach of popliteal fossa block offers greatest patient comfort but has a few challenges accessing the nerve. We describe a novel ultrasound-guided distal transverse or crosswise approach to popliteal sciatic (CAPS) block performed in five patients in the supine position without unsteadiness of the knee or hip joint. The popliteal sciatic nerve block (PSNB) is a widely practiced regional anesthesia (RA) technique to provide anesthesia or analgesia for below-knee surgeries. It is either used alone or in combination with a femoral or saphenous nerve block. The use of ultrasound for PSNB improves the success rate and reduces the risk of complications . The ultThe sciatic nerve (SCN) enters the popliteal fossa through its apex and diverges into the tibial nerve (TN) and common peroneal nerve (CPN). However, this divergence can occur at any level from its origin  to its termination in the popliteal fossa. Cadaveric studies reported the divergence of the SCN at 0-11.5 cm, even up to 18.5 cm above the popliteal crease, where TN and CPN leave the common paraneural sheath ,10. BaseFor this case series, five patients admitted to our hospital for unilateral below-knee surgeries under the central neuraxial block (CNB) were enrolled. The written consent for the procedure, CAPS block, and publication was obtained from each patient or their relatives. Approval from our Institutional Review Board was also obtained for this case series. The CAPS block was performed in the postanesthesia care unit on the operated limb when full motor power had returned following CNB to the non-operated lower limb . The demographic parameters, diagnosis, types of surgery, and anesthetic techniques are shown in Table The success of the CAPS block technique depends on the appropriate positioning of the ultrasound transducer over the lateral aspect of the distal thigh in the transverse plane and targeting SCN at or above the point of divergence. Hence, CAPS block can be performed at 5-10 cm proximal to the popliteal crease at or above the point of divergence where popliteal vessels may or may not appear in the ultrasound image. A high-frequency linear  or a low-frequency curvilinear (Sonosite rC60xi/5-2 MHz) ultrasound transducer was placed in a transverse plane perpendicular to the skin over the lateral aspect of the lower thigh. It is positioned on or below the intermuscular groove formed by the vastus lateralis and biceps femoris muscles. It corresponds to the upper border of the patella, just proximal to the level of the popliteal crease Figures , 1b. TheIf required, a folded towel, soft pillow, or foam leg elevator may be placed underneath the calf muscles to create room for the probe placement Figure . The proThe distance of the probe from the popliteal crease, the depth of SCN from the skin on the ultrasound image Figure , the disA 22-gauge, 100-mm echogenic nerve block needle was inserted perpendicular to the skin from a lateral to medial direction using an out-of-plane technique Figure . After p-1 every six hours and ketorolac 0.5 mg.kg-1 every eight hours as a part of multimodal analgesia. The rescue analgesic fentanyl 1 mcg.kg-1 was administered when the pain score was 4 on a numeric rating scale, and the time was noted . All the patients were followed up for the next 24 hours.The type of transducer, electrostimulation, number of injections (single or multiple) needed to achieve the “donut sign,” post-block distal scanning to visualize LA spread around two divisions (TN and CPN), and the duration of analgesia was also recorded Table . PostopeWe used electrostimulation in all the cases to confirm the identity of the SCN and recorded the first evoked motor responses Table . The bloOur CAPS block technique successfully blocked the SCN in all five patients without technical difficulty or complications. None of the patients needed any human assistance for limb positioning during the performance of the block. The folded sheet was also not required in any of our cases. It provided adequate postoperative analgesia. The SCN, paraneural sheath, and subparaneural space were distinctly visualized. On neurostimulation, plantar flexion and inversion were elicited in three cases and dorsiflexion in two patients.In the popliteal region, the SCN is surrounded laterally by the biceps femoris (long and short heads) and medially by the semitendinosus and semimembranosus muscles Figure . SCN divThe success of the CAPS block technique depends on the appropriate positioning of the ultrasound transducer over the lateral aspect of the distal thigh in the transverse plane and targeting SCN at or above the point of divergence. We observed divergence of SCN about 2 - 6 cm proximal to the popliteal crease while performing CAPS block. However, this divergence can occur at any level from its origin  to its termination in the popliteal fossa. The SCN divides about 0-11.5 cm cephalad to the popliteal crease, and the superficial femoral artery enters the popliteal fossa as the popliteal artery about 8.5-11.5 cm above the popliteal crease ,10,12,13Sinha and Chan first described the ultrasound-guided PSNB in 2004 in the prone position . In 2007We believe the CAPS block would overcome the difficulties faced by other techniques. The supine position is convenient for the patient and the anesthesiologist. In the out-of-plane technique, the needle enters perpendicular to the skin. A shorter needle trajectory can cause less tissue damage than in-plane needling. Hence, it would be less painful for awake and unanesthetized patients. The chance of vascular puncture or LA systemic toxicity is also relatively less as it is performed away from vessels. This is a case series that has its limitations. The performer's experience might play a role in the success. The motor block effect of the CAPS block needs to be evaluated especially in the light of enhanced recovery after surgery. Also, the feasibility of CAPS block in obese patients and catheter placement for continuous CAPS block needs to be evaluated. Thus, the CAPS block requires further investigation in the form of prospective randomized controlled studies, cadaver studies, and dye studies in both pediatric and adult age groups.CAPS block is an easy, safe, and effective alternative approach to PSNB. We performed the CAPS block without altering the limb position or experiencing technical difficulties. It successfully provided the desired analgesia in all patients. We believe that the CAPS approach is simple and convenient for the patient and the anesthesiologist. Combining CAPS block with saphenous nerve block can provide complete analgesia for below-knee surgeries. However, further research and comparative trials are needed to evaluate its analgesic efficacy and safety profile over other approaches.


Internet-based programs can help provide accessible and inexpensive behavioral health care to those in need; however, the evaluation of these interventions has been mostly limited to controlled trials. Data regarding patterns of use and effectiveness of self-referred, open-access online interventions are lacking. We evaluated an online-based treatment designed to address stress, depression, and conflict management, the Dartmouth PATH Program, in a freely available and self-guided format during the COVID-19 pandemic.The primary aim is to determine users’ levels of stress and depression, and the nature of problems and triggers they reported during the COVID-19 pandemic. A secondary objective is to assess the acceptability and usability of the PATH content and determine whether such a program would be useful as a stand-alone open-access resource. The final objective is understanding the high dropout rates associated with online behavioral programs by contrasting the use pattern and program efficacy of individuals who completed session one and did not return to the program with those who came back to complete more sessions.Cumulative anonymous data from 562 individuals were analyzed. Stress triggers, stress responses, and reported problems were analyzed using qualitative analysis techniques. Scores on usability and acceptability questionnaires were evaluated using the sign test and Wilcoxon signed rank test. Mixed-effects linear modeling was used to evaluate changes in stress and depression over time.P=.03), and there was a significant decrease in depressive symptoms over all time points (P=.01). Depression severity decreased on average by 20%  between sessions one and two.A total of 2484 users registered from April through October 2020, most of whom created an account without initiating a module. A total of 562 individuals started the program and were considered in the data analysis. The most common stress triggers individuals reported involved either conflicts with family or spouses and work or workload. The most common problems addressed in the mood module were worry, anxiousness, or stress and difficulty concentrating or procrastination. The attrition rate was high with 13% (21/156) completing the conflict module, 17% (50/289) completing session one of the mood module, and 14% (16/117) completing session one of the stress module. Usability and acceptability scores for the mood and stress modules were significantly better than average. In those who returned to complete sessions, symptoms of stress showed a significant improvement over time (Conflicts with others, worry, and difficulty concentrating were some of the most common problems people used the programs to address. Individuals who completed the modules indicated improvements in self-reported stress and depression symptoms. Users also found the modules to be effective and rated the program highly for usability and acceptability. Nevertheless, the attrition rate was very high, as has been found with other freely available online-based interventions.ClinicalTrials.gov NCT02726061; https://clinicaltrials.gov/ct2/show/NCT02726061 Community epidemiological surveys estimate that as many as 30% of adults in the United States are affected by a mental disorder, yet less than half see a physician, and only a quarter are treated properly ,2. A numTechnology can increase access to care by providing secure, inexpensive, and easily accessible treatment tools. Internet-delivered CBT (ICBT) has existed for 20 years, and a number of controlled trials and meta-analytic reviews have demonstrated the effectiveness of this approach -10. AlthOur study evaluated a web-based program in a freely available format. The Dartmouth PATH Program is a multimedia-based computerized CBT tool designed to address stress, depression, and conflict management. The program was developed as a psychosocial training and treatment resource for the National Aeronautics and Space Administration (NASA) with the aim of addressing psychological challenges endured by astronauts on long duration spaceflights . The opeComparably, the COVID-19 pandemic has also been associated with mental health challenges related directly to the virus’ morbidity and mortality, and indirectly by the impact of physical distancing and stay-at-home orders . AccordiThe program was freely available as part of NCT02726061. Participants needed to agree to participate using an online consent form. We evaluated responses to the PATH program during the COVID-19 pandemic from April through October 2020. The data collected were fully anonymous and self-reported. Our objectives were to determine the levels of stress and depression as well as the nature of problems affecting individuals during the pandemic, assess the acceptability and usability of the PATH content and determine whether such a program would be useful as a stand-alone open-access resource, and understand the high dropout rates associated with online behavioral programs by contrasting the use pattern and program efficacy of individuals who completed session one and did not return to the program with those who came back to complete more sessions.The PATH program is an interactive, media-intensive CBT-based program that interacts with users in real time and delivers individualized feedback based on self-reported responses . In addiThe conflict module teaches participants how to approach conflict and reach effective solutions using CBT principles. It includes a conflict briefing, an interactive conflict simulation, a cognitive restructuring exercise, and a training module on interest-based negotiation . An evalAt the end of each one of the four sections in the conflict module, users received a survey on how valuable, feasible, and realistic the simulation was on a scale of 0 to 4 ranging from “Strongly disagree” to “Strongly agree.” The questions covered whether the activity had too much or too little information and if it was interesting or valuable.Like the conflict module, the stress module also uses CBT principles and focuses on stress management and resilience training. The module consists of six approximately 1-hour sessions. A randomized controlled trial of a version of the stress module reported significant reductions in perceived stress and increases in perceived control over stress . Each seUsers’ progress was tracked at the beginning of each session through a self-reported survey regarding satisfaction with their progress since the last session on a binary scale . Participants were also asked how often they practiced each skill learned in the previous session on a scale ranging from 0 to 3 .Participants completed the Perceived Stress Scale-14 (PSS) questionnaire at the beginning of each session to assess the degree to which events were perceived as stressful since their last visit. The questions in the PSS were meant to convey feelings and thoughts experienced in the last month on a 0 to 4 scale ranging from “never” to “very often.” The questionnaire contains 14 questions, and the final score is obtained by reversing the scores on seven of the positively stated items and then summing across all 14 questions. The PSS has established adequate internal and test-retest reliability, and correlates with life event scores .The depression module uses problem-solving treatment and consists of six sessions lasting 30 to 60 minutes. In these sessions, a mentor guides users through a step-by-step problem-solving therapy tool. Participants were first asked to identify and clarify a problem, establish an achievable goal, brainstorm solutions to the problem, evaluate pros and cons of each solution, develop an action plan to implement the selected solutions, and finally schedule enjoyable activities they will do during the next week. At the end of each session, a summary printout was available containing the problem selected, the action plan developed in that session, and a list of the selected scheduled enjoyable activities to be completed before the following session . The proParticipant progress was tracked through surveys regarding satisfaction with the amount of effort spent trying to solve the problem on a 1 to 10 scale ranging from “not satisfied at all” to “extremely satisfied” at the beginning of each session.Users completed the Patient Health Questionnaire-9 (PHQ-9) Item Depression Scale at the beginning of each session to assess depression symptoms they had experienced for the past 2 weeks. The questionnaire had participants rating nine questions concerning their depression symptoms on a 0 to 3 scale ranging from “not at all” to “nearly every day.” The final score was calculated by adding up the total scores from each question, which ranges from 0 to 27 . The questionnaire has demonstrated strong test-retest reliability and internal consistency .The study was approved by the Dartmouth Committee for the Protection of Human Subjects. The online program was made freely available to anyone interested in participating in the study. Users were required to accept an online consent form and then create a username and password of their choice. The data were acquired during the period of the COVID-19 pandemic from April through October 2020.Users were able to browse the program’s website freely and to choose to go through the cognitive behavioral modules they selected at their own pace. The main page contains all the potential choices for users to choose from, which includes the three primary modules of depression, stress, and conflict management; a guide on how to use the program; self-assessments to guide users on finding relevant content based on individual needs; and other resources and publications.Participant age and gender were collected upon signing up in the study.The Post-Study System Usability Questionnaire (PSSUQ) is a 19-item self-report questionnaire used to assess user satisfaction with system usability at the end of a study.The items are scored on a 7-point scale (1-7) on the strength of agreement with each statement . The scale ranges from “Strongly agree” (1) to “Strongly disagree” (7), and a “Not applicable” indicates answers outside the scale. The overall score is obtained by summing across scores for all questions (1-19), and the evaluation is further subdivided into three subscales: system usefulness (questions 1-8), information quality (questions 9-15), and interface quality (questions 16-18). The overall scale and its subscales have shown adequate levels of reliability, validity, and sensitivity ,31. InitThe Acceptability of Self-Guided Treatment (AST) is a 16-item self-report questionnaire developed in previous research on an earlier version of the depression module as a stand-alone treatment for depression . It compR2020a (v9.8.0).To evaluate for statistically significant improvements in PSS (stress module) and PHQ-9 (mood module) scores, we performed the nonparametric Wilcoxon signed rank test using MATLAB (MathWorks). Since the dropout rate of sessions three to six were too high, this test was only performed to assess the median of differences of paired samples for both PSS and PHQ-9 questionnaires from sessions one and two at a 5% significance level . The PSSUQ and AST analyses were also performed using the sign test to evaluate whether the given scores were statistically significant against the neutral mean value of each questionnaire’s scale. A right-sided test for the median of the AST scores and a left-sided test for the median of the PSSUQ scores were performed at a 5% significance level. This tested the hypothesis that the median scores of the AST questionnaires were higher than neutral (more acceptable) and that the median scores of the PSSUQ questionnaire were lower than neutral (more usable). To determine whether stress and depression would show statistically significant improvement over time, we conducted linear mixed-effects modeling to account for the time-varying natures of the variables and missing data. We entered time as the fixed effect and the intercept of subjects as random effects into the model. ATLAS.ti software (ATLAS.ti Scientific Software Development GmbH) was used to synthesize the qualitative data for both the stress and mood modules . The numerical values are presented in this study with mean (SD), median (range), or both. The statistical analyses were conducted using MATLAB Between April and October 2020, a total of 2484 users registered with the program website in response to media coverage of the program during the COVID-19 pandemic. Of those who registered, 1321 (53.2%) self-identified as females and 1042 (41.9%) as males with a mean age for the entire group of 44 (SD 15.2) years. The majority of those who registered did not interact with the program and therefore were not included in the treatment dropout rate. The dropout rate involves leaving treatment before its completion  and may A total of 562 participants interacted with the program modules and were included in the data analysis. At baseline, 156 users interacted with the conflict module, 289 with the mood module, and 117 with the stress module. The conflict module had four subsections and was completed by 21 users . SessionThe Conflict Introduction Questionnaire was completed by 102 participants  and was P=.19). The individual analysis of the PSS scores across sessions one through six is shown in The mean PSS score baseline for session one, including all participants (n=61), was 30.0 (SD 6.6). The average PSS of session one for those who returned for more than one session (n=8) was 28.6 (SD 5.9) while session two (n=8) was 27.3 (SD 9.1). The difference in scores for each participant between the first and last completed sessions was analyzed using the Wilcoxon signed rank test and was found to not be significant (P=.03).To determine whether stress symptoms would improve significantly over time, we used a linear mixed-effects analysis of the relationship between time and PSS scores for each participant throughout all sessions completed. We entered time as the fixed effect and the intercept of subjects as random effects into the model. The decrease in symptoms in relation to time was significant  versus those who completed multiple sessions (n=7). Individuals were able to select multiple responses .The stress triggers were grouped into 10 major categories: (1) conflicts with family or spouse; (2) work or workload; (3) conflicts with friends or neighbors; (4) financial concerns; (5) health concerns; (6) boredom and lack of productivity; (7) time away from friends and family; (8) internal stressors, which included negative views about oneself, uncertainty of future, and fear of failure; (9) COVID-19 and isolation; and (10) current political climate. The responses were categorized into each of the major groups, and only one category was counted per person .Conflict with family or spouse and work or workload were the two most common stressful triggers among participants. In contrast, conflicts with friends or neighbors and financial concerns were ranked third by those who completed multiple sessions, while health concerns and boredom or lack of productivity were ranked third by those who completed just one session. The response percentage rate for each category can be found in The emotional response to perceived stressors were also grouped into 10 major categories: (1) hopelessness, (2) worry or anxiety, (3) anger, (4) irritability, (5) frustration, (6) sadness or depression, (7) fear or panic, (8) obsessions or overthinking, (9) cynicism, and (10) burnout or overwhelmed. The responses recorded allowed for multiple responses, and each response was placed into the appropriate category . The most common responses were hopelessness, worry or anxiety, anger, irritability, and frustration among participants. The response breakdown can be found in The time spent on each part of session one in the mood module was calculated in addition to the retention rate throughout the session. It had five major steps, including completing the PHQ-9, defining the problem, selecting a goal, brainstorming solutions, and developing an action plan. The time elapsed began when the participant started each section until the section was completed, including participant idle time and time away from the computer. The calculation included both individuals who completed the section in one and multiple sittings. For this reason, participant completion times totaling over 100 hours were not included in the calculation, and the mode and range are included in The type of problems selected by participants in the program was evaluated to better understand the kind of issues experienced by participants and differences, if any, among individuals who only completed one session (n=145) versus those who completed multiple sessions (n=8). The problems were grouped into 13 categories: (1) worried, anxious, overthinking, or stressed; (2) difficulty concentrating or procrastination; (3) problems with overeating or undereating; (4) not enough exercise; (5) negative feelings about oneself; (6) loss of interest or lack of motivation; (7) anger, irritability, or frustration; (8) problems with work; (9) problems with sleep; (10) lack of social activities or hobbies, or isolation; (11) problems with relationships ; (12) problems with weight; and (13) financial problems. Those who completed multiple sessions ranked worried, anxious, overthinking, or stressed and difficulty concentrating or procrastination as the highest among the major problems. In contrast, those who completed just one session had problems with work and not enough exercise as the highest ranked. The complete analysis is shown in P=.06). The depression severity levels between the two sessions decreased by an average of 20.0% (SD 35.2%). The mean PHQ-9 score for session three (n=4) was 5.8 (SD 3.1), session 4 (n=1) was 5.0, session 5 (n=1) was 6.0, and session 6 (n=1) was 5.0. The individual analysis of the PHQ-9 scores across sections one through six is shown in The change in depression severity levels was analyzed using the PHQ-9. The mean PHQ-9 score baseline for session one, including all participants (n=289), was 9.98 (SD 6.23). The average PHQ-9 of session one for those who returned for more than one session (n=8) was 11.0 (SD 5.4) while session two (n=8) was 8.0 (SD 4.2). The difference in scores between the first two sessions, analyzed using the Wilcoxon signed rank test, was not significant .The linear mixed-effects model analysis of the relationship between time and PHQ-9 scores with time as the fixed effect and the intercept of subjects as random effects showed depressive symptoms decreased significantly over time  selected more activities on average than those with lower PHQ-9 scores who completed only one session (n=35). A visual analysis is shown in The most common type of activities selected among participants were entertainment, social, and physical activities. The response breakdown for each category is shown in This secondary analysis consisted of evaluating the usability and acceptability of the program and differences perceived by those who completed session one and did not return to the program versus those who came back to complete more than one session.Lower PSSUQ scores indicated a more agreeable usability experience. A total of 22 individuals completed this questionnaire at the end of session one in the mood module, of which 5 individuals went on to complete more than one session. The mean overall usability score after session one given by those who completed one session (n=17) versus those who completed multiple sessions (n=5) was 2.7 (SD 1.8) and 3.2 (SD 2.2), respectively.A total of 10 individuals completed the usability questionnaire at the end of session one in the stress module, of which 4 of those participants completed more than one session. The mean overall score after session one given by those who completed one session (n=6) and multiple sessions (n=4) was 3.4 (SD 2.5) and 4.1 (SD 2.4), respectively. Mean scores for each one of the PSSUQ subscales are presented in Higher scores on the AST questionnaire indicate a higher degree of acceptability. A total of 25 individuals completed the questionnaire at the end of session one in the mood module. The mean acceptability score given by those who completed session one only (n=20) was 5.7 (SD 0.7), and the mean score given by individuals who completed multiple sessions (n=5) was 5.9 (SD 0.4). Both of these were significantly different from neutral.A total of 12 individuals completed the questionnaire at the end of session one in the stress module, of which 5 individuals completed multiple sessions. The mean acceptability score for the stress module after session one given by those who completed one session (n=7) was 5.5 (SD 0.7), and the mean score given by those who completed multiple sessions (n=5) was 5.3 (SD 0.4). These scores were significantly different from neutral.The mean acceptability score for the conflict module (n=20) was 5.3 (SD 1.2) and statistically significant from neutral. Mean scores for each AST item are presented in The difference in response patterns was further evaluated between individuals who completed the first session only and those who completed multiple sessions for the item “I would rather use a computer to help myself privately than go to a therapist.” In the stress module, 43% (3/7) of individuals who completed only one session chose “agree,” while 57% (4/7) chose “neither agree nor disagree.” A third of participants who completed multiple sessions chose “somewhat agree,” 50% (3/6) chose “neither agree nor disagree,” while 17% (1/6) chose “somewhat disagree.” In the mood module, 45% (9/20) of participants who completed only the first session agreed with the statement and 30% (6/20) were neutral, while 25% (5/20) disagreed with the item. Individuals who completed multiple sessions in the mood module mostly agreed with the item at a rate of 60% (3/5), while 40% (2/5) remained neutral. The complete analysis can be found in This study evaluated the use and effectiveness patterns of a free and unrestricted web-based program designed to provide cognitive behavioral–based approaches to stress, depression, and conflict management during the COVID-19 pandemic. The program provided insight into the kinds of problems people were experiencing during the pandemic and seeking help for. In the stress module, the greatest source of stress was conflicts with family or spouse. Feelings of hopelessness, worry, anxiety, and anger were common. The problems selected for the mood program focused on issues of worry, anxiety, overthinking, and stress. Those who returned to complete more than one mood session tended to have problems with worry, stress, and anxiety.Those who returned to complete more than one stress or mood module showed significant reductions in self-reported stress and depression, although they did not rate the program as high for usability as those who completed only one session. Individuals who completed multiple sessions of the mood module also rated the program higher for acceptability than those who completed only the first session and tended to agree that they would rather help themselves privately with a computer rather than go to a therapist. For the stress module, individuals who completed only one session rated the program with slightly higher acceptability scores than those who returned to complete multiple subsequent sessions. On average, users thought the conflict training was useful and would recommend it to a friend. In the conflict module, the interest-based negotiation exercise was the highest rated.The usability score describes the perceived ease of use of the technology applied in this program. It is based on a scale of 1 to 7 (ranging from “strongly agree” to “strongly disagree”), where a higher usability score infers a lower perceived ease of use by the participant. A psychometric evaluation of the PSSUQ established norms for this questionnaire . The amoThe PSSUQ score results provided one surprising finding. Those who completed more sessions  perceived the program as less usable than those who completed just one session . This result was unforeseen, as one would expect participants rating the programs with a low usability score would be likely not to continue. The significance of this finding is limited by the small number of people in these groups who have usability data. Nevertheless, it is interesting that those who chose to continue were not necessarily those who rated the program highly for usability. For the mood module, one possible explanation for continuing was the participants feelings about whether they wanted to use a computer rather than a therapist. Although 25% of those who completed only one mood session disagreed with the statement “I would rather use a computer to help myself privately than go to a therapist,” none of those who did more than one session did.Ratings on the AST showed that participants found all three modules to be acceptable. The mean score  for the mood, stress, and conflict modules were 5.7 (SD 0.7), 5.5 (SD 0.7), and 5.3 (SD 1.2), respectively, which were all significantly different than average. These results are similar to mean scores for the same modules when they were used in isolated and confined environments  . The HI-P<.001 on a 0-4 scale). From the Hypothesis Testing Questionnaire, participants indicated that the program was somewhat easy to understand , and the feedback somewhat helped them understand why responses were correct or incorrect . Lastly, the conflict-based negotiation activity was found to be both enjoyable  and highly valuable for learning about conflict management .The conflict module consisted of four sections, each concluding with a short questionnaire. The Conflict Introduction and Conflict Simulation Questionnaires were found to be interesting and valuable ; however, the scores indicated a significant improvement over time (P=.03). There was no difference in the response pattern of the selected stressful triggers and perceived emotional response to stress between those who completed one session versus multiple sessions. The major stressful triggers selected by participants included conflict with family or spouse and work or workload. Noticeably, COVID-19 and isolation, political climate, and time away from friends and family were also mentioned as stressful triggers. Lastly, the most common emotional response to perceived stressors were hopelessness, worry or anxiety, anger, and irritability.Symptoms of stress as measured by PSS scores between sessions one and two did not significantly decrease . However, there was a significant decrease in the depressive symptoms over time (P=.01). Finally, the average number of enjoyable activities selected by participants indicated that those with higher depression levels and who completed multiple sessions selected more enjoyable activities on average than those with lower PHQ-9 scores that completed only one session.There was a mean 20.0% (SD 35.2%) decrease in depression severity level, as measured by PHQ-9 scores between sessions one and two, which was not statistically significant  of participants who completed only one session and 60% (3/5) of those who completed multiple sessions of the mood program indicated that they either agreed or strongly agreed with the statement that “I would rather use a computer to help myself privately than go to a therapist,” indicating that a significant number of people want to use a computer-based or web-based approach for issues like depression. The acceptability and usability of the programs was good, and those who completed activities showed improvement in self-reported stress and depression. Despite all this, however, very few people returned to continue their work on problem solving or stress management after the first visit. One possible explanation for this is that, no matter how acceptable, usable, or effective a program may be, CBT requires time and effort to be successful. Just like other activities that provide benefits but require effort , maintaining the motivation to complete the programs is difficult. Programs like these likely need to be used within a supportive environment with a human touch that can provide encouragement and ongoing support .Data collected was uncontrolled since this was essentially a pragmatic trial during a time when there was heightened interest in the program. In contrast to previous studies of computer-based and online-based interventions evaluated in the setting of clinical trials, we analyzed anonymous data of a self-guided freely available program where users were self-referred.This evaluation of the PATH program has many limitations, with the most significant likely being the high attrition rate, which is consistent with other research on self-help interventions ,34,39-42Additionally, the data reported here are only from those who completed an activity in the program and then opted to complete a questionnaire, resulting in a selection bias. We do not have data from those who did not complete activities or who may have stopped because of difficulties with either usability or acceptability. The small sample size for those completing multiple sessions restricted our interpretations as to whether this program had a significant effect on reducing mental health symptoms. Given these limitations, any generalization from these results needs to be evaluated with caution. Nevertheless, the opportunity to evaluate a program like this during the COVID-19 pandemic was unique.The programs were initially developed for use at NASA, and the conflict module included examples and situations based in the space program. The high acceptability and usability ratings for the conflict program indicate that these examples could be understood and applied to the public outside of NASA. Nevertheless, feedback from some participants who completed the conflict module included questions regarding the meaning of some of the words used, such as “EVA” (extravehicular activities). The vocabulary included in some of the scenarios could have created a barrier, hindering the full understanding of the module.The sample was also not well characterized because, to maintain participant anonymity, only minimal data were collected about the users. This makes it difficult to gauge how confounding variables such as ethnicity, socioeconomic status, or education level would affect the results and attrition rate of this study. Additionally, the Dartmouth PATH program had a high exposure within the Dartmouth hospital and college campus, which could have contributed to a sample skewed toward well-educated individuals. These factors could affect the results and dropout rates. Those with more experience with computers may be more likely to continue with the program compared to those with lower education backgrounds and less technological capability . Future During the COVID-19 pandemic, those who came to the program seeking self-help for behavioral programs and engaged with the program rated conflicts with family or spouse as a major source of stress. Feelings of hopelessness, worry, anxiety, and anger were commonly reported. The problems selected for the mood program focused on worry, anxiety, overthinking, and stress. Despite the high attrition rate, this study shows that an open-access online behavioral program aiming to treat depression, stress, and conflict management can be effective and rated highly for usability and acceptability by users. This suggests the main issue is not content but instead finding the best implementation strategy. A significant proportion of users reported that they preferred to address behavioral health programs on their own using an online resource. Those who returned for additional visits tended to report more issues with worry, stress, and anxiety, and on average rated the programs less usable than those who completed just one session. For acceptability, there were differences in response patterns between the mood and stress module. Although those who returned for more visits in the mood module rated the program as more acceptable, the opposite trend was true in the stress module. The ultimate value of this program as a stand-alone resource will depend on understanding the reasons for the low completion rates and addressing them effectively. Strategies such as progress tracking, professional support, and weekly reminders may help increase users’ adherence to online programs . Attriti


Teen pregnancy prevention in the United States has traditionally focused on the development, testing, and subsequent implementation of a set of evidence-based programs (EBPs), recommended nationally. However, these existing EBPs often do not prioritize the most at-risk or vulnerable populations.3) project was funded to facilitate the development of new, innovative programs to reach disparate populations. Through a mixed methods design, iTP3 evaluated the process and resulting innovative programs from five iterative cohorts of funded organizations, referred to as Innovators. iTP3 utilized both a traditional funding model with more traditional methods of capacity building assistance, but transitioned over time to a design-focused funding model in which organizations and individuals developed innovative programs through an intensive human centered design process.The Innovative Teen Pregnancy Prevention Programs (iTPEvaluation results showed that the resulting portfolio of programs had differences in the types of programs resulting from the differing funding models. Notable differences among programs from the two funding models include program length, along with personnel, time, and resources needed to develop and manage.Both traditional and design funding models led to innovative programs, with notable differences in the development process and resulting programs. Federal support for teen pregnancy prevention and adolescent health programs has made significant progress in the last two decades. In 2007, the first list of evidence-based programs (EBPs) for teen pregnancy prevention (TPP) was released, which identified 15 programs shown to delay sexual debut, increase contraception use, and decrease teen births . These pAs of 2017, there are over 40 EBPs for TPP, which have contributed to the decrease in teen birth rates in the United States . HoweverCurrent TPP EBPs, which focus on reducing sexual risk behaviors including sexual initiation, number of sexual partners, frequency of sexual activity, contraception use, pregnancy, and sexually transmitted infections, prioritize limited underserved subpopulations . Many pr3) project, led by Texas A&M University, was funded in 2015 by the U.S. Department of Health and Human Services’ Office of Population Affairs (OPA) , to support and enable innovation in TPP through a 5 year funding period [The Innovative Teen Pregnancy Prevention Programs  to support their program development work. iTP3 defined programs as, “A systematic strategy (activity(s), policies, procedures, interventions, etc.) that can be, or are, replicated or repeated in one or more settings.” Twenty-seven unique organizations, across 5 cohorts, were funded through the iTP3 project from 2015 to 2020. Annual funding amounts ranged from $8500 to $100,000. Funding amounts varied because each year iTP3 altered its funding structure based on lessons learned. Initially, iTP3 utilized a traditional grantee model (cohorts 1 and 2), which identified recipients and provided funding for independent activities. Individualized CBA was provided by the iTP3 project team and external experts on a case-by-case basis, based on Innovator requests and needs identified by iTP3 personnel. Potential CBA focused on target populations, program design, implementation, evaluation, and other topics.The iTP3 funded organizations are referred to as Innovators throughout this manuscript. See Table 3 cohorts.Cohorts 3–5 utilized a design-focused funding model, which sought to build organizational capacity through developing skills in human centered design (HCD). HCD is a process that includes building empathy for the target population, generating ideas, developing prototypes, testing, and iterating products or programs with a goal of developing solutions to be implemented . Cohorts3 team as well as external subject matter experts. Group- and individual-level CBA was available each year but differed among cohorts. Innovators in the design-focused cohorts received much more intensive CBA in comparison to the more traditional cohorts as they were required to participate in trainings whereas the traditional cohorts only received CBA on an as needed basis. A detailed description of the iTP3 model of innovation will be available in other literature.The approach to support Innovators evolved based on continued quality improvement efforts. Common methods of CBA delivery included webinars, conference calls, online modules and resources, learning collaboratives, peer sharing calls, site visits, virtual meetings, and in-person trainings. CBA providers were determined based on Innovators’ needs and included members of the iTP3 project, a program evaluation was conducted. While the evaluation explored many aspects of the iTP3 project, the primary evaluation question aligned with the funding objective, which was: To what extent did iTP3 result in the development of a portfolio of innovative teen pregnancy prevention programs? The purpose of this paper is to report on that evaluation question and provide insight on innovative program development in public health.To understand the implementation and outcomes of the iTPN = 28) participated in various data collection activities, rather than having individual participants. Organizational representatives accounted for each of the Innovator entities funded through iTP3. These representatives consented to participate in evaluation activities on behalf of their organization, and all data collection tools were approved by the Texas A&M University Institutional Review Board prior to implementation. The evaluation was conducted by an evaluation team, internal to the iTP3 project.To capture the most relevant information, the evaluation was designed to be formative and process-based, utilizing a mixed methods approach . Due to the nature of this project, 28 organizations  and aided in updating the program model. Results are formatted to show the evolution of the project based on CQI processes.Throughout iTPSeventeen Innovators were funded across 2016–2019, using a traditional funding model. Each year, Innovators submitted an application for continued funding based on their progress and the viability of their projects. Four Innovators were engaged for the entire 4-year time period, 2 for 3 years, 2 for 2 years, and 9 for a 1-year period.Innovator programs represented a variety of priority populations including ethnic and racial minority youth; older teens; pregnant and parenting teens; lesbian, gay, bisexual, transgender, queer and questioning (LGBTQ+) youth; youth in foster care; youth with intellectual and/or developmental disabilities; depressed teens; homeless and runaway youth; and pre-service teachers. Programs also targeted ecological levels of intervention including interpersonal relationships, organizational interventions, community-based-intervention, and individual behaviors. As was expected in the initial funding of innovative interventions, not all programs were fully developed for testing. Rather some only made it through the initial stages of exploration or program development. See Table Consistent with a traditional funding model, all Innovators began the program with a defined program idea which they planned to implement. The planned programs were at differing stages of development; however due to the nature of the initial application, all Innovators had a defined target population and at least the start of what they perceived to be an innovative idea to build out. Beginning with Cohort 2, began to incorporate some HCD activities, concepts, and training with traditional Innovators. However, given their program development progress, HCD was used more similarly to CQI to iterate portions of the programs rather than develop full ideas.3. All Innovators had experience with TPP and most had previous experience with program development. They used this experience to address a gap identified from past programming, apply lessons learned from past programming to their iTP3 program, and apply or adopt a TPP program for a new priority population.All 15 funded Cohort 1 Innovators were interviewed regarding their program and their experience with iTP3 and CBA providers, and utilizing their organization’s embedded culture of innovation. Long-term, they hoped to have a program ready for a large-scale trial, improve adolescent health, have a new program component, and/or have a program that could be implemented in other communities.Additionally, all Innovators considered themselves to be innovative thinkers. Innovators planned to push innovation further through engaging partners, working with iTP3 program. These challenges were related to their 1) priority population, 2) ability to develop and disseminate a program, 3) organization, 4) advisory board, 5) partners, 6) ability to replicate the program, and 7) ability to plan for sustainability.Despite experience with both TPP and program development, Innovators identified seven primary challenges to their iTP3.Cohort 2 included eight total Innovators; six returning from year two and two new. All eight innovators completed interviews. Interviews assessed the program planning and implementation process, innovation, and successes and challenges experienced by Innovators. Overall, Innovators felt their core definition of innovation in TPP stayed largely the same but expanded, meaning that they still considered their target population and/or setting innovative, but they began to appreciate more the need to include new strategies, stakeholder input, and often a more holistic approach to solving the problem. Innovators reflected their overall long-term goals remained the same as indicated in their initial applications; however, their process of reaching the end goal changed in some cases. Innovators felt they made progress toward reaching their goals, but more work was needed. This is reflective of this iterative process and push for HCD and systems thinking that was a core aspect of iTP3 to alter the typical individually focused TPP metrics to focus more on the program development process and encompass non-individually focused interventions. This flexibility from the top allowed project teams a unique opportunity to focus on development and truly meet the needs of their priority population, as opposed to being held to strict, unchangeable metrics and ideas initially proposed. While Innovators were required to develop work plans at the beginning of their project period, it was emphasized that the work plan was an iterative document and should change based on lessons learned, target population input, data, and other relevant factors.There was recognition that innovation is a process rather than an outcome. Innovators recognized that innovation required flexibility and openness to making changes, as well as collaboration with others. In addition to reflecting on their particular programs, Innovators expressed appreciation for the flexibility of the funding model as the ability to make changes and iterate programs is crucial for development of an innovative approach. From the beginning of the grant, OPA allowed iTP3 CBA). Overall, these Innovators appreciated the flexibility of this funding model and the opportunity to develop an innovative program through a mechanism that allowed for iteration based on feedback and results. They felt iTP3 and innovation funding in general was important to advance the field through reaching underserved populations, fostering cross-sector collaborations, community relevance, and the opportunity to look at problems in new ways and develop creative solutions.In year four, seven Innovators, all funded the previous year, continued another year of program development. All seven of these Innovators were interviewed, once again. Key findings built upon findings from previous interview iterations. Innovators felt their definition of innovation and how their program was innovative had not changed, but rather solidified throughout the past year. They were still largely working toward the same goal(s) identified in their initial interviews in year two or three (depending on Innovator), but in some cases had expanded their goal to incorporate new elements and reflect lessons learned and/or priority population feedback. Continuing Innovators largely identified HCD and systems thinking as useful and were able to incorporate it broadly into their program development, but several felt the concepts would have been more useful if introduced during their first funding cycle  was funded concurrent to cohort 2 of the traditionally-funded Innovators. The purpose of this new iteration was to give organizations an opportunity to develop a new idea through an intensive HCD training process rather than beginning with an already defined idea. This gave the Innovators the opportunity to fully utilize HCD throughout the entire program development process. The use of HCD earlier in the process ensured incorporation of multiple levels of feedback including target population, organizational members and leaders, community members, and others who were relevant to the process. This design process was further iterated in subsequent years.Cohorts 3 through 5 (2017–2020), consisted of three separate models for facilitating innovation through intensive HCD training. Cohort 3 was considered a pilot year for the design funding model, as it overlapped with the traditional model. The design funding model provided intensive CBA in HCD, in an effort to facilitate the process of innovation among program developers.3 personnel facilitated the intensive HCD process to develop innovative ideas beginning with completing ethnography to gain insights about the target population before embarking on a process of multiples rounds of diverging and converging ideas through guided design activities to ultimately converge on one idea for each organization. Each of the four organizations pitched their program idea at the end of the Design Bootcamp, in a competition for program funding. Pitches were judged by iTP3 personnel, as well as community stakeholders who had knowledge of community needs and resources. The winning Innovators were given a small amount of funding to develop their program idea over a 6 month time period. They were required to have monthly check ins with an iTP3 liaison and report on program development and/or implementation process.Cohort 3, a Design Bootcamp, was the first iteration of the design funding model. The Design Bootcamp was created to both teach Innovators the principals of HCD and facilitate HCD over a four-day period, resulting in an innovative program. In total, two Innovators were funded as a result of one Design Bootcamp in which four organizations participated in a four consecutive day Bootcamp. iTPThe two programs funded through the Design Bootcamp both targeted rural populations in the same community through community-based (out of school) interventions. However, the interventions were quite different. One intervention entailed adapting an existing HIV/AIDS curriculum for pregnancy prevention, using a community health worker model. The other focused on the creation of a sexual and reproductive health (SRH) game for middle school students. See Table 3 team developed four- and two-day HCD workshops to accommodate different schedules through a Design Bootcamp model. To further address this challenge, iTP3 transitioned to a Design Sprint model in which full-day design sessions were separated by several weeks. The time between sessions was intended to allow for organizations to further develop their idea and gather feedback between sessions.Throughout the process, it was quickly evident that requiring a full week for design work was an unrealistic expectation for many professionals. For this reason, the iTP3 personnel. Therefore each of the Design Sprint funded organizations convened their own Design Sprint focused solely on the local community, in which multiple stakeholder organizations collaborated to develop a single, community-specific innovative idea. The agenda for each Design Sprint differed based on the progress and needs of the collaborative group as HCD includes an array of activities designed to facilitate innovative development. This differed from the Design Bootcamp in that organizations within a community did not compete with one another, but rather collaborated with one another.Cohort 4, the Design Sprint model, utilized much of the same HCD methodology as the Design Bootcamp  but was divided into multiple one-day workshops over the course of several months. Organizations were supported to bring together teams of people within their local communities to participate in the Design Sprints, facilitated by iTPFour distinct programs resulted from the Design Sprints in four very different communities. The resulting programs focused on training the staff of youth serving organizations, a music-based program for young males in juvenile justice settings, adult caregivers and adolescents through SRH videos, and creation of a guide for policy development for communities and schools. See Table Interviews were conducted with representatives from all four Cohort 4 organizations. Cohort 4 Innovators felt that HCD and systems thinking were useful in their program development and appreciated the opportunity to bring together a diverse group with varying perspectives. Because of the fluidity and intentional flexibility of this model, each Design Sprint team participated in different activities and a process tailored to their community. For this reason, it was more difficult to identify distinct themes across Innovators. However, these Innovators recognized the need to involve stakeholders and gain priority population input, and felt that the skills gained through the Design Sprint process were not only valuable in this project, but also transferrable to other projects and could be used throughout their broader organizations.3 team, documented design activities completed and notes on the design session including process, engagement, reflections, pivot points, and lessons learned. In reviewing the reports, the iTP3 team recognized a challenge in traveling to communities to conduct Design Sprint sessions, and the desire for community members to be equipped to lead design activities and use HCD strategies in other aspects of their work. The team decided it would be more feasible to train community members to be design leaders that conduct Design Sprints with stakeholders in their community, rather than iTP3 traveling to communities and leading design sessions. This would increase the number of communities served and sustain the use of HCD beyond iTP3-funded projects.After action reports, completed by the iTP3 approached its final year, the team added a focus on sustainability of not just programs, but also equipping individuals at the local level to use HCD and systems thinking methodologies beyond this project. For this reason, Cohort 5 shifted to a Design Cadre model in which individuals convened for three in person trainings in which Design Cadre participants were trained to lead design sessions, similar to the Design Sprint process, within their own organizations or communities. Additionally, this model of training facilitators lessened the travel burden on iTP3 team members as the iTP3 team did not have to travel repeatedly to all sites to facilitate design sessions.As iTP3 team. This served to increase program design capacity and sustainability within communities. Eleven individuals were funded to participate in the Design Cadre to receive in person and virtual training. Following the training, they were expected to facilitate a Design Sprint consisting of a minimum total of 20 h of design work to develop a program idea within their organization or community. In total, ten organizations were represented in the design process as two individuals represented one organization. Following Design Sprint sessions conducted in communities, Cadre participants were asked to present or ‘pitch’ their initial program idea to the iTP3 project team. After the initial presentations, ten individuals (nine organizations) were invited to further iterate their idea and prepare a full pitch to be presented at one final in-person meeting, for a chance at receiving funding to pursue implementation of their program idea. Based on the final pitches, eight organizations were provided funding in varying amounts to continue program development activities.The final iteration of Design Funded Innovators made the transition to empowering individuals to lead the design process within their own communities rather than relying on the external facilitation of the iTPOf the nine programs that were pitched, the programs focused on nine separate priority populations. Examples of priority populations identified for these programs include homeless and runaway male youth, School Based Health Center Youth Action Councils, health educators, and youth of varying age ranges. Settings represented rural and urban environments, community-based settings, school-based settings, and virtual settings. The program ideas were formulated and proposed, but not fully developed and executed due to the format of the cohort, timeline barriers imposed by the onset of COVID-19, and community-based restrictions enforced soon after the programs were funded for development. See Table Interviews were conducted with 10 of the 11 Cadre participants in Cohort 5. Interviews focused on the participant’s perceptions of the design process, the resulting program, innovation, and other relevant domains. After the Cadre experience, most felt prepared to lead design sessions to facilitate innovative program development in their communities with half of participants making recommendations for additional training needs to help with full program development. All participants considered the in-person group trainings to be the most beneficial part of the experience. Within the in-person group trainings, participants identified experiencing the design methods, practicing facilitation, and one-on-one coaching as the most valuable aspects.Cadre participants were asked to share the most important characteristics for innovation. The most common response was the need to have appropriate people involved in the design process including relevant stakeholders and end users, and ensuring that those involved are committed to the process. Participants shared that having the appropriate time to dedicate to the process and capacity to conduct the sessions was very important from a logistical perspective. Support for the process was also highlighted by participants particularly noting the importance of having buy-in from leadership, teams and the overall organization. Additionally, broad knowledge of design skills for both the facilitator and team was noted as important, as well as other individual characteristics including being willing to try something new and fail, having flexibility to change course, and being able to step back and listen.3, formal and informal data were used to refine and iterate the program model using CQI, with the goal of facilitating the development of innovative TPP programs. The resulting iTP3 evaluation provided a comparison between a traditional model of funding innovative ideas versus investing in building the capacity of local individuals and organizations to facilitate the development of innovative ideas.Throughout the five-year project period for iTP3, a total of 27 organizations were funded, with 29 programs emerging and reaching varying levels of program development. Twelve organizations received funding for multiple years, ranging from two to 4 years. Programs with potential to impact the field of TPP and adolescent sexual and reproductive health emerged from both funding models. However, equipping local individuals with skills to facilitate innovative program development was the model that ultimately resulted from years of iteration. iTP3 recognized this funding as a unique opportunity to fund new types of program development whereas other funding streams were already supportive of more traditional program development methods that started with a defined idea and progressed on a somewhat linear path through development and testing. While iTP3 was able to see the initial stages of both funding models, the timeline of the grant did not allow for many fully developed ideas that could be rigorously tested. Additionally, due to the nature of innovation, many of the resulting programs are not structured in a way to fit typical methods for rigorous evaluation. For this reason, we are not able to definitively say which model was more successful if success is defined as programs ready for rigorous testing. In order to fully test this new cohort of programs, there will need to be changes to move away from the traditional individually-focused expectations for evaluation. In total, three programs planned under the traditional funding model either received funding or applied for rigorous testing. However, none of the innovation focused funding model programs were prepared for this step. Many of these programs continued to be developed, still contributing to the portfolio of innovative programs. For this reason, it was important to look at other factors in determining potential success and impact of programs.In looking at the overall portfolio of innovative programs that was developed through iTP3 program addressed the dimensions of 1) participation and leadership, 2) skills, and 3) resources. With this, the local facilitator collaborates with a partner organization or network of partners engrained in the community to develop sustainable, relevant programs, both now and in the future through maximizing these and other dimensions of community capacity.Goodman and colleagues identified ten domains of community capacity that can be applied in this model of equipping individuals to be better able to facilitate within their communities . Through3 decided to continue funding for promising interventions from the traditionally funded model rather than abandoning the interventions when the direction of the overall program shifted to a design-focused model. These organizations were already equipped to carry out programs, but benefited from having iTP3 funding as an opportunity to deviate from traditional planning methods, typically employed within their organizations.As a team, iTP3 progressed and new program development iterations were utilized, the type of programs developed changed. In the beginning, programs were highly structured, congruent with the higher funding level and traditional style of application. Some of the examples of innovative programs that emerged from the traditional funding model are as follows. Through 3 years of funding, iTP3 funded a(n):Multi-level program to engage homeless and runaway youth with a comprehensive program that includes sexual and reproductive health (SRH), including access to long-acting reversible contraceptives (LARC) in addition to wrap around services.Program working to provide capacity building through a multi-level approach to training foster care professionals, creating organizational policies and practices that support SRH education and referrals, and enhancing the physical environment of foster care agencies with SRH materials and messages.Intervention for depressed young women utilizing a texting modality to provide real-time support and access to information when the information was most needed. Following iTP3, the organization received funding for rigorous testing.As iTPIn Cohort 3, the Design Bootcamp cohort, one organization developed a SRH game for middle school students. After several iterations of the prototype, the resulting game encompassed four games focused on knowledge and skill development regarding anatomy, healthy relationships, accessing information, and building connections with trusted adults to be equipped for navigating future relationships. This intervention was fully prototyped, pilot tested with several new communities, and ready for rigorous testing and scaling.Through Cohort 4, the Design Sprint cohort, one team developed a program idea to formulate a policy guide for communities and schools. The goal of this guide was to develop a tool that organizations can use to navigate the complexities of policy development and equip practitioners with best practices and tips for working through the process.In Cohort 5, the Design Cadre cohort, one organization developed an innovative health education experience utilizing a youth-friendly mobile unit to provide SRH information in a comfortable, easily accessible way. This mobile unit can travel to schools, community locations, or wherever there is a need. This program was funded for the initial development, but with the onset of COVID-19 was not fully built out prior to the end of funding.In addition to the programs described above, a different set of programs emerged from the design-focused cohorts with a few notable differences. These programs were awarded less funding and the funding that was distributed was based on the design process. A few examples of the programs that emerged are described below.While both types of funding models resulted in the development of innovative programs, there were notable differences. First, based on the amount of funding and the guarantee (or lack thereof) of future funding, the programs differed in scale. The programs described under the traditional funding model take more of a comprehensive approach and require much more personnel time and other resources to manage. Conversely, the design funding model resulted in programs that could be implemented and built out in phases based on funding availability. Additionally, the design-focused funding model allowed for smaller scale testing and “failure” of program components, as is typical of a design thinking process. This ensured that large amounts of time and money were not wasted on programs that were not innovative or effective for the target population.3 funding and technical assistance through both the traditional funding model and the design funding model. However, it is unclear whether the programs developed will be successful in reducing teen pregnancy or improving adolescent health as the programs are currently in the process of pursuing opportunities for rigorous testing. Several programs funded under the traditional funding model are ready for rigorous testing and have the appropriate personnel and preliminary data to do so. While the design funding model resulted in the development of innovative programs, these programs may not fit the criteria for rigorous testing in the traditional manner.Overall, the data indicates that innovative programs were developed as a result of iTPThis study gleans implications for funders, researchers, and practitioners in behavioral health. First, funders can utilize the information gleaned from this study to better tailor their resources to support, advocate for, and invest in innovative behavioral health projects utilizing efficient development processes, such as HCD , 20. WhiNext, researchers should partner with groups developing innovative programs to study and compare the process and strategies used in the development process. Such understandings could inform future program development efforts to be more efficient and cost-effective, while maintaining forward momentum. Additionally, researchers should be at the forefront of innovation funding so that research and evaluation methods can be adapted to accurately measure innovative programs. In this case, the program model and results presented here show basic adaptations in assessment which can be expanded upon in future research to ensure that innovative programs are measured according to their program characteristics rather than adhering solely to the individual-based outcomes that are traditionally used to measure TPP EBPs.Lastly, practitioners interested in developing innovative programs can apply the lessons learned to their own work and embrace a HCD approach to develop more meaningful programs and services with their end-users. The use of HCD in this manner shows the potential to adapt business methodologies or other practices used in other fields for relevance and advancement in health. While this paper specifically makes the case for use of HCD and to a smaller extent systems thinking, openness to non-traditional methodologies can improve the interdisciplinary nature and effectiveness of health programs.3, two major unforeseen and unavoidable events limited or changed the direction of progress and should be noted.During iTP3 and all other OPA grantees in this Tier of funding received notice that funding was cut and the five-year project would end at the end of year three . OPA grantees did not receive notice the funding had been reinstated until June 2018; therefore, the iTP3 team was unable to plan for and begin year four recruitment and selection until the project year was already underway.At the start of year three, iTP3 team was not able to fully see how the programs would be carried out, as intended. Additionally, follow-up with past Innovators was limited.During year five, the COVID-19 pandemic halted the ability to complete a final systems thinking training with Design Cadre Innovators and also limited the ability of Cadre Innovators to start or continue the development of their proposed innovative program. For this reason, the iTP


The family environment plays an important role in the development of children’s energy balance–related behaviors. As a result, parents’ energy balance–related parenting practices are important targets of preventive childhood obesity programs. Families with a lower socioeconomic position (SEP) may benefit from participating in such programs but are generally less well reached than families with a higher SEP.This paper describes the application of the Intervention Mapping Protocol (IMP) for the development of an app-based preventive intervention program to promote healthy energy balance–related parenting practices among parents of children (aged 0-4 years) with a lower SEP.The 6 steps of the IMP were used as a theory- and evidence-based framework to guide the development of an app-based preventive intervention program.Samen Happie! through a process of continuous cocreation with parents and health professionals. In step 5, community health services were identified as potential adopters for the app. Finally, in step 6, 2 randomized controlled trials were designed to evaluate the process and effects of the app among Dutch parents of infants  and preschoolers . These trials were completed in November 2019  and February 2020 .In step 1, behavioral outcomes for the app-based program  and sociocognitive  and automatic  determinants of energy balance–related parenting were identified through a needs assessment. In step 2, the behavioral outcomes were translated into performance objectives. To influence these objectives, in step 3, theory-based intervention methods were selected for each of the determinants. In step 4, the knowledge derived from the previous steps allowed for the development of the app-based program Samen Happie! to promote healthy energy balance–related parenting practices among parents of infants and preschoolers. Through the integration of theory, empirical evidence, and data from the target population, as well as the process of continued cocreation, the program specifically addresses parents with a lower SEP. This increases the potential of the program to prevent the development of obesity in early childhood among families with a lower SEP.The IMP allowed for the effective development of the app-based parenting program Netherlands Trial Register NL6727, https://www.trialregister.nl/trial/6727; Netherlands Trial Register NL7371, https://www.trialregister.nl/trial/7371. Although childhood obesity rates have been reported to stabilize in many developed countries , the prenormal child body weight development [Parents are considered the main agents of change in effective preventive intervention programs for childhood obesity, especially in the first years of life . Parentselopment . Moreoveelopment . In geneelopment . For phyelopment ,15. Howeelopment . More geelopment . Moreoveelopment ,16. For elopment . Our parelopment ,18 whileelopment .In addition, our program uses an innovative approach to address 2 frequently reported limitations of traditional (parent focused) obesity prevention programs: the costly, time-intensive face-to-face setting ,20 and dSamen Happie! using the Intervention Mapping Protocol (IMP). The IMP is a widely used, standardized intervention planning format that helps intervention developers to incorporate empirical findings from the literature, effective behavioral change methods and their practical applications, and data collected in a representative population [Samen Happie! is to stimulate healthy energy balance–related parenting practices to prevent early childhood obesity among children of families with a lower SEP. In this paper, we inform readers about the development of the Samen Happie! program based on the 6 steps of intervention mapping.Therefore, we developed the app-based prevention program pulation . Althougpulation -41. The Samen Happie!) will be used throughout this paper. A possible translation is “Happy Together,” but this does not reflect the play on words the title indicates in Dutch.The Dutch title of the app-based parenting program (Samen Happie! program was developed through the 6 iterative and nonlinear steps of the IMP: (1) conducting a needs assessment; (2) preparing matrices of change objectives; (3) selecting theoretical methods and practical strategies; (4) developing the intervention program; (5) planning for adoption, implementation, and sustainability; and (6) planning the program evaluation [The aluation . We collThe first step of the IMP was to conduct a needs assessment of our target group  to build a logic model of the health problem . Our neeIn the second step of the IMP, the performance objectives were defined for the behavioral outcomes specified in step 1. These performance objectives constituted behaviors that are expected to contribute to achieving the program goal when performed by the target group . By crosThe third step of the IMP evolved around the selection of theory- and evidence-based change methods to affect the determinants selected in step 1. We aimed to select a limited number of theoretical methods per determinant as interventions that use a small number of behavior change techniques are generally more effective for people with a lower SEP than interventions that use a larger number of techniques . We thenThe fourth step of the IMP involved building the program themes and components and drafting, pretesting, and producing the program materials based on the information gathered in the previous steps . The devSamen Happie! app. This is one of the reasons that digital health programs in particular need a delivery system  to get the program to its intended participants [The fifth step of the IMP involved the identification of potential program users  and the design of a program implementation plan . We planicipants .The sixth and last step of the IMP involved the development of a program evaluation plan . We deveOur logic model of the health problem is presented in Ample research has established the intake of energy-rich foods and sugar-sweetened drinks -49 and lParenting practices are broadly divided into 3 overarching dimensions of food parenting  that canBoth general parenting and parental well-being may moderate the associations between parental energy balance–related parenting practices and child EBRBs. With respect to general parenting , researSamen Happie! program, with respect to performing healthy parenting practices related to children’s EBRBs, the following determinants are targeted: parental knowledge, attitudes, self-efficacy, and habitual behavior .For the selection of determinants, we were informed by the results of our focus groups and the empirical literature. Moreover, the I-Change model  was usedKnowledge plays an important role in changing EBRBs and is a basic component of existing preventive intervention programs for childhood obesity . It is pKnowledge should be targeted by carefully considering the beliefs of the target group . Some paSelf-efficacy refers to a parent’s beliefs in their capabilities to organize and execute a course of action  in particular situations . In geneParents often report a discrepancy between what they intend to do in terms of energy balance–related parenting practices and what they actually do . For insnonconscious methods might be particularly effective for people with a lower SEP as these methods are less dependent on literacy capabilities [The theoretical methods we selected for each of the determinants  were derived from the taxonomies described by Kok et al  and Michbilities .Maria. Try to eat healthy when she is around”). Finally, the data on children’s age were used to present parents with developmentally appropriate information via age-based modules (see Step 4: Developing the Intervention Program), which could enhance the perceived relevance of the information, thereby increasing engagement [In addition to potentially important theoretical methods, the form in which an intervention is delivered is also a key ingredient in behavior change interventions  and mighgagement .In phase 1 of the app development process, the results from our focus groups and discussions with youth health care professionals informed the development of the prototype of the app and the way in which we tailored the materials to our target group. For instance, based on parents’ negative affective attitudes toward water consumption, we asked them whether they would provide their children tea (without sugar) as an alternative, and they affirmed that they would do so. Therefore, we drafted the app content with “water or tea,” instead of focusing entirely on water consumption. Other examples of content that we based on the suggestions of parents from our focus groups included providing information about saving money on groceries and tips to stimulate vegetable consumption.In phase 2, the content of the preventive intervention program was drafted based on national health care guidelines, relevant literature on parenting practices in relation to child EBRBs, and previous intervention projects ,114. ForThe final version of the app (phase 3) was launched in September 2018. Each of the 5 modules consisted of 2 types of activities: lessons and challenges. Screenshot 2 in Healthy Living Desk, an intervention database of the Dutch ministry of Public Health, Welfare and Sports. To facilitate nationwide dissemination and aid in intervention sustainability, the Samen Happie! program was submitted to this database and accepted in September 2020.We identified national community health services and child day care centers as potential adopters of the app-based program, as representatives of these organizations were involved in the development and execution of the program. The program implementers will be youth health care professionals and pedagogical staff working at these organizations, and their tasks are to bring the app to the attention of parents (whose children they perceive as being at high risk for obesity) and motivate them to use the app. The implementation of the app-based program can support the daily practice of the program implementers in 2 ways. First, the app could function as an addition to usual care, given that youth health care professionals indicated that standard consultations are generally too brief to give parents elaborate, well-rounded advice. In this sense, it is advantageous that the app is an easy-to-use product that does not require detailed instructions from a health care provider. Second, the app could function as an educational material that creates a legitimate opening to discuss topics such as food parenting and body weight , which hThe process and effects of the app-based preventive intervention program were evaluated in 2 separate trials, the designs, eligibility criteria, procedures, and measures of which are explained hereafter.Samen Happie! app and a waitlist control condition. In addition, trial 2 included a third condition in which parents received the app and 2 additional group sessions organized at locations where their child attended preschool. The third condition was a separate condition that did not interfere with the procedures of the RCT, as presented in this manuscript. Detailed information about the development and evaluation of these group sessions has been described elsewhere. Hence, this manuscript will concentrate on the intervention conditions in which parents received access to the app-only and control condition. The parents in both of these conditions completed a web-based baseline measurement (T0) and follow-up measurements at approximately 6 months (T1) and 12 months (T2). Both trials were randomized controlled trials (RCTs) with 2 parallel arms: an intervention condition in which parents received access to the A power analysis using G*Power (version 3.1) indicated a minimum of 200 participants in trial 1. This calculation was based on child BMI as the outcome variable, which was assumed to have a mean of 16.67 (SD 1.70) . We furtIn the end, the participants in trial 1 were 357 parents  of infants aged 5 to 15 months at baseline (ages corresponding to modules 1 and 2 of the app). Trial 2 was conducted among 153 parents  with toddlers aged 18 to 55 months at baseline (ages correspond to modules 4 and 5).To assess whether parents were eligible to participate in the trials, they completed a web-based screening that contained questions about their educational attainment and their child’s age and health status. Parents were respectfully refused participation when their child was younger than 5 months or older than 15 months , younger than 20 months or older than 55 months , or when their child had a chronic disease or disability that affected normal development. Parents with multiple children could only participate with one child and only in 1 of the 2 trials. We strived to include at least 50% of parents with lower or medium-level SEP and used educational attainment as a proxy for SEP . Parents with higher educational attainment  were not actively discouraged from participating in the trials.Parents were recruited offline  and online , for which we particularly considered locations that are often visited or used by parents with a lower SEP. Interested parents who fulfilled the eligibility criteria received an email in which they were asked to provide consent for their participation. After consenting, the parents were forwarded to the web-based baseline questionnaire. Randomization for each trial took place after the baseline measurement by means of a simple randomization procedure performed by an independent researcher using SPSS version 24. Among research with large sample sizes, this procedure can be trusted to produce equal samples in terms of numbers and covariates . ParticiTwo types of process evaluation data were collected: self-reported data and app user data.We assessed parents’ self-reported app use, their user experience of the app, and their suggestions for app improvement. In the 2 follow-up questionnaires (T1 and T2), we asked parents whether they downloaded the app (and why), whether they still had the app installed on their phone (and why), and how many times they used the app. Regarding user experience, we asked parents to rate several indicators of functionality , design, and content  on a scale from 1 (bad experience) to 7 (good experience). Parents also rated the app as a whole on a scale from 1 to 10, with higher scores indicating higher appreciation. Finally, we asked open-ended questions about the ways in which the app could be improved.Preliminary analyses of parents’ self-reported app evaluation data on the first follow-up measurement (T1) showed that most parents in the intervention condition of trial 1  and almost half of the parents in the app-only intervention condition of trial 2  reported that they downloaded the app. Most of these parents  indicated that they had used the app multiple times since installation but were not using it anymore at T1. Around a quarter of the parents  indicated that they still used the app multiple times per month. Parents in both trials generally appreciated the functionality, content, and design of the app. They graded the app with an average score of 6.7 (SD 1.45) in trial 1 and 7.2 (SD 1.05) in trial 2. The most important suggestions parents gave for improvement of the app included the incorporation of more detailed and elaborate parenting information, a clearer structure of the presented information , the option to look for specific information through a search function, and the integration of other parents’ perspectives and experiences .favorite, and their answers to quiz questions.In addition, to objectively assess parents’ exposure to the preventive intervention program, their app usage was automatically monitored and collected in an online database. This database allowed us to examine the lessons and challenges that parents started and/or finished, the specific lesson cards they saved as The primary outcome measures of the effect evaluation were EBRBs of the child , weight-for-height z scores , and BMI z scores  and parents’ parenting practices related to their child’s EBRBs. The following secondary outcomes were assessed: parents’ general parenting style, parental well-being , and parents’ EBRBs . An overview of the constructs, variables, and assessment points addressed in the evaluation of the program can be found in Child weight-for-height and BMI z scores were calculated using height and weight data reported by the parents in the questionnaires. We asked parents to draw this information from the measurement overview in the child’s personal  file, which is updated by the youth health care professional each time the parent and child visit the child health clinic. During the second follow-up questionnaire (T2), we additionally asked parents to send us a picture or screenshot of this measurement overview. This strategy not only allowed us to compare the information parents provided in the questionnaires with that in the child’s file but also allowed us to collect more detailed anthropometric data as the overview contains height and weight measurements from the moment of birth to present day. Moreover, in the second follow-up questionnaire, we asked parents for their permission to be contacted again 12 months and 48 months after T2, so that they could send us a picture or screenshot of the updated measurement overview. This information allowed us to examine the effect of the preventive intervention program on the BMI of the child until approximately 2 years after the last follow-up measurement.The trials were completed in November 2019  and February 2020 . We are currently in the process of data cleaning. Effect analyses are thus underway, and the first results are expected to be submitted for publication in 2021.Samen Happie! was developed primarily for these families and aims to stimulate healthy energy balance–related parenting practices from early childhood, before unhealthy energy balance–related habits have been established. More specifically, the program promotes both structured and autonomy-supportive practices and limits coercive controlling parenting practices with respect to all relevant energy balance–related determinants of childhood obesity , with the ultimate goal of preventing children aged 0 to 4 years old from being overweight and obese. The successful development of the program was aided by the use of the IMP. The process of stepwise decision making made this large-scale and complex project manageable and contributed to thorough considerations. Regarding the selection of eminent EBRBs, for instance, we initially selected only parenting practices related to the children’s dietary intake but we decided to also include practices relating to sleep, physical activity, and screen time after an extensive literature search and discussions with the target group. Moreover, insights from recent literature reviews facilitated the selection of the most promising energy balance–related parenting practices . By facilitating a collaboration with experts and the target group, the IMP assured that the intended end users of the program were involved in multiple stages of program development. Overall, by integrating theory, empirical studies, professional knowledge, and the needs and preferences of the target group through continuous cocreation, we increased the chances of developing an effective preventive intervention program for childhood obesity [The need for effective preventive intervention programs for childhood obesity is high, particularly among families with a lower SEP. The app-based parenting program  obesity .Samen Happie! program is its focus on both sociocognitive  and automatic  determinants, bridging the well-known gap between health parenting intentions and behaviors [Besides the use of the IMP, the app-based parenting program has several other notable strengths. First, previous digital preventive intervention programs for childhood obesity focused solely on the sociocognitive determinants of energy balance–related parenting . A uniquehaviors . Furtherehaviors . By usinSamen Happie! website [Samen Happie! program improve by offering them access to both the website and the app.One of the challenges of mobile health interventions is to keep users engaged for longer periods, which is particularly important for interventions targeting behavior change maintenance . Althoug website  has beenThe app-based prevention program also has some limitations. With respect to the design of the trials, it was not possible to blind both participants and investigators to the allocation of conditions . As we used a waitlist control condition, the participating parents knew they would eventually receive an app about healthy parenting. However, neither the participants nor the investigators knew which trial condition the participants would be allocated to before randomization took place. Although double blinding in RCTs is generally recommended, methodological studies have shown that adequate allocation concealment is most important in minimizing bias ,123. Morterrible twos and food neophobic phase [In addition, future preventive intervention programs for childhood obesity should consider involving both caregivers. Although we intentionally targeted only primary caregivers (who turned out to be primarily mothers) for the recruitment, program materials, and questionnaires of our program, recent research has indicated that parents within a family differ in the energy balance–related parenting practices they perform . This hiic phase ,126, espic phase ) counselic phase .Samen Happie! to promote healthy energy balance–related parenting practices among parents of infants and preschoolers. By applying the IMP, including continued cocreation, the program specifically addressed the needs of parents with a lower SEP through a tailored program content and through theory-based behavior change techniques. This increases the potential of the program to prevent the development of obesity in early childhood among families with a lower SEP.In conclusion, the IMP allowed for effective development of the app-based parenting program