Produce prescription programs aim to improve food insecurity (FI) and nutrition but their effectiveness is unclear. We conducted a pilot study to demonstrate the feasibility and explore the potential impact of a family-based, home-delivery produce prescription and nutrition education program. We measured enrollment, satisfaction, participation, and retention as measure of feasibility. Adult participants answered pre-post self-report questionnaires assessing FI, child and adult fruit and vegetable intake, and culinary literacy and self-efficacy. To understand participants’ lived experiences, qualitative interviews were conducted at the 6-month time point. Twenty-five families were enrolled. Feasibility measures indicate participants were generally satisfied with the program but there were important barriers to participation. Qualitative data revealed themes around reduced food hardship, healthy eating, budget flexibility, and family bonding. Fruit and vegetable consumption increased in a small subgroup of children, but post-intervention intake remained below recommended levels, particularly for vegetables. FI scores were not significantly different post-intervention, but qualitative findings indicated improved access and reliability of food. This is the first intervention of its kind to be evaluated for feasibility and our results suggest the intervention is well-received and supportive. However, further study, with a larger sample size, is needed to understand factors influencing participation and assess effectiveness. Almost 15% of US households with children reported experiencing food insecurity (FI) in 2020 , reducedAssociated with FI, eating a diet low in fruits and vegetables places individuals at increased risk of heart disease, obesity, and certain cancers ,12, all Produce prescription programs have been implemented in primary care settings to address FI and diet quality and many targeting adults have been evaluated for feasibility and impact ,26. Few The Children’s National Hospital Family Lifestyle Program’s (FLiP) Home-Delivery Produce Prescription Initiative (FLiPRx) aims to address FI and diet-related disease risk in families with young children living in under-resourced settings in Washington, DC and its surrounding metropolitan areas. Many families in Washington, DC experience disproportionately high levels of FI, health inequities, and severely limited access to healthy foods . The FLiFLiP is a family-centered, clinical-community collaborative started in 2017 at Children’s National Hospital (CNH) in Washington, DC alongside the Young Men’s Christian Association (YMCA) of Metropolitan Washington, and the American Heart Association (AHA), to better address FI and diet-related chronic disease. This collaborative body, in partnership with 4P Foods, a local food hub, designed the FLiPRx program alongside community stakeholder support and guidance. In November 2020, participants were recruited from two CNH outpatient clinics located in Washington DC’s most historically disadvantaged and under-resourced areas. During well child clinic visits, adults who presented with children aged 0–5 years old were universally screened by clinicians for FI with a 2-question screener  . Adults The intervention consisted of two parts: (1) fresh fruit and vegetable delivery; and (2) virtual nutrition education. For the food delivery, enrolled families received a 12-month supply of bi-weekly  delivery of fresh produce. The produce delivery was approximately 8 pounds of seasonal, locally sourced, fresh fruits and vegetables (adjustable by request). Families received texts from the delivery team when their box of produce was on its way and again when it was dropped off at the doorstep or preferred location. For the educational component, our virtual evidence-based nutrition education was based on the SNAP-Ed curriculum and was culturally tailored to our target audience. The culturally tailored curriculum prioritized shared norms, beliefs, expectations, and behaviors common in our local predominantly African-American community. We aimed to establish racial concordance  and highLastly, the design of our program aimed to address multiple barriers to healthy eating associated with FI. The large amount of local produce provided aimed to address not only the quantity of food available in the household but the quality, thereby supporting the notion of enhancing the nutrition security of families and supporting the economic investment in local community. The home delivery of food ensured reliability in an urban community where transportation and full-service grocery stores are limited. In addition, we aimed to increase nutrition and culinary knowledge using culturally tailored education delivered by trusted community-based partners and clinicians.We conducted a pilot mixed-methods longitudinal behavioral intervention to assess the FLiPRx program design and feasibility and to explore impact. This was a quasi-experimental approach without a control, using a pre-post quantitative evaluation and qualitative interviews. We chose the mixed-method design because qualitative assessments are a valuable tool to gain a better understanding of the lived experiences of participants and to put the quantitative findings into perspective.Quantitative measures used to assess feasibility were rate of program enrollment, retention, satisfaction survey response rate, and participant-reported satisfaction with program content. Quantitative measures to assess impact were pre-post questionnaires with questions derived from standardized measures of FI, eating behavior, home environment, and culinary and health self-efficacy. Adult participants completed the self-report survey at baseline and the same survey again at the 12-month conclusion of the program. Surveys were conducted online using REDCap electronic data capture tools . QualitaInclusion criteria for eligibility were: (1) positive FI status: families must screen positive for being “at risk” for FI by HVS screen, (2) reference child aged 0–5 years old, (3) positive diet-related chronic disease risk factor, and (4) willingness to complete monthly surveys. Chronic disease risk was defined for reference children younger than 2 years old as having any of the following criteria: abnormal weight gain (>2 standard deviation (SD) increase in growth on either weight-for-age or weight-for-length based on The World Health Organization (WHO) growth charts within the last 1 year), overweight status (weight-for-length ≥97.7% WHO growth charts), or adult caretaker has reported a diet-related chronic disease such as hypertension or high blood pressure (BP), pre-diabetes, diabetes (DM), heart disease, stroke, or obesity. Disease risk was defined for reference children older than 2 years old as any of the following: abnormal weight gain , overweight or having obesity (BMI >85th%), elevated BP (>90th%), pre-diabetes, or DM. After being screened for eligibility, adult caretakers gave verbal consent to participate in the program, which was approved by the Children’s National Hospital Institutional Review Board.We assessed program feasibility by monitoring enrollment, response to satisfaction surveys, participant-reported satisfaction, monthly class attendance and video views, and retention. Rate of enrollment was determined by the number of eligible families who agreed to participate divided by the number of families screened.The rate of response to satisfaction surveys was determined by the number of completed satisfaction surveys divided by the number of enrolled participants during the response period.Satisfaction was assessed by self-report surveys, which included questions about satisfaction with the produce offerings and educational components, and utilization of the produce provided. Responses to satisfaction questions ranged from 1  to 5 (completely satisfied/very much like) and a sliding scale to indicate percentage of produce utilized. Invitations to satisfaction surveys were sent to all enrolled adult participants by text and email and three automatic email reminders were sent if needed. Satisfaction surveys were sent bi-weekly during the first 6 months of the program, then were sent monthly during the last 6 months of the program . We asked all enrolled participants to complete the satisfaction surveys but there were no specific disenrollment criteria related to survey completion.Participation was assessed by monthly virtual class attendance as observed by program staff present at each class session and self-report virtual class attendance and video views at post-intervention.Six- and twelve-month retention was determined by the number of enrolled families  at the 6-month and 12-month time points divided by the total number of originally enrolled participants.Participants were asked their age, gender, marital status, household size, income, education level, employment status, federal supplemental nutrition program participation, child age, height, and weight to calculate BMI and BMI category at baseline.We used the six-item (12-month) USDA screener  to obtaiWe asked adults to answer questions derived from food frequency questionnaires about their own eating habits and their child’s eating habits at baseline and post-intervention. For adult intake, we asked about frequency of daily intake in the past week (“About how many cups of fruit (NOT 100% pure fruit juice) did you eat each day?” and “About how many cups of vegetables (NOT 100% vegetable juice) did you eat each day?”) and provided guidance about serving size . Daily frequency categories were converted into a continuous numerical variable (intake score) by assigning an arbitrary numerical value (1–7) to each categorical serving size option .For child intake, adults answered questions about frequency and portion size of fruit and vegetable consumption (excluding potatoes) adapted from the National Cancer Institute’s Eating at America’s Table Study Quick Food Scan  and the The FNPA screening tool is designed to evaluate the home environment and family practices that may contribute to a child becoming overweight. It has been shown to have good internal consistency and utility for predicting children’s risk for becoming overweight . We usedThe PHCS is used to measure a participant’s perceived ability to manage their own health outcomes . We askeThese questions are specific to participant’s shopping behaviors and self-confidence in buying and using healthy foods. Four questions were derived from the Cooking Matters Questionnaire . ResponsTo conduct the qualitative assessment, we developed an interview template with the following research objectives in mind: to assess the program feasibility and satisfaction with the produce and nutrition education offerings, and explore the impact of the program on FI and eating behaviors. The instrument contained a set of open-ended questions, accompanied by probes, which provided structure to the discussion, while allowing the flexibility needed to allow for spontaneity and candor. The instrument was designed to elicit feedback on the following topics: (1) opinions of the produce delivery and the nutrition education content, (2) benefits experienced from the produce delivery and nutrition education offerings, (3) influence of program on motivation to prepare and eat fruits and vegetables in the family, (4) impact of program on perceived FI, and (5) suggestions for program improvements.To schedule the interviews, the study team attempted to contact the original 25 participants by phone, approximately 6 months into the program. The research team was unable to contact 10 participants after multiple attempts due to the phone being disconnected, or no response to email/phone call. Thus, of the 25 participants, 15 were reached by phone and invited to participate in the interview, for which they would receive a USD 20 incentive. Twelve adults agreed and completed the interview. One-on-one interviews were conducted by LF securely online using Zoom and were recorded. One of our two research assistants  joined each interview as a note taker. After interviews were completed and recordings were transcribed, the research team used a thematic analysis framework manually to analyze the qualitative data. Members of the research team  read the participant responses to become deeply familiar with the content. After this initial review, members of the research team  met as a group to review an initial set of “themes” or commonalities generated from participant statements. Inductive coding techniques were used to ensure preliminary theme development was identified by the content of the data rather than by preconceived models or concepts. Themes were considered overarching topics while subthemes were more specific topics that fell under each theme. These were placed in a codebook with example quotes that represented each theme and subtheme. Members of the research team  then used this codebook to independently analyze and code each interview. Codes were reviewed frequently within the research team; after saturation was reached, emerging themes and subthemes were finalized. Any discrepancies were resolved through group consensus .t-test was used to examine pre-post group differences in continuous variables (FI score and child fruit and vegetable intake) and chi-square was used to evaluate pre-post differences in categorical variables (FI category and adult fruit and vegetable intake frequency). To explore the relationship between FI and diet, we used Pearson correlation to evaluate the relationship between FI score and fruit and vegetable intake in adults (intake score) and children . Alpha was set at 0.05. Analysis was conducted using GraphPad Prism 9.3.1 statistical software .Univariate descriptive statistics  were used to describe the sample demographics and feasibility and outcome variables, and Shapiro–Wilks test was used to explore normality of the data. This study was not powered to show statistical significance; however, for the sake of exploration, we conducted exploratory bivariate statistical analyses. Student’s Over the course of the 12-month program, approximately 4000 pounds of produce was delivered to participants’ homes along with 72 recipe and skill cards. Twenty-six “FLiP Tip” physician videos were sent by text message to enrolled participants and 12 monthly virtual cooking classes were held.The study population was recruited by convenience sampling from two Children’s National Hospital outpatient clinics and is representative of families living in under-resourced areas with a predominantly African-American demographic in Washington, DC, USA. In December 2020, 33 families were referred and screened by phone for diet-related disease and 25 eligible families were enrolled in the program. Therefore, rate of enrollment was 75.8% (25/33). Screening, recruitment, and enrollment were completed in 3 weeks.The rate of satisfaction survey completion was lower than expected (<50% response rate) during the first 6 months of the program. To encourage greater survey participation, we revised our survey strategy to include a USD 5 incentive for completing a survey; multiple telephone, text, and email invitations, reminders, and follow-ups were sent; and we reduced frequency of surveys from bi-weekly to monthly. There were no other changes to the study. During the next 6 months, satisfaction survey response rate increased to an average of 77% over the course of the remaining 6 months.Satisfaction survey responses revealed that most families were very satisfied with the program, indicating they liked the produce and the educational components. On average, participants reported 77.5% of the produce was used or frozen for future use. Regarding produce satisfaction, 80% of respondents were very or completely satisfied with produce variety and 80.5% said they tried a new food when one was delivered . Regarding recipe liking and accessibility, 55.9% said they made the recipe (exactly or modified), only 15.9% said they did not have the tools or ingredients to make the recipe, and 11.8% said they did not like the recipe. Regarding the video-based nutrition education content, 76.5% indicated they somewhat or very much liked the FLiP Tip videos. Post-intervention self-report survey questions asked participants what percentage of videos they viewed (out of 26) and what percentage of classes they attended (out of 12). The average percentage of videos viewed per participant was 66.8% +/− 28.1% and the average percent of classes attended per participant was 63.1% +/− 22.7%. Observed monthly attendance of enrolled families was between 4%  and 27% .At the 6-month time point, we contacted all 25 enrolled participants to describe our revised survey strategy and to ask about any barriers to their continued participation in the program. At this point, 7 were disenrolled in the program and their produce delivery and nutrition education ended. Of the 7 who ended participation at the 6-month time point, 2 were lost to follow-up (unable to reach) and 5 decided to end their participation voluntarily, largely citing time and responsibility constraints. Retention at 6 months was 72% (18/25) and attrition was 28% (7/25). At 12 months (program completion), 15 families completed the post-intervention survey; therefore, 12-month retention was 60% (15/25) and 12-month attrition was 40% (10/25).2. See All adult participants were African-American females with a mean age of 29.9 +/− 5.8 years old. Median household size was 1 adult and 3 children. Unemployment rate of adult participants was 40%, 48% had a high school diploma, and 40% were making less than USD 10,000 per year. A majority of families have used either SNAP (64%), TANF (56%), and/or WIC (56%). Because diet-related disease risk in parents was an inclusion criterion, not surprisingly, adult participants had a high prevalence of obesity (56%) and high BP (40%). Average adult BMI was 33.7 +/− 1.9 kg/mt = 0.74, p = 0.46) but represents a trend in the expected direction, indicating an improvement in perceived FI. We also examined the pre-post change in FI category distribution. Fewer adult participants ranked in the very high FI category post-intervention. At baseline, 32% reported experiencing very high FI vs. 7% post-intervention. These pre-post differences were not statistically significant  but indicate a trend toward a difference in the expected direction  indicated they were giving their child formula, of which 5 were exclusively formula-fed. Among those using formula, the predominant type was dairy-based (71.4%) vs. soy-based (28.6%). Four adults reported giving breastmilk, of which three where exclusively breastfeeding. There were 3 (27%) adults who reported giving solid food . In general, adults reported starting solid foods at 1–3 months , 4–6 months , 7–9 months , or 10–12 months . However, this question did not specifically ask adults to report solely on this practice for the reference child; therefore, adults may be reporting when they historically started solid food for other children in the home. t = 1.7, p = 0.09). The average daily cup equivalent of vegetables increased from 0.34 +/− 0.27 at baseline to 0.52 +/− 0.32 post-intervention, representing a 34% increase and an absolute change of 0.18 cup equivalents. This pre-post difference was trending toward statistical significance . See At baseline, there were 17 adult participants who answered a food frequency questionnaire for children. At post-intervention, there were 14 . The average parent-reported baseline and post-intervention daily intake of fruit was somewhat close to the amount recommended by the USDA Dietary Guidelines (0.5–2 cup) and close to the national average for young children (0.8 +/− 0.04 cups) . VegetabWe wanted to know if children were meeting the recommended daily intakes at baseline and post-intervention. We defined “adequacy” of daily intake as the low end of the recommended range for age groups, that is, 0.5 cups of fruit for children under 1 year old and 1 cup of fruit for children over 1 year old, 0.6 cups of vegetables for children under 1 year old, and 1 cup of vegetables for children over 1 year old. At baseline, very few children met this conservative recommended intake for vegetables (6%), but a larger proportion of children met the fruit recommendation (41%) and only 1 (5%) met both fruit and vegetable recommendations. At post-intervention, the proportion of children who met the vegetable and fruit recommendations increased, but “adequate” vegetable intake was still low (21%), while more than half met the fruit recommendation (57%) and 20% met both fruit and vegetable recommendations.t-test) to test the significance of the pre-post differences in average fruit and vegetable intake. The change in fruit and vegetable intake was greater in the subgroup of participants with complete pre-post data but the change in vegetable intake was smaller. There was a 43% increase in average daily fruit intake (0.76 +/− 0.53 to 1.3 +/− 0.72), and only a 29% increase in average daily vegetable intake (0.44 =/− 0.30 to 0.62 +/− 0.32). The difference in average fruit consumption was statistically significant  and the difference in average vegetable intake was not .If we only look at the subset of 10 participants with complete pre-post data, we can use the more sensitive statistical test (paired p = 0.07) but the difference in fruit intake was not statistically significant . See Adult self-reported that baseline and post-intervention consumption of both fruits and vegetables was generally lower than the recommended amounts (at minimum 1.5 cups of fruit and 2 cups of vegetables) . At basep = 0.002) and vegetable  intake. As FI score increased, fruit and vegetable consumption decreased. The relationship was statistically significant for fruit but was only trending toward being statistically significant for vegetable intake. The relationship between adult FI score and fruit and vegetable intake did not persist at the post-intervention time point. In children, there was a similar general trend, a negative correlation between baseline FI score and baseline fruit  and vegetable  intake, but none of the correlations were significant.We examined the quantitative relationship between FI score and fruit and vegetable consumption using Pearson correlation. To conduct correlation analysis, we created a continuous numerical variable for adult fruit and vegetable intake by assigning an arbitrary numerical value (1–7) to each categorical serving size option . In adults, we found an inverse relationship between baseline FI score and baseline fruit . We wanted to compare baseline characteristics of participants who finished and who did not finish. More non-finishers reported income greater than USD 25,000 per year (60%) vs. finishers (6.7%). The average baseline FI score was lower in non-finishers vs. finishers (2.4 +/− 1.7 vs. 3.1 +/− 2.1) and there were fewer non-finishers with baseline “very high” FI vs. finishers (20% vs. 40%), indicating non-finishers may have had lower perceived FI and higher income, but we cannot say for certain, as these differences were not statistically significant because the sample size was very small. Baseline scores for modified FRM, PHCS, and FNPA did not differ in non-finishers vs. finishers. Baseline child and adult fruit and vegetable intake did not differ by parent finish status. Although we cannot make any conclusions about the differences in finishers and non-finishers at baseline, it is worth exploring in subsequent studies.Four preliminary themes were derived from qualitative interviews: (1) reduced food hardship, (2) support for family-driven behavioral change, (3) increased economic flexibility, and (4) opportunities for family bonding.Within each of these main themes were subthemes. The subtheme for theme 1 was improved food accessibility and reliability. For example, participants described how having a consistent source of food, exposure to a wider variety of food, and direct delivery to their home improved the quantity and quality of food available in the home. This supports our quantitative results described in Under theme 2, we specifically identified eating and cooking behaviors as those that were impacted. As an example, participants described trying new foods and learning new kitchen and food preparation skills as a result of engaging in the program. However, we did not find this reflected in our quantitative assessment of culinary self-efficacy as described in Under theme 3, participants described how the program eased some of the impact on household economics and financial strain at the end of the month when SNAP benefits are low. More than 60% of participants in the intervention were using SNAP, and by providing food, families were able to use their money on other non-food expenses. Participants describe using money on non-food expenses such as bills, gasoline, other grocery foods, gifts, entertainment, and dining outside the home. Under theme 4, participants indicated they felt a greater sense of family togetherness around food preparation and eating and that the program specifically encouraged child engagement in the cooking process. See The aims of this study were to examine the feasibility of implementing a home-delivery produce prescription and virtual nutrition education intervention and to explore the impact of the intervention on FI and fruit and vegetable intake in children and adults. We found enrollment of the target population was feasible and the program was well-received; however, early measures of program participation indicated there were unanticipated barriers to participation. In both quantitative and qualitative assessments, adult participants reported positive thoughts and feelings about the fruit and vegetable delivery and an increase in intake of fruit and vegetables in a subgroup of children. FI scores generally went down but the change was not significant. Semi-structured interviews revealed positive impacts of the program on food access, family fruit and vegetable consumption, and the family budget, and shared enthusiasm for the preparation and use of fruits and vegetables at home. This is promising evidence to support our exploratory hypothesis that the FLiPRx intervention can impact FI and fruit and vegetable intake, but we are not able to draw conclusions about the causality of this outcome. More work is needed to understand barriers and facilitators of program participation and the impact of the program on nutrition literacy, eating and purchasing behaviors, and FI.Although the study was not specifically powered to test for a statistically significant increase in fruit and vegetable intake, we explored the impact of the program on eating behavior change. We found a post-intervention increase in fruit intake in a subgroup of children, although intake was still generally lower than the federally recommended amounts. The proportion of adult participants that met intake recommendations somewhat increased from baseline to post-intervention by 20% (32% to 40%) and by 50% (20% to 40%) for fruit and vegetables, respectively, but this was not statistically significant. Child average intake of fruit and vegetables generally increased post-intervention ; however, the consumption at baseline and post-intervention was below recommended levels, with vegetable intake farther below the recommendation than fruit intake. This is consistent with data reporting an innate predilection towards more sweet taste in children . The proFew others have reported longitudinal quantitative assessments of child fruit and vegetable intake in response to a primary-care-based produce prescription program. We identified three relevant publications with quantitative fruit and vegetable results in children. A produce prescription program in Flint, MI with children aged 2–18 years old showed increased fruit intake (by 0.19 cups) and no change in vegetable intake at 6 months evaluation time point , but redHaving observed lower than optimal fruit and vegetable intake post-intervention, we recognize there is room for even greater improvement in consumption in participants of this program. However, there was no explicit intake goal in this pilot study; we simply wanted to know if pre-post intake would differ by any amount. A goal of future work could be to help participants achieve a specific intake goal in line with the federal recommendations for a healthy diet. Moreover, it is a well-known phenomenon that any behavior change takes time and consistent effort. Our qualitative findings suggest increased familiarity and involvement in the family food preparation process may be a key factor in promoting fruit and vegetable consumption in children. The short duration of this study could mean we have not yet seen the eventual increase that will take place with continued exposure to fruits and vegetables, given participants continue purchasing and involving children in the preparation of fruits and vegetables after they finish the program. In addition, because of the rich descriptions of fruit and vegetable intake in the qualitative evidence, it is possible that the methodology for collecting fruit and vegetable intake measures was not adequate to show the families’ lived experiences. Although it can be a burdensome effort, future work should use more rigorous dietary assessment methods, such as the repeated 24 h recall .Another consideration regarding the magnitude of change we observed in fruit and vegetable consumption is that there are many complex drivers to intake; among them are cost and accessibility, which this study specifically addressed, but also taste preference and familiarity  and the In qualitative interviews, we heard from participants that a unique benefit of the FLiPRx program is the stability and increased accessibility that the home-delivery model offers. In qualitative interviews at the six-month mark, participants described the consistency and reliability of the FLiPRx produce delivery as contributing to a decreased perception of FI, particularly because the food was delivered to their home on a consistent basis. From quantitative assessments, we reported fewer participants in the very high FI category post-intervention (7% vs. 32% at baseline) and the average FI score was slightly lower by 15% (2.8 to 2.4), but neither change was statistically significant. Other produce prescription programs have reported improvements in adult perceptions of FI, for example, Aijer reported a 94.1% decrease in the prevalence of FI at the end of the program  using thFamilies experiencing FI often limit the amount, variety, and quality of food in an attempt to save money and to limit food waste  and descThe reduction of families in the very high FI status indicates that the extremes of FI may very well be buffered with a reliable source of increased benefits such as produce prescriptions or more robust SNAP dollars. We also reported a theme of greater economic flexibility, and we found this intervention allowed participants to spend money on necessities and luxuries rather than/or in addition to enhancing their regular food purchases. This may lead to unintended consequences such as increasing purchase of food outside the home or snacks, which can potentially have an unintended negative impact on healthy eating behaviors. However, it may also lead to purchasing more high-quality food for the family because of the financial freedom. An exploration of purchasing habits through qualitative and/or receipt analysis should be conducted to better describe short-term and long-term food purchasing habits within produce prescription interventions. Programs that address FI by supplying food resources must also address nutrition education and food resource management support to facilitate healthy food purchasing habits even when additional monetary resources are available. Another study reported that a food prescription program allowed individuals to spend money differently without unintended negative consequences , but thiWe noticed early participation was low. Other produce prescription studies have reported limited engagement of participants  and idenIn terms of improvements to curriculum, participants suggested a useful resource would be a comprehensive guide to the produce they received. Recognizing that individuals learn in multiple ways, including visual, auditory, tactile, etc., to augment our video and live learning sessions, we created a print guide to fruits and vegetables that we will provide to all future participants. This printed book contains an indexed list of all produce that might be delivered during the program and provides tips on how to wash, store, and use the produce, especially for young children. To help families engage with the virtual cooking classes and recipes we provide, we have designed a culinary package to be delivered at enrollment that contains staple ingredients, tools, and spices that families might use to prepare the recipes during the intervention.Produce prescription programs may play a role in buffering FI and improving diet quality but they alone are not a sustainable solution to tackling the FI needs families face. The goal of our produce prescription program is to provide food and education resources with the hope of promoting sustained behavior change. By providing food and education resources as well as skills to improve self-efficacy, we hope that families begin to utilize their own resources to purchase and prepare more produce once they are finished with the program. Federal nutrition programs and tax credits remain some of the most important resources for addressing FI and diet quality as an essential safety net for households with limited income and financial resources. Produce prescription programs may play a supportive role, to serve as a short-term buffer, yet when participation ends, families must be given access to further resources to continue to address the unmet social need while promoting healthy behaviors. Effective interventions must link participants with other community programs and ensure connection to meaningful federal nutrition programs and tax credits. In this way, the FLiPRx program is prioritizing sustained nutrition security and attempting to address some of the social and socioeconomic barriers to healthy eating. Longer-term follow-up is required to truly understand whether this program has any sustained impact on FI and eating behavior.The sociocultural context in Washington, DC during our program implementation period may have contributed to unique local barriers to participation. Most notably, our first virtual interactive cooking class was scheduled for January 6th, 2021, a day that is now infamous due to the violent actions that occurred on the nation’s capital down the street from the homes of our participants. Additionally, the city has grappled with difficult and tragic instances related to race relations and systemic racism, exacerbated by the COVID-19 pandemic. These social issues greatly impacted the most vulnerable populations already struggling with community disinvestment, limited opportunities for well-paying jobs, affordable housing, childcare, and education. This cluster of sociocultural disruptions increased the stress already prevalent in the lives of our participants, who were typically single parents, caring for young infants and children with low incomes. As voiced by our participants, these stressors made it particularly difficult for caretakers to make time to attend our structured non-recorded live virtual class. We believe these factors contributed to barriers to participation and future research will need to continue to adapt to families by offering more efficient opportunities for participants to engage with instructors and build skills and knowledge.Outside of social factors, some of our study design factors may have contributed to lower-than-expected participation. For example, attendance was not formally monitored and was not mandatory to continue receiving produce deliveries. We think participation will improve if attendance is mandatory and if we provide more engagement from program staff earlier and more consistently in the program . Additional facilitating factors would be to offer classes at multiple days and times to accommodate all schedules.There are strengths and limitations to this study. Strengths include the longitudinal design and use of validated assessment tools. Another strength is the intentional design of our intervention to address multiple barriers around FI, including quantity and quality of food, variety, appeal, reliability, and transportation challenges. The USDA 6-item screener not only measures the presence of FI but detects the depth of FI, and the fruit and vegetable questions assessed total intake by asking about portion sizes in addition to frequency. However, the 6-item screener is limited in exploring FI severity and depth and is unable to identify FI severity at the level of children, instead focusing on the household overall. We experienced quick recruitment and enrollment, which indicates that this program can be seamlessly integrated into clinical systems for easy enrollment from clinical screening. A limitation is that the food frequency tool is a self-report measure with potential for recall bias or differential response bias . Other limitations include short duration, no control, and a small sample size, limiting the generalizability of our finding and long-term sustainability of the outcomes of this pilot. We also recognize that there is potentially a self-selection bias, whereby people with relatively healthy habits were the ones who agreed to participate in our study. There may also be a social desirability bias, whereby adult caretakers overestimate the amounts of fruits and vegetables they or their children eat to appease the research team. Another source of limitation is attrition; those who did not complete the study may have differed in some factor of interest. For example, there were some notable divergences in income and FI characteristics of finishers and non-finishers, possibly indicating income status, and perceived FI category may play a role in attrition. Another limitation is that we were not able to use web analytics to quantify the degree to which video-based intervention components were utilized by participants; we only have self-report data. This is a change we have implemented in the next iteration of the program. Lastly, this intervention was conducted during the COVID-19 pandemic, and due to a variety of robust federal and local support mechanisms (as described earlier), there are confounding factors we cannot control for, which may have impacted our findings.We found the FLiPRx intervention was feasible and well-received by participants, but notable barriers to participation need to be addressed in subsequent iterations. Multipronged interventions that aim to enhance nutrition and culinary literacy and overcome socioeconomic and environmental barriers to healthy eating in historically marginalized and underserved populations may play a role in promoting fruit and vegetable intake in this population. More work is needed to understand the role of this intervention on FI and improving eating habits in adults and children long-term and its impact on health outcomes in addition to cost-effectiveness. Future work must include more robust qualitative assessments to better understand barriers and facilitators to participation, change in fruit and vegetable intake, FI status, and sustainability of behavior change. Additionally, robust quantitative assessment tools must be utilized to better capture longitudinal dietary and spending behavior change. Randomized controlled trials with longer-term follow-ups would provide a clearer understanding of the impact of this program and its effectiveness and help determine whether it is a viable adjunct to support improved diet quality in families who rely on federal nutrition programs. Finally, our intervention is designed to address the social needs and social risk of families through a clinical-community collaborative effort and is not designed to address the deeper social determinants of health that play a major role in overall health. However, future work must continue to develop more effective strategies to work with families and health systems to impact social change through policy and community initiatives.


We developed and piloted a novel art-based online skills program led by a peer mental health professional with lived experience of complex mental health, including Borderline Personality Disorder (BPD). Key challenges of living with BPD and emotion dysregulation were addressed through artmaking informed by a dialectical framework and skills, to evaluate acceptability and efficacy.A structured, manualised 2-hour weekly arts-based skills program was piloted for people with BPD over 18 weeks. Evaluation included both quantitative and qualitative measures at commencement and completion.Cohen’s d = 1.77). Qualitative thematic analysis found participants had improved capacity to regulate emotions and tolerate distress, improved connection with others, enhanced understanding of the self, and higher hope for living well. We found that artmaking facilitated processes and helped the expression of difficult emotions, symbolise challenging relationships, and facilitate greater self-understanding. Participants reported high levels of satisfaction, and 77.4% reported that the program had increased wellbeing.Thirty-eight participants enrolled in the program , and 31 completed (82% retention). Multilevel modelling analysis of the primary outcome variable Difficulties in Emotion Regulation Scale (DERS) demonstrated a large improvement over time (effect size This novel artmaking program for emotion dysregulation and BPD was acceptable and potentially effective. Peer facilitation using arts-based skills is a modality of therapy for further investigation. Personality disorders are a highly prevalent mental health condition, with the global prevalence of any personality disorder estimated at 7.8% . BorderlOne example of a well-studied therapy is based on dialectical principles . The traditional model consists of group and individual therapy, with access to phone coaching as needed, often for at least 12 months duration . Recent Art psychotherapies have been increasingly recognised as a valid treatment for many mental health conditions, utilising all forms of artistic expression such as visual and performing arts and creative writing to facilitate expression and identity formation . ResearcPeople living with BPD report a desire for both helpful coping and survival strategies, and improved capacity to engage in meaningful activities and relationships in building their life worth living , 38, 39.The global coronavirus pandemic , has significantly impacted access to mental health services and increased psychosocial stress , 44, in The present pilot study evaluated an online arts-based skills program for people living with BPD, developed and led by a peer mental health professional. The evaluation was designed to assess the acceptability of the program, its role in ameliorating symptoms, and to evaluate if it improved emotion regulation.The program involved one 2-hour session every week for 18 weeks. The program was designed to be delivered on a secure online platform due to the global pandemic (Covid-19). Examples of practical adaptations for an online environment included providing access to art materials and workbooks to participants via mail delivery, consideration regarding positioning of the camera on the facilitator and participants and their artmaking. Clinical considerations included managing group process in the online space by providing opportunities for sharing and connection in diverse ways  and considering safety and management of potential risk. Online delivery of the program allowed broad engagement across many locations and within populations with increased vulnerabilities and considerations such as mobility and access issues.Minimal contact with the facilitator was negotiated outside of the sessions, and connection was promoted through between-session art sharing via email or on a private social media platform. The groups were closed and capped at 10 participants. The pilot included 4 programs running concurrently from July to December 2020. Participants were able to self-select the program that best suited their schedule.The program included skills based on dialectical principles and also specialist topics on self-stigma, invalidation and anger. See Table The 2-hour sessions included an Acknowledgement of Country and brief welcome with topic review . The session then moved towards providing information on the first skill (15 min) with artmaking time (20 min) and participant reflection and feedback (20 min). This process was repeated for the second skill of the session, with a brief 5-minute conclusion. Homework tasks were incorporated prior to the first and last sessions to enhance engagement and reinforce learning.The lead author (MJ) designed and conducted the program, and the third author (RC) was a peer model/support facilitator. The lead author has Dialectical Behaviour Therapy, counselling, Narrative Therapy and art psychotherapy training and lived experience of complex mental health including BPD. They are a registered Art Psychotherapist, hold a Masters of Art Therapy, and are an Intentional Peer Support Practitioner. The peer facilitation was grounded by the IPS model of practice and a trauma and diversity informed framework. No fidelity assessments were utilised.The primary role of the psychotherapist facilitator was to provide a safe and inclusive therapeutic environment, that was sensitive to group and individual processing needs. They used the arts-based program to experientially explain and practice skills. Sharing and self-expression in a safe and trauma-informed manner was encouraged, including managing any individual considerations , whilst maintaining the frame of the group . The psychotherapist facilitator demonstrated skilled behaviours, modelled effective boundaries, and provided considered and therapeutically-oriented self-disclosure of lived experience where appropriate. The lived experience facilitation was demonstrated by providing personal artwork examples and using personal narratives connected to the overall topic and skills being discussed. Replication of this intervention would require an Art Therapist with lived experience, which could be sourced using programs such as the BAAT “Dual therapist” program or with a peer worker co-facilitator. It is essential that the facilitator conducts the program within both the DBT and IPS framework. This includes maintaining a non-judgmental approach, being open to the experience of the participants, actively engaging in co-creating, meaning-making and mutuality, dialectically recognising that there is no absolute truth, and maintaining a hopeful stance.The program was advertised by contacting personality disorder services, wider mental health services, advocacy groups and research centres across Australia. The program was evaluated on people with lived experience of Borderline Personality Disorder. Participants were required to have received a diagnosis of Borderline Personality Disorder from a psychiatrist, however diagnoses were not repeated by the research team. Most participants had some prior treatment experience. Participation in the program was paused if they became acutely unwell or suicidal and required urgent mental health care. Participants were considered to have dropped-out of the program if they did not attend three sessions in a row or complete the final two sessions. All participants provided informed consent to participate in the pilot program, including the collection of evaluation data. The art therapy program was approved by the Institutional Review Board (University of Wollongong HREC#220).38 participants enrolled in the program. 89.5% of participants identified with she/her pronouns, and 10.5% identified with they/them pronouns. Participants reported a mean age of 33.6 . Most participants resided in metro regions across Australia (57.9%), with the remaining living in regional or remote areas of Australia (28.9%) or internationally (13.2%). International participants were located in New Zealand, the United Kingdom and United States.All participants were living with Borderline Personality Disorder. Participants also reported comorbidities including anxiety disorders (68.4%), post-traumatic stress disorder (44.7%), depressive disorders (55.3%), eating disorders (26.3%), bipolar disorders (15.8%), attention deficit hyperactivity disorder (5.3%), autism spectrum disorder (5.3%), adjustment disorder (2.6%), psychosis (2.6%), and chronic physical health conditions (15.8%).Prior experience with therapy varied. Most participants had previously completed a therapy program (39.5%), started but not completed a therapy program (18.4%), or was either currently engaged in a therapy program or on a waiting list (13.2%). Some participants stated that a therapy program wasn’t available to them (13.2%). 68.4% of the participants reported that they regularly attend sessions with a mental health professional, and 50% stated that they often rely on the support of family members or friends.We tested whether ongoing engagement with a mental health professional at baseline impacted DERS scores at pre and post-program. There was no significant difference on the baseline DERS total score between those who reported regularly engaging with a mental health professional  compared to those who were not regularly engaged , t [36] = -1.35, p = .186. At post-program this lack of difference remained between groups , t [29] = − 0.705, p = .486. In addition, a one-way ANCOVA was conducted to determine whether a statistically significant difference occurred between participants who reported regularly engaging with a mental health professional compared to those who were not on the post-program DERS controlling for the pre-program DERS as a covariate. There was no significant effect of engagement with a mental health professional on the post-program DERS after controlling for the pre-program DERS, F  = 0.17, p = .686.Prior to the program, participants reported variable engagement with artmaking, where 31.6% stated that they usually spent less than 1 h a week artmaking pre-program, 28.9% 1 to 2 h a week, 23.7% 2 to 3 h per week, and 15.8% over 4 h per week.Participants completed a brief questionnaire prior to commencing the program (baseline), at the mid-point (9 weeks) and at completion (18 weeks). Measures included both quantitative and qualitative approaches to assess acceptability of the program, and pilot data regarding effectiveness on symptom improvement.The DERS  is a 36-Acceptability is an important measure of participant experience and satisfaction of a pilot program and has implications for future replications. Acceptability is often measured through a combination of simple quantitative scales, and qualitative feedback. This method has previously been utilised by pilot programs in the treatment of people with BPD e.g., , 55). In. In55]).How has the group affected your understanding of therapeutic skills and theory?’, ‘How has the group affected your sense of self?’, and ‘How has the group affected the way you view “art as healing” and/or “art as therapy”?’. The qualitative data provides participant feedback on satisfaction, experience and acceptability of the program.The post-program evaluation also included free-text qualitative questions including ‘The pre and post-program main outcome (DERS) was assessed using a multilevel modelling approach (IBM SPSS Statistics 26 Linear Mixed Models). Utilisation of this statistical approach allowed for analysis of all data as intention-to-treat , with tiQualitative data was analysed using a Husserlian phenomenological approach, previously described and applied in qualitative data analysis . This apt [36] = − 0.19, p = .85. In addition, there was no significant demographic difference between those who completed compared to those who did not complete based on identified preferred pronoun X2 [1] = 1.009, p = 1.00 (adjusted by Fisher’s Exact Test), or age t [36] = 0.004, p = .997.31 participants completed the program . There was no significant difference on the baseline DERS total score between those who completed  compared to those who did not complete the program , t (417) = 3.803, p = .035. It is likely that this is due to most of the sample having already completed, or currently engaged in, a therapeutic program.On the main outcome variable, DERS, the baseline mean was 113.34 (SD = 25.10). This was compared to comparison groups in the literature using this measure. The current average baseline DERS was lower than previously reported by research with people diagnosed with BPD  = 9.80, p < .0001, and female controls  = 1.39, p = .17. Therefore, the variable of interest showed improvement at post-program when compared to a university sample pseudo-control group previously reported in the literature.The post-program mean on the DERS was 84.52 . This remained significantly higher than previously reported by female controls , t [49]s above; ), t  = 34.05, p = .000. Figure Cohen’s d = 1.77, indicating a very large effect size.Linear mixed model analyses demonstrated a significant reduction in the DERS total score over time, F  = 16.64, p = .000; goal-directed behaviour, F  = 17.26, p = .000; impulse control, F  = 20.10, p = .000; emotional awareness, F  = 9.08, p = .000; access to emotion regulation strategies, F  = 19.49, p = .000; and emotional clarity, F  = 14.96, p = .000. Figure To further elucidate the significant improvement of DERS over time, linear mixed model analyses were performed on all the subscales. All subscales demonstrated a significant improvement over time; non-acceptance of emotional responses, F , 42.42 =The lived experience facilitation of the program was important to participant’s engagement. From the outset, 71.1% endorsed that the peer facilitation made it more likely for them to participate, and 10.5% stated that this was the only reason they registered. At completion, 100% of participants stated that the facilitator having lived experience positively impacted on their decision to stay in the program. In addition, 100% of participants endorsed that their experience in the current program would positively impact on their decision to engage in further therapy or support interventions in the future if they contained art-based skills.At completion, 93.5% of participant’s stated that the program had increased their understanding of therapeutic skills and principles, and 93.5% endorsed that the program had helped participants to implement skills in ways that they hadn’t been able to before. Additionally, 80.6% stated that the program supported them to stay living well, and 77.4% that the program had increased their wellbeing. 90.3% endorsed that the program helped them understand themselves , 80.6% increased self-compassion and self-worth, and 83.9% that the program allowed the participants feel seen, heard and validated. 61.3% stated that the program increased their social interaction skills, and 87.1% increased their confidence in using art as a tool for healing. Finally, 67.7% stated that the program added benefit to the verbal therapies the participant was engaged in, and 67.7% that the program provided peer support that they had never had access to before. Taken together, these results indicate high acceptability of the program. Acceptability of the program was further elucidated through the qualitative feedback.Thematic analysis of the qualitative data revealed many significant statements regarding the impact of the program on meaningful change for people living with BPD. Table “It’s made a few skills easier to understand, and making them more interactive, makes me want to practice them more often.”Participants reported improved understanding of therapeutic skills and principles through artistic practice. The practical nature of the skills was a common theme, facilitating improved accessibility and meaningfulness of otherwise abstract or theoretical concepts. This, in turn, enhanced practical application, increased capacity for coping and regulating emotions, and willingness to try new strategies. One person living with BPD stated:“I was able to learn about some of the theories being explored through the activities. The activities and artworks created in each session encompassed multiple… skills which showed how they were all interlinked rather than separate skills to be used one at a time. While the activities aren’t necessarily strategies to take with me ‘on-the-go’  they do help in opening me up to a more realistic understanding of [the skills].”Another person living with BPD commented on improved understanding of the integration of therapeutic skills, and understanding of the scientific and theoretical underpinnings:Participants also expressed the benefits of enjoying the process of artmaking. Many participants reported that this process facilitated the increase of positive experiences, thus increasing their capacity to cope and engage in self-care.“It was made easier to understand by being facilitated by someone with lived experience and I felt more validated, understood, and more comfortable engaging. This was not only more accessible for me but it was delivered in a way that kept me coming back each week.”Participants expressed the benefits of engaging in a supportive group process, including practicing being vulnerable and connecting with others. Some participants commented that this was a challenging process that also allowed them to practice being skilful. Participants also reported the value of the peer-facilitation in providing lived experience perspectives, and authenticity to the process and skills. It was acknowledged that lived experience examples of how skills could be applied was useful to increasing understanding and willingness to try new skills. Participants reported that the lived experience also facilitated validation beyond what non-peer mental health clinicians can provide. One participant said:“Now I know that all my feelings are valid, I can accept myself as a whole being rather than feeling “split” between good and bad. When I feel upset, I know how to self-soothe and validate and then I move on to check the facts and problem-solve. I really feel that I accept myself more as a person - it’s been many years in progress, but this group has helped me to stop feeling shame and hatred towards myself, as I now understand how to “look after” myself. I feel so much more stable than I ever have.”Improvements in understanding of the self were reported by participants, including greater awareness of internal processes. In particular, participants reported greater connection to their thoughts and feelings. Participants commented that the program facilitated better self-expression of their internal world. Many participants reported sharing their artwork with others involved in their support systems, providing a platform to facilitate deeper understanding and discussion. In addition, participants reported the therapeutic value of the program. Many participants reported experiencing a strengthened self-concept, including a deeper appreciation of their own values and beliefs. Some participants also reported the benefits of being able to process important emotions and events to facilitate wellbeing.“I used to say I had no genuine sense of self and that I would change myself to please others. Since then I have learnt about boundaries, the importance of values and how we can create our own story by reclaiming the hurtful and invalidating things we are told by society. While I’m not yet a completely self-assured individual, I definitely have more of an idea of ‘Me’ than 18 weeks ago.”Another person living with BPD commented:“I feel much more confident in myself and my decisions. I feel more valuable and like my feelings and thoughts matter.”Participants expressed a greater sense of hope for their own ability to live well following the program. This was often linked to connecting with a peer mental health professional and witnessing the possibility of living well during this process. Many participants also found a deeper understanding of the self and capacity to regulate as instilling hope for a life worth living. In addition, many participants also reported an increased sense of empowerment following the program. One participant powerfully stated:The present study evaluated a peer-led art-based skills program for people living with BPD. A multilevel modelling analysis of the primary outcome variable (DERS) demonstrated significantly improved emotion regulation over time, with a large effect size. Significant improvement was observed across all subscale domains of this measure, indicating wholistic improvements in emotion regulation over the course of the art therapy program, and thus reduced symptomatology of Borderline Personality Disorder.Taken together, results show that participants found the approach to be acceptable, and potentially effective. Acceptability of the program is further supported by the strong retention (82%). This is compared to 67% retention in a study of mentalisation-based treatment program with an art therapy component for people with lived experience of BPD , 77% in Many participants commented on the benefits of the peer facilitation aspect of the program. Often it was explained that this provided an authentic space to understand the skills, increasing practical application and willingness. Whilst the mechanism of symptom change in this program is currently unknown , the IPS model of mutuality in meaningful relationships appeared important to the participants. This is consistent with previous research on the perceptions of people living with BPD, where peer workers were described as being a role model of the use of skills, and providing the opportunity for genuine validation with similar lived experience . This stThe artmaking component of the program was also highly valued. Participants reported improved capacity to both understand and express themselves through the process of artmaking. In addition, participants reported greater applicability and integration of the skills. People living with BPD reported a stronger sense of their own values, strengths and internal processes, describing the capacity for artmaking to facilitate the process of developing self-concept and identity. Whilst traditional evidence-based treatments for BPD show significant benefits in acute symptom improvement , it is The online format of the program may have also contributed to an inclusive treatment pathway for people who have challenges accessing traditional therapeutic approaches. The current program was designed for the online format due to the global pandemic. Accessibility was improved using the online modality, including those due to Covid-related social restrictions, geographical considerations, and the intrinsic interpersonal difficulties associated with the disorder. The Relational Model suggests that the underlying difficulty of BPD stems from problematic relationships developed with the self and other over time . As suchThe present study has limitations. The small sample size limits the generalisability of the findings. The pilot involved an uncontrolled pre-post evaluation design, and therefore further research is required to replicate and extend on these results to increase confidence in the benefits of the program, and whether these are maintained over time. In order to maximise inclusivity and respect the diagnosis that participants self-identify with to describe their experience, diagnosis of BPD or other comorbid conditions were not formally assessed by the researchers involved in this study. Future research may consider integrating formal assessment procedures to confirm the diagnostic profile of participants. This may facilitate greater understanding of whether the program assists people living with BPD more than other diagnostic or comorbid groups. Of interest, this information may also assist to understand how people with emotion dysregulation understand and self-identify with their difficulties, and the therapeutic preferences of people with lived experience . In addition, the program was delivered online in the context of the Covid-19 global pandemic, which allowed participation from regional and remote areas, and international locations, who otherwise would not have been able to attend the program. However, whilst other art therapy programs have demonstrated successful implementation using telehealth , it is The present pilot study evaluated a peer-led art-based skills program for people living with Borderline Personality Disorder. The results are promising, demonstrating strong acceptability of the program, positive and meaningful qualitative feedback, and large improvements in emotion regulation across time. The use of peer facilitation founded in the principles of Intentional Peer Support was found to be valuable in providing authentic validation and promoting willingness and hope for living well. In addition, the arts-based approach, informed by the dialectical framework and skills, was beneficial in promoting accessibility of the strategies and development of a sense of self. We found evidence that art therapy facilitated by a peer with lived experience assisted in the recovery of people living with Borderline Personality Disorder.


Beyond physical pain, patients with coronavirus disease 2019 (COVID-19) experience psychological anxiety during and after quarantine, often facing negative perceptions when returning to their communities. This study evaluated a health consultation program in Korea for post-quarantine patients with COVID-19, designed to help them return to their communities. The program was conducted from 9 March to 5 June 2020, in Daegu, Korea. In total, 20 doctors and 504 recovered patients were surveyed via questionnaire. The survey, comprising open-ended questions rated on a five-point Likert scale, was based on the Context–Input–Process–Product program evaluation model. Reliability was assessed, and descriptive statistics were obtained. A regression analysis was performed on factors affecting product (output) areas. As a main result, both doctors and recovered patients evaluated the program positively. The mean program effectiveness score was 4.00 in the doctors’ evaluations and 3.95 in the patients’ evaluations. Moreover, the input and process variables affected the product. This first-of-its-kind health consultation program proved to be an effective practical intervention for patients returning to the community after an infectious disease; it also highlights aspects that could increase satisfaction in systemized subsequent programs, with input and process areas for patients and doctors. In December 2019, the first case of coronavirus disease 2019 (COVID-19) was reported in Wuhan, China, as pneumonia with an unknown cause. Subsequently, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spread rapidly, with the World Health Organization declaring it a pandemic in March 2020. Social distancing measures were enacted worldwide .Korea has maintained a social distancing policy since the onset of the pandemic. After November 2021, the Korean government sought approaches to slowly phase out social distancing. However, due to continued serious disease transmission, the government had to maintain its social distancing policy. Korea’s policy requires that those infected self-isolate for at least 10 days, even if asymptomatic. The isolation period is longer for those with symptoms, especially if the symptoms are persistent. Studies show that, in addition to the physical symptoms of COVID-19, patients also experience psychological symptoms, such as anxiety, due to the isolation . Some alIn Daegu, Korea, the spread of SARS-CoV-2 was rampant after March 2020, and certain religious groups were particularly affected . The rapTo address this issue, the Daegu-Gyeongbuk branch of the Korean Academy of Family Medicine developed a Follow-up Health Consultation Program, operated by Daegu City and the Daegu Medical Association, for patients who had recovered from COVID-19 . Twenty Several models for program evaluation exist; these include the Context–Input–Reaction–Output model, the Kirkpatrick model, the Logic model, and the Context–Input–Process–Product (CIPP) model. Of these, the CIPP model is frequently recommended for evaluating educational programs. Stufflebeam discussed the potential of the CIPP evaluation method in 1971 . The CIPThe Follow-up Health Consultation Program goals included: identifying the physical symptoms that remained after the patients emerged from quarantine, arrangements for the treatment of these symptoms in clinics or hospitals, and addressing the psychological problems associated with the patients’ return to their communities. The program and its protocol are described in detail in Kim et al.’s study .As a part of the program, doctors call patients to check on their physical health and provide them with health education and psychological counseling. They also advise patients if additional treatment is needed and coordinate hospitals visits. The patients in the program receive educational brochures with guidelines related to their recovery, face masks, and hand sanitizers.The program involved 20 consulting doctors and 1679 patients and was conducted from 9 March to 5 June 2020. A total of 75 patients—who submitted incorrect mobile numbers, did not respond to doctors’ calls, or had been admitted to hospitals two or more times—were excluded.After the program ended, evaluation questionnaire surveys were sent to all participating patients and doctors. The schematic flow of the study is presented in The doctors’ survey comprised 18 items—of which six were open-ended questions—assessed using a five-point Likert scale from “1 = strongly disagree” to “5 = strongly agree”; a higher score reflects greater satisfaction with the program. The structured questionnaires were sent to and returned by the doctors via e-mail . Table 1The patients’ survey included 16 items—of which three were open-ended questions—assessed using a five-point Likert scale. As with the doctors’ scores, a higher score denoted greater satisfaction with the program. The patients received a link to the online survey (Google Survey) through text messages sent after the consultation period ended . Table 2p-value below 0.05 was considered statistically significant.We assessed reliability using Cronbach’s alphas for each area in both questionnaires. Descriptive statistics (mean and standard deviation [SD]) were used to summarize the data pertaining to each item. A multiple regression analysis was performed using the context, input, and process evaluation results of patient data as independent variables to determine the factors affecting the evaluation of the product (output) area. Open-ended questions were assessed using qualitative analysis. All statistical analyses were performed using IBM SPSS Statistics version 26.0 software . A The mean age, sex, and mean work experience of the doctors are presented in The mean age of the patients was 40.79 years. Of them, 77.2% had no comorbid diseases, while 7.8% had been quarantined for less than one week, and 16.2% for more than one month .In the doctors’ evaluations, all areas received a mean score of three or more out of five. Each evaluation area received a high average score: 4.30 for the validity of the goal in the context area, 4.20 for the usefulness of support items in the input area, 4.00 for the responsiveness to calls in the process area, and 4.00 for satisfaction with the program in the product (output) area. In the process sub-areas, the degree of interference with work and personal time scored higher than three, indicating that the doctors felt somewhat inconvenienced by their participation in the program .All areas received an average score of three or higher out of five in the patients’ evaluations. The mean score for being informed of the program’s purpose in the context area was 4.22, for the expertise of doctors in the input area was 4.10, for easy-to-understand explanations from doctors in the process area was 4.24, and for satisfaction with phone consultations in the product (output) area was 4.10. The lowest mean score (3.64) was in the process area, which measured whether the support provided was timely .2 = 0.71, p < 0.001). The scores of the input and process variables were significantly associated with program satisfaction, while context area scores were not , program usefulness and satisfaction were also rated favorably. However, the mean score for the degree to which participation in the program interfered with their work or personal time was also high. This implies that doctors perceived some disruption in their daily lives because of their participation in the program, primarily because they had to make time for phone consultations during or after work, while also completing their usual job tasks. Regarding phone consultations, 60% of the calls took between 11 to 20 min, exceeding the usual time allocated for doctors’ outpatient clinic consultations . AdditioIn the context area, doctors evaluated the possibility of achieving the program’s goals in a very positive light. However, in the product (output) area, goal achievement scored lower than in other areas, such as program satisfaction. This may be due to the lack of face-to-face consultations to identify patients’ physical problems, which was a goal of the program. Doctors expressed dissatisfaction with the limitations of distance consultations and their own limited understanding of the new infectious disease. According to a United States study on COVID-19, telemedicine has been shown to be useful in infectious disease pandemic situations; however, it has limited effectiveness when there is a need to address physical symptoms . In futuThe patients positively evaluated all program areas. Specifically, high scores were assigned to the expertise of doctors, easy-to-understand explanations, and patients’ satisfaction with phone consultations. Furthermore, the mean scores for program effectiveness and goal achievement were 3.95 and 3.91, respectively, indicating a high overall evaluation of the program. However, the mean score for whether support items were provided in a timely manner was low, at 3.64. As support items were delivered through the postal system, there was a two- to three-day delay due to insufficient administrative support, especially as the number of patients increased. In future programs, private or alternative systems should be considered for delivering support items in a timely manner. The patients’ open-ended responses indicated that, overall, their anxiety was resolved, and their concerns addressed, confirming the program’s usefulness.The psychological difficulties of those infected with COVID-19 have been recognized as an important issue globally ,19,20,21In terms of reducing post-infection social stigma or negative perceptions, no significant effects were observed. One possible reason may be the lack of face-to-face consultations. Moreover, the evaluation did not allow us to assess individuals’ specific social situation, such as psychiatric medical history and economic condition. Additionally, many of the patients surveyed belonged to specific religious groups. This factor should be considered in future programs when setting goals and establishing processes.The regression analysis determined which CIPP area variables affected a patient’s satisfaction with the program. Program satisfaction was high when the input and process variables were evaluated positively. Generally, in the case of education or improvement programs, if satisfaction with the resources and operational process is low, the satisfaction with the program will also be low ,24. TherIn this study, the context variables did not affect program satisfaction. In contrast to the doctors’ surveys, patients evaluated only “being well-informed of the program’s purpose” as a context variable—and very positively so—such that there was no statistical significance. Nonetheless, if the context is not clearly presented in the program, it can significantly reduce the likelihood of achieving its goals . Thus, tDespite its clear contributions, several limitations of the study warrant further comment. First, as the study considered a single city dominated by a specific religious group, it is difficult to generalize the results. However, we believe that our study’s findings are valuable, as the operation of such a program and its evaluation are necessary to address the emotional distress caused by an infectious disease such as COVID-19. Second, not all questionnaire items in the study were part of the CIPP evaluation. However, if a questionnaire is too closely based on an evaluation method, or if several different surveys are conducted, the response rate may decrease; this is especially true for patients with COVID-19 who may feel overburdened, given that they are experiencing the physical effects of the disease as well as the psychological effects of social issues. Thus, questions outside the CIPP evaluation areas were added to achieve our objectives. In the future, the questionnaire design could be improved by including only a manageable number of questions based solely on the evaluation model. Third, when researchers interpret the opinions expressed in open-ended responses, there is a risk of subjectivity. However, each of the six authors independently extracted the opinions and reached an agreement, after which the results were integrated; thus, we attempted to minimize the influence of each researcher’s subjective judgment.Despite these limitations, this novel study contributes to the literature by systematically evaluating the first post-quarantine health consultation program for patients with COVID-19. Furthermore, as the program was conducted in Daegu, where the first serious outbreak occurred in Korea in 2020, the program represents a positive and proactive move by the government.Ultimately, the health consultation program in Korea for recovered patients with COVID-19—who continue to have physical, psychological, and social problems after isolation—was deemed effective, useful, and highly satisfactory. Accordingly, developing and operating a well-structured systematic follow-up health consultation program should be considered an effective intervention for patients recovering from infectious diseases, especially those who are likely to suffer psychological problems due to social isolation.


Prior studies suggest that mobile health physical activity programs that provide only weekly or daily text-based health coaching evidence limit the efficacy in improving physical activity in adolescents with overweight or obesity. It is possible that incentives, combined with health coaching and daily feedback on goal success, may increase program efficacy; however, such programs have not yet been tested with adolescents with overweight and obesity.This study aims to examine the feasibility and acceptability of a 12-week, incentive-based, mobile health physical activity program with text-based health coaching, goal setting, and self-monitoring for adolescents with overweight or obesity. Program adherence and changes in tracked physical activity , body mass, and body fat are assessed.A total of 28 adolescents aged 13 to 18 years with a BMI ≥90th percentile participated in the program. Of the 28 participants, 2 (7%) were lost to follow-up; thus, data from 26 (93%) participants were used in analyses.P=.006) and steps (P<.001) and significant pre- to posttest improvements in body fat percentage (P=.04).Participant-reported acceptability was high, with all mean ratings of text-based coaching, Fitbit use, and the overall program being >5 on a 7-point scale. In addition, 85% (23/26) of participants reported that they would like to continue to wear the Fitbit. Program adherence was also high, as participants wore the Fitbit on 91.1% (SD 12.6%) of days on average and met their weekly goals for an average of 7 (SD 3.5) of 11 possible weeks. There were no demographic  differences in the percentage of days participants wore their Fitbit. Across the 12-week study, there were significant improvements in tracked daily active minutes (The pilot program improved adolescent physical activity and physical health. A larger factorial design trial with adaptive daily goals may clarify the role of each program component in driving physical activity. Adolescence is a high-risk period to have overweight and obesity , which mmHealth physical activity intervention programs can reduce the cost and time burden for participants , and theIncentives may be an effective method of increasing engagement in health behaviors in adolescents , and incThis manuscript describes the pilot-testing of a 12-week mHealth incentive program, which targets increased physical activity in adolescents with overweight or obesity. This study is a single group, pre–post pilot of a physical activity program with fixed daily and adaptive weekly physical activity goals, incentives for meeting goals, daily text-based feedback about goal success, and weekly text-based supportive coaching. The primary aim is to examine program acceptability  use, and overall program impressions). Secondary aims are to examine (1) program adherence , including demographic factors related to Fitbit use, and (2) changes in physical activity . Finally, an exploratory aim is to identify the possible benefits of this program on body mass and body fat.Participants included (N=28) adolescents with overweight or obesity . Most participants  reported having private health insurance. Inclusion criteria included the following: age 13 to 18 years; BMI percentile >90; having broadband wireless internet at home; living at home; and permission from the pediatrician to engage in a physical activity program , which was noted in electronic health records. Although the 85th and 95th percentile are the standard cutoffs for having overweight and obesity , respectEthics approval was received from the Dartmouth Hitchcock Medical Center institutional review board (study ID 29591). The pilot trial protocol is available by contacting the corresponding author. At the time of the study, a registry of pilot trials was not required. In addition, this was an open trial; thus, preregistration did not occur.A partial Health Insurance Portability and Accountability Act waiver was obtained, which allowed for the recruitment of participants from a general pediatric clinic at Dartmouth Hitchcock Medical Center using data from electronic medical records to identify potentially eligible adolescents based on age and BMI. Letters were sent to parents of potentially eligible adolescents, which provided instructions on how to opt out of the recruitment process . After the 2-week opt-out period, research staff called families to tell them more about the study, assess their interest in participating, and determine eligibility  to self-monitor their activity levels throughout the program. Participants then downloaded the Fitbit mobile application on their smartphone and created a Fitbit account, the credentials of which were shared with study staff for monitoring. The staff ensured that the Fitbit was set to track daily goals of 60 moderate to vigorous minutes, as recommended by the Centers for Disease Control and Prevention  and Amerweekly goal, praising them if they did and providing encouragement if they did not; these texts also informed adolescents of the amount of incentives earned that week, if applicable, and the goal for the upcoming week. SMS text message content was standardized and focused specifically on activity goals. However, if technical problems emerged with using or syncing the Fitbit, coaches provided support to adolescents and parents in navigating the technology. To provide immediate monetary reinforcement during the baseline and shaping phases, incentives for adherence to program weekly goals were loaded onto a reloadable debit card once per week for the first 9 weeks. To assess whether physical activity habits were formed rather than activity levels being contingent upon immediate monetary reinforcement, incentives for adherence to program weekly goals were only loaded at the end of the final week for the last 3 weeks in the program .An incentive program commenced on the first Monday following completion of the baseline visit to ensure consistency of feedback across participants. Following the program, adolescents and parents attended a poststudy assessment and returned the Fitbit and Fitbit charger and iPod Touch if distributed. Adolescents earned US $100 for completing each assessment that included a lengthy functional magnetic resonance imaging session , although functional magnetic resonance imaging data were not analyzed in this study.Program acceptability was assessed via participant report of the helpfulness of the text-based coaching , experience of using the Fitbit  and overall experience with the program . Participants rated each question on a 7-item scale, with higher scores indicating more positive experiences. In addition, participants were given a Fitbit Charge HR to track their activity, including active minutes and steps. At the end of the intervention, participants indicated whether they would like to continue wearing the Fitbit after the end of the program.Program adherence was assessed via the percentage of days participants wore their Fitbit  and the number of weeks each participant met their goals and obtained bonuses. Accordingly, participants’ earning totals were considered as indicators of program adherence.valid day of Fitbit use is often characterized as having at least 1500 steps tracked [As described above, all participants were provided with a Fitbit Charge HR as part of the program. Steps and active minutes were tracked while the Fitbit was worn. Fitbit’s algorithm considers active minutes to be those with ≥3 metabolic equivalents and only calculates active minutes after 10 minutes of continuous activity. Steps and active minutes were automatically summed daily. The Fitbit Charge has been used in prior studies to measure tracked activity, and a  tracked ,30. On tA Tanita TBF-300A scale was used to measure body fat and weight at baseline and after the program. The Tanita scale uses bioelectrical impedance analysis, or gentle electric signals through the body, to measure body fat. The Tanita TBF-300A scale has been shown to be a reliable and valid measure of body fat ,32 and wt tests were also used and compared the percentage of days participants wore their Fitbit based on sex, age, and percentage of the 95th BMI percentile for age and sex. Third, program adherence subgroups were identified, as determined by participants’ weekly activity goal success during the first and second half of the program. Fourth, to assess changes in tracked daily active minutes (dependent variable) and steps taken (dependent variable) during participation in the program, multilevel growth modeling was used. Time  was included as an independent variable. Due to the small sample size, demographic covariates were not included in the multilevel models. A separate model was computed for active minutes and steps. To improve model fit, for the model with steps as the dependent variable, the step count variable was transformed, with participants’ daily step count divided by 100. Full information maximum likelihood estimation was used to account for missing data (<10%). Finally, paired-sample t tests were used to examine changes in body mass  and body fat from baseline to after the intervention. Change scores were used , with negative change scores indicating decreases in body mass and body fat.First, descriptive statistics were used to examine program acceptability and adherence. Second, to further characterize adherence to daily Fitbit use, correlations and two-tailed Full descriptive statistics of program acceptability are documented in t test t24=−1.08; P=.29), age , or percentage of the 95th BMI percentile for age and sex . Participants also met their weekly goals and bonus goals for an average of 7 (SD=3.5) and 5 (SD=3.8) of 11 possible weeks, respectively, each earning an average of approximately US $300 of the US $510 maximum.Full descriptive statistics of program adherence are documented in Three program adherence subgroups were identified: sustained program adherence, nonsustained program adherence, and limited program adherence. Participants categorized as exhibiting sustained program adherence  included participants who met at least 80% of their weekly goals across the full program period . Participants categorized as demonstrating nonsustained program adherence  included participants who met 80% of their weekly goals in the first half of the program during shaping  but not during maintenance and fading . Participants categorized as showing limited program adherence  included participants who did not meet 80% of their weekly goals during either half of the program. Of the 28 participants, 1 (4%) exhibited delayed program adherence .P=.006) and steps (P<.001), with an average increase of 20.41 (SD 34.27) tracked active minutes per day and 924.00 (SD 2141.59) tracked steps per day found across the entire intervention. There was also a significant intercept by slope effect in the daily active minutes multilevel model (P=.005), indicating that adolescents who demonstrated greater active minutes per day at the start of the intervention also demonstrated the greatest increases in active minutes per day over the full intervention period. Full test statistics are presented in Findings demonstrated a significant increase in tracked daily active minutes (P=.04). Change in percentage of the 95th BMI percentile for age and sex was nonsignificant (P>.05). Full test statistics are documented in Findings demonstrated a significant decrease in body fat percentage from week 1 to week 12 .This was a single group pre–post pilot study testing a novel mHealth physical activity program for adolescents with overweight and obesity. Despite the lack of a control group, this study provides preliminary support that an mHealth program with incentives and text-delivered support and feedback may be an effective nonintensive alternative for increasing tracked physical activity and reducing body fat in adolescents with overweight and obesity. Adolescents reported a positive experience with each program component  and with the program as a whole. There was a low attrition rate . These indicators of program acceptability were consistent with program adherence data that showed that participants wore their Fitbit over 90% of program days, and Fitbit adherence was consistent across sex, age, and percentage of the 95th BMI percentile for age and sex groups. In addition, most participants regularly met their weekly goals, especially in the shaping period of the program from weeks 1 to 6, during which 64.3% (18/28) of participants met at least 80% of their weekly goals. In addition, participants acquired bonuses for exceeding their predetermined goals in about half of the total weeks (5.3 of 11 possible weeks), suggesting positive program outcomes. Across all enrolled participants, the program led to significant improvements in tracked time spent engaging in daily active minutes with metabolic equivalents ≥3 and steps, as well as significant reductions in body fat percentage, although no significant changes in body mass were noted. Notably, these changes in health and health behavior were achieved with minimal human resources ; thus, the findings might not be generalizable to racially or ethnically diverse samples and adolescents from low SES backgrounds. In addition, the sample includes adolescents at or above the 90th BMI percentile, such that the generalizability of findings to those adolescents with overweight but between the 85th and 89.9th BMI percentile is unknown. Although there was an even distribution of girls and boys, an assessment of differences in changes in physical activity and physical health based on sex could not be conducted. Notably, pubertal status was not assessed in this study, and pubertal changes over the course of the program may have affected physical health results.usual activity levels, and the true baseline level of activity before receiving the Fitbit was unknown. However, such an impact would lessen the increase in activity over the course of the intervention.In addition, it is possible that participants who enrolled in the study were more motivated to increase physical activity than those who declined. Nonetheless, we used universal outreach  to identify those individuals who may be eligible but otherwise were not seeking behavioral care for the management of overweight or obesity. In addition, owing to the novelty of the Fitbit and monitoring of activity by the text-based health coach, there may have been an increase in activity during the baseline period relative to There also may have been measurement errors, as a recent meta-analysis indicated that Fitbit estimates of step count and moderate to vigorous active minutes might be less accurate than research-grade accelerometers , and theThere are also limitations in the program structure. First, owing to the lack of a universal guideline, participants were given a 10,000 daily step goal, which is lower than the physical activity recommendations used in other studies  . Second,clinically significant. To allow for comparison of findings across studies, it should also be a priority to establish a universal cutoff for Fitbit wear time (or adherence), including the minimum number of steps and minutes required to be considered a full day of wear time, as well as universal guidelines for daily step and active minutes for adolescents, specifically. Future studies could investigate the incremental efficacy of each program component using a factorial design to compare the efficacy of an SMS text message only program, incentives only program, and SMS text message plus incentives program in increasing physical activity among adolescents with overweight or obesity. Researchers might also consider investigating the differential effects of varying incentive magnitudes on changes in tracked physical activity. Given that health insurers are poised to deliver incentives for health behavior change, a randomized trial of this program could advance the evidence-based use of incentives by health insurers. Researchers may also examine whether other incentive types, including parent-led behavior contracting and provision of rewards other than incentives , may evoke similar results. An adaptive program might also be tested, in which participants are initially offered smaller incentives for a brief period, with participants who demonstrate limited program adherence switching to a more intensive program . In addition, the text-based mHealth program used in this study could be automated by using technology such as the Fitbit application programming interface to automate goal setting and deploy reinforcing SMS text messages every day. The program could also be lengthened to support greater reductions in body fat, as well as be modified to target only increased active minutes or only step counts. Importantly, all future iterations of the program should implement an intervention mapping framework [This pilot study provides preliminary support of the acceptability and feasibility of a remote, mHealth program comprising tracked physical activity goals, incentives, and text-based support from health coaches to improve tracked physical activity and reduce body fat in adolescents with overweight and obesity. Replication of findings in a randomized trial by involving a larger, more diverse sample is warranted, including youth with BMI percentiles >85th, and may further clarify the utility of the program in addressing the pediatric overweight and obesity epidemic. Importantly, future studies should explore individual-level factors  that may predict program adherence and changes in tracked activity levels, as well as examine whether program effects are ramework .Future research may also consider assessing mediators and moderators of change. Other adaptions that might be tested include personalized daily adaptive goals  and incentives for meeting daily goals to increase the number of adolescents meeting weekly goals. In addition, a longer fading period  may support sustained physical activity improvements and long-term program response. Finally, future research should investigate whether participation in an incentive program may increase motivation for additional lifestyle behavior changes  associated with overweight and obesity, as well as guide the long-term prevention of having overweight and obesity and their common comorbidities during adulthood. Combined, such avenues for future program development may allow for a highly scalable, accessible, and sustainable program for adolescents with overweight and obesity.