Ovarian cancer (OVCA) patients may carry genes conferring cancer risk to biological family; however, fewer than one-quarter of patients receive genetic testing. “Traceback” cascade testing —outreach to potential probands and relatives—is a possible solution. This paper outlines a funded study (U01 CA240747-01A1) seeking to determine a Traceback program’s feasibility, acceptability, effectiveness, and costs. This is a multisite prospective observational feasibility study across three integrated health systems. Informed by the Conceptual Model for Implementation Research, we will outline, implement, and evaluate the outcomes of an OVCA Traceback program. We will use standard legal research methodology to review genetic privacy statutes; engage key stakeholders in qualitative interviews to design communication strategies; employ descriptive statistics and regression analyses to evaluate the site differences in genetic testing and the OVCA Traceback testing; and assess program outcomes at the proband, family member, provider, system, and population levels. This study aims to determine a Traceback program’s feasibility and acceptability in a real-world context. It will account for the myriad factors affecting implementation, including legal issues, organizational- and individual-level barriers and facilitators, communication issues, and program costs. Project results will inform how health care providers and systems can develop effective, practical, and sustainable Traceback programs. Ovarian cancer (OVCA) is the deadliest gynecologic malignancy and the sixth leading cause of cancer death ,2,3. In BRCA1 and BRCA2 genes are a significant risk factor for OVCA [BRCA1 mutations and lifetime risks of 69% for breast cancer and 17% for OVCA for females with BRCA2 mutations [Germline mutations in for OVCA . One stuutations ,9,10. Alutations ,12.BRCA mutations are inherited in an autosomal dominant pattern, the first-degree relatives of OVCA patients have a 50% chance of inheriting the mutation. It is recommended that at-risk relatives be referred for BRCA testing (termed “cascade testing”); however, the uptake among eligible relatives is low—only about 20–30%, [Furthermore, since  20–30%,  even whe 20–30%, . Therefo 20–30%, ,16.Numerous barriers at the system, provider, and patient levels impede the uptake of genetic testing among OVCA patients and their relatives. At the system level, several questions remain regarding the real-world feasibility, acceptability, effectiveness, and costs of Traceback screening programs given the current structure of the US health care system and its national and state privacy laws ,18,19. ABRCA mutations: (1) proband identification, (2) proband testing, and (3) cascade testing of at-risk relatives [A National Cancer Institute (NCI) workshop proposed the Traceback framework as a means of creating a standardized testing process where none previously existed ,18. The elatives . The funThe Traceback framework is a staged model involving patients, family, providers, and systems to increase identification of individuals with hereditary cancer risk. In 2016, international experts in genetics, medical and gynecological oncology, clinical psychology, epidemiology, genomics, cost-effectiveness modeling, pathology, bioethics, and patient advocacy convened with NCI scientists to develop the three-phase Traceback cascade testing framework. The Feasibility and Acceptability of Cascade Traceback Screening (FACTS) study will use the Traceback framework to identify OVCA patients who were not previously referred for current standard genetic testing and offer genetic testing for the familial risk for breast or ovarian cancer to patients and their relatives . Our oveFACTS is a prospective observational feasibility study to determine the acceptability, feasibility, and effectiveness of a Traceback cascade testing program in multiple populations and health systems. Our specific aims are:Aim 1: Evaluate a legal solution to facilitate Traceback cascade testing across states and health systems, including examining the potential to use the Public Health Exception for the Health Insurance Portability and Accountability Act (HIPAA) at the federal and state level and reviewing privacy laws that impact the notification of family members.Aim 2: Engage stakeholders to determine culturally appropriate language and communication strategies to facilitate the genetic testing of OVCA patients and relatives.Aim 3: Describe the uptake, effectiveness, and feasibility of a Traceback program in three health systems serving different populations.3a.Phase I Proband identification: Identify previously diagnosed individuals without current standard genetic testing to measure and compare outcomes at the proband, organization, and population levels.3b.Phase II Genetic testing of probands: Offer genetic testing to individuals identified in Aim 3a using the culturally appropriate language and communication strategies identified in Aim 2 to measure and compare clinical outcomes at the proband, family member, organization, and population levels.3c.Phase III Traceback cascade testing of relatives: Approach individuals from Aim 3b with a genetic result related to hereditary cancer using Aim 2-derived language and strategies to encourage cascade testing of relatives to measure and compare clinical outcomes at the proband, family, organization, and population levels.Aim 4: Explore the implementation, service, and clinical outcomes related to a Traceback program at the proband, family member, provider, system, and population levels as guided by the Conceptual Model for Implementation Research to examine the contextual barriers and facilitators of a Traceback cascade testing program [ program ,27.We will conduct a multisite prospective, observational, feasibility study of the implementation, service, and clinical/health outcomes expected from a Traceback program . The FACThe study sites are Geisinger, Kaiser Permanente Mid-Atlantic States, and Kaiser Permanente Washington, representing geographic diversity, racial and ethnic diversity , diversiAll three sites have clinical geneticists and/or genetic counselors on staff. Standard practice at each site is to offer genetic counseling and subsequently provide multi-gene panel testing as part of usual clinical care . GeneticAim 1: Legal solution to facilitate Traceback cascade testingThe primary outcome for Aim 1 is a description of each state’s privacy law. The secondary outcome is guidance for the public health exception to facilitate cascade testing. A separate manuscript will be published in the future that details the methods behind data collection and data analysis for this aim.Aim 2: Determine culturally appropriate language and communication strategies to facilitate Traceback cascade testingData collection. Aim 2 will engage stakeholders in discussion to determine culturally appropriate language and communication strategies about genetic testing for individuals with OVCA and Traceback cascade testing of relatives. Aim 2 will also explore other barriers and facilitators to Traceback programs , Kruskal–Wallis, and Pearson Chi-square tests as appropriate.Aim 3a will enumerate the number of individuals with ovarian cancer within each system (Traceback Phase I). The number and percentage of those with and without a prior genetic result will be summarized and compared across systems using the Pearson Chi-square test. Any differences found to vary across systems  will be considered as potential confounding variables; a logistic regression model will be adjusted for these variables to estimate the unbiased effects of system differences. The adjusted percentages and odds ratios (ORs) with 95% confidence intervals (CIs) will be presented, with the emphasis on the estimate of the effect rather than the significance level. Similarly, we will summarize the number of women in the sample who are living and still receiving care in their respective system regardless of genetic testing status, compare this number across systems, and use a logistic regression model to adjust for potential confounding. Lastly, we will use the Kaplan–Meier method to estimate the survival curve. Aim 3b will examine the genetic testing uptake among living eligible probands and other outcomes as listed in Aim 3c focuses on measuring similar outcomes in relatives. We will follow the same summary and modeling of the outcomes as described for Aim 3b.Aim 4: Evaluate Traceback program outcomes using the Conceptual Model for Implementation ResearchData collection. Aim 4 will focus on implementation outcomes to explore the feasibility, acceptability, sustainability, and cost of a Traceback program for ovarian cancer using the Conceptual Model for Implementation Research  in non-White populations. Ultimately, this study is the first to combine legal, micro-costing, qualitative, and quantitative research with the implementation of science outcomes to provide a comprehensive exploration of best practices for implementing a Traceback program for OVCA patients and their families. The results of this study could also encourage an increase in genetic testing for OVCA genes.The FACTS project will generate information useful for health care providers to design and implement Traceback testing programs for OVCA and other conditions. FACTS includes an in-depth legal analysis of HIPAA privacy laws, which will inform how relatives may or may not be contacted, as well as culturally appropriate messaging and outreach strategies to reach target populations. FACTS will offer insight into how to operationalize and incorporate resources such as the time-sensitive family testing options offered by nationwide laboratory companies into a cohesive Traceback cascade testing program. From a clinical perspective, this study will provide critically needed data on the prevalence of cancer predisposition gene mutations  and for genomics professional organizations and networks. Findings will be disseminated through the Health Care Systems Research Network (HCSRN), a network of 19 health system research centers, as well as to the larger genomics research community through networks such as eMERGE, IGNITE, and ClinGen. Study results will be published in peer-reviewed journals and shared with operational leaders within health systems who are seeking to replicate the program. Tools and learnings created from the micro-costing, engagement activities, and the assessment of implementation outcomes will be consolidated as appropriate into the guidance and toolkits to be disseminated.There are potential limitations to the project described here. First, the Traceback model includes several proposed methods for proband identification to improve the notification of at-risk relatives, including using pathology samples from deceased ovarian cancer patients as a source of genetic information; this project omits that approach. Once the regulatory components and the feasibility of a Traceback program are outlined as described above, future work may explore the use of pathology samples followed by outreach to at-risk family members. Second, the number of interviews planned is insufficient to capture the experiences and preferences of individuals and relatives from gender minority groups ; however, these individuals exist in our populations, and Aims 3 and 4 will identify reach and uptake in such individuals. A follow-up study is warranted to explore the preferences for outreach and communication within these groups. Third, this project is taking place amid the COVID-19 pandemic, which will result in unintended but necessary changes to the study approach. The project team is tabulating changes due to COVID-19 and will incorporate that information into the study findings.Few studies have examined the implementation outcomes of cascade testing, and this project addresses this critical unmet need ,27,32,33


The co-occurrence of domestic violence with alcohol and other drugs significantly increases the severity of abuse and violence experienced by family members. Longitudinal studies indicate that substance use is one of few predictors of men’s continued use of, or desistance from, violence. Recent developments in men’s behaviour change programs have focused on men’s attitudes and behaviour towards their children, and the exploration of interventions that address the needs of all family members. However, the research evidence is limited on the most effective elements of men’s behaviour change programs in promoting the safety and wellbeing of child and women victim survivors.This study aims to build on the existing evidence by trialling the KODY program which addresses harmful substance use by men who also perpetrate domestic violence; the safety and wellbeing of women and children; the needs of children in their own right, as well as in relationship with their mothers; and the development of an ‘all-of-family’ service response. The evaluation of these innovations, and the ramifications for policy development to support less fragmented service system responses, provide the rationale for the study.Caring Dads standard’ (the comparison group). Psychometric tests will be administered to fathers and mothers at baseline, post-program and at 3-month follow up. Data collection will occur over three years.A quasi-experimental design will be used to assess the primary outcomes of improving the safety and wellbeing of mothers and children whose (ex)partners and fathers respectively participate in KODY , when compared with ‘By building the evidence base about responses to co-occurring domestic violence and substance use, this study aims to develop knowledge about improving safety outcomes for women and children, and to better understand appropriate support for children in families living at the intersection of domestic violence and substance use. It is anticipated that study findings will point to the ramifications for policy development to support less fragmented service system responses.https://www.anzctr.org.au/) was lodged on 20 December 2021 (Request number: 383206)—prospectively registered.An application for registration with the Australian and New Zealand Clinical Trials Registry ( The co-occurrence of domestic violence (DV) with alcohol and other drugs (AOD) significantly increases the severity of abuse and violence experienced by family members . ProgramCaring Dads programs developed by Scott, Kelly, Crooks and Francis (2018) in Canada, and adapted and evaluated in Australia and internationally [Caring Dads program in Australia identified a range of issues which the proposed innovative program and evaluation seeks to address to improve health and wellbeing outcomes and the effectiveness of the program. These include a) the problems associated with the co-occurrence of DV with AOD issues; b) the requirement to develop safe ‘all-of-family’ responses, particularly for families who are not separating; c) the lack of attention to the support needs of children whose fathers are attending Caring Dads; and d) the requirement for evaluation measures for programs to address fathering, DV and AOD.A new wave of programs for men who use violence which focus on their fathering, attitudes and behaviour towards their children, has seen tionally –7. The rKids First (Caring Dads) and Odyssey House Victoria) integrates and enhances the work of two non-government organisations in Melbourne, Australia. Kids First Australia auspices a Caring Dads program. Odyssey House Victoria provides a range of alcohol and other drug treatment programs including services for parents affected by problematic substance use and their children, such as Kids in Focus, a family centred program that emphasises the safety and wellbeing of children in addition to providing parenting and family support.The quasi-experimental trial discussed in this paper was designed to pilot and evaluate an all-of-family response to the complex issues for children, women and men living with AOD and DV. The KODY program  backgrounds. In addition, a considerable number of men display co-occurring mental health issues [Victorian domestic homicide reviews indicate that between 50–60% of murders occur in the context of the use of alcohol or other drugs  suggestih issues . Unemploh issues , 14, 15.h issues .There is widespread recognition that DV is predominantly a gendered issue of men’s violence towards women . HoweverDespite the established importance of DV and AOD to child outcomes, comparatively little notice has been taken of attitudes and behaviours towards children in interventions for either of these problems. AOD programs, although well researched for their efficacy regarding reduced use of substances, have very seldom considered or included improved fathering and better child outcomes as a program or evaluation outcome. Recent overviews of research on men’s behaviour change (MBC) programs stress the need for robust evaluations of more targeted programs that respond to the diverse needs and characteristics of the men referred , 24. CurCaring Dads programs and focusing on children as well as women victim survivors. The ‘all-of-family’ model is a DV-informed framework developed by the Safe and Together Institute to address the needs and responses to each family member living at the intersection of DV and AOD [A more recent, although still early, development in the area of MBC programming , 28 and  and AOD . This ap and AOD .Kids in Focus program; and the development of an ‘all-of-family’ service response. The evaluation of these innovations and the ramifications for policy development to support less fragmented service system responses provide the rationale for the proposed trial.Through assessing the effectiveness of the innovative KODY program, this study aims to address many of these issues: harmful substance use by men who also perpetrate DV; the needs of children in their own right, as well as in relationship to their mothers through the Odyssey House Victoria The primary aim of the study is to test the effectiveness of the KODY program in supporting safety for mothers and children with experiences of DV perpetrated by men with problematic substance use. It is hypothesised that the program will:Caring Dads;1) improve the safety and wellbeing of women whose (ex)partners participate in KODY Caring Dads.2) improve the safety and wellbeing of children, whose fathers participate in KODY Secondary outcomes include fathers’ use of DV and of AOD (see Table Caring Dads standard’ [Caring Dads program (T2) and three months after the end of the Caring Dads program (T3). Supplementary analyses of participation data obtained through file reviews will be conducted to examine the extent to which intervention effects vary according to ‘dose’ level of expanded service. In the context of this trial, ‘dose’ levels refer to the number of group work sessions attended by fathers, the degree of engagement by mothers with the Child and Family Wellbeing Practitioner and the Kids in Focus activities in which children, mothers and fathers participate.The effectiveness of the KODY program will be tested with a quasi-experimental design comparing outcomes between KODY  and ‘tandard’   consists of the following components.The trial intervention (KODY) is a new model of service that integrates the work of both Kids First and Odyssey House Victoria to address the complex issues for children, women and men living with AOD and DV. The ‘Group work—17 sessions, manualised content, focus on engaging men to examine their fathering, increasing child centred fathering, including respect and support of children’s mothers, and changing patterns of harmful fathering.Case Coordination—case coordination between group intervention and mother contact practitioners.Services for mothers and children—mothers of children with whom fathers have contact are provided support and safety planning through a Child and Family Wellbeing Practitioner.Caring Dads standard’ program plus the following changes and additions.KODY  comprises the ‘Group work—‘Caring Dads standard’ program with added curriculum materials and specialist AOD facilitation of group sessions.Case Coordination—Collaborative case coordination for fathers by KODY DV and AOD practitioners. Additionally, Kids in Focus offers all-of-family case management, which will be integrated with Caring Dads case coordination.Services for mothers and children—In addition to support from the Child and Family Wellbeing Practitioner, linked intervention with all family members  through Kids in Focus, including parenting education and support and recreational and therapeutic groups for children and their families.Caring Dads. Adherence to intervention protocols will be monitored through regular meetings with program staff, recording of participation across time and program components, and observation of selected KODY Caring Dads group sessions.The intervention group will be implemented by a co-facilitation team with expertise in working with fathers to address DV and AOD. The comparison group will be implemented by professionals experienced in DV and the delivery of The target group for study is fathers who use DV and AOD. Fathers and their (ex)partners eligible to participate in either the trial or the comparison programs will be eligible for the study. In both groups, eligible fathers must have contact with at least one child, be able to understand and speak English, and have capacity to maintain participation and attend group work regularly, bearing in mind their substance use status. Mothers participating in trial and comparison programs are also eligible for the study.Kids in Focus as part of the intervention group. The comparison group will comprise fathers who are referred (or self-refer) to a ‘Caring Dads standard’ program. These fathers will have been identified as using DV or being at risk of using DV.The trial program (KODY) will constitute fathers referred for their problematic AOD use and identified by AOD case managers as using DV, or at risk of using DV. KODY fathers must be receiving support through Odyssey House Victoria clinicians. These men, their children with whom they have contact, and the children’s mothers, will be offered support through Caring Dads standard’ program will occur through referrals from a range of services, including other programs within Kids First and Odyssey House Victoria and external services such as the statutory child protection service, and other Melbourne-based DV and AOD services.Recruitment into KODY and into ‘Caring Dads standard’ programs will be assessed and invited at intake by program staff to participate in the evaluation, as will men’s (ex)partners. Researchers will follow up with interested participants to obtain informed consent which will be confirmed at each stage of data collection. Program staff will be consulted regarding any possible safety issues.Fathers entering KODY or ‘Caring Dads standard’ groups are planned for each year of the study, each with up to 10 participants. Assuming a medium effect size (based on Cohen’s d), a power of 0.80 and a two-tailed test across independent groups, this study will need to involve reports from 128 mothers, 64 in each of the KODY and comparison groups. Reported participation rates for evaluations of Caring Dads programs range from 62% [Kids in Focus program for women and children, this study will need to enrol 170 families across groups.Three KODY groups and three ‘from 62%  to 97% [from 62% . AssuminThe primary outcome is reported improvements in the safety and wellbeing of women and children affected by the behaviour of KODY participants. Primary and secondary outcomes, and proposed measures, are outlined in Table The primary outcome of improved safety and wellbeing of women is measured by the Composite Abuse Scale Standard Form , a 30-itSeveral measures are used to assess the primary outcome of improved safety and wellbeing of children. These include the Emotional Dysregulation Scale, the Interpersonal Mindfulness in Parenting scale, the Brief Infant–Toddler Social and Emotional Assessment and the Strengths and Difficulties Questionnaire, as outlined below.Caring Dads’ focus on explicitly teaching skills related to emotion regulation. In the KODY evaluation, fathers answer questions about their own emotion regulation abilities, while mothers are asked to report on their (ex)partner’s emotion regulation abilities. Items are scored on a 7-point scale (‘not true’ to ‘very true’). Items assess domains of emotional experiencing, cognition and behaviour.The Emotional Dysregulation Scale  is a 12-Caring Dads program, with items suitable for assessing fathers’ skill improvement, as well as their ability to regulate their emotions when parenting, complementing findings from the Emotional Dysregulation Scale. Items are scored on a 5-point scale . Items assess mindful parenting across five dimensions: listening with full attention, non-judgmental acceptance of self and child, compassion for self, emotional awareness of self and child, and self-regulation in parenting.The Interpersonal Mindfulness in Parenting Scale  is a 27-The Strengths and Difficulties Questionnaire  is a 25-The Brief Infant–Toddler Social and Emotional Assessment  is a 42-The secondary outcome of fathers’ use of DV relates directly to the primary outcomes of improved safety and wellbeing for women and children, as DV is seen as a major risk. Further, the secondary outcome of fathers’ use of AOD, will also contribute to the primary outcomes, given the evidence that where fathers use both DV and AOD, the severity of abuse, and therefore the risk to safety is significantly higher . The secFor the KODY evaluation, the research team has created an 18-item measure, adapted from an existing measure used in Project Mirabal, a multi-site longitudinal study of domestic violence perpetrator programs, conducted between 2009 and 2015 in the UK . The adaThe adapted Mirabal uses items from four of the six indicators of change outlined in Project Mirabal, asking mothers to assess their (ex)partner’s respectful communication, fathering, awareness of self and others, and their own perceptions of their children’s experiences (e.g. ‘My children worry about the safety of their mother’). Fathers who have used violence are provided with a similar range of statements, across the same domains.Caring Dads trial [Caring Dads program. Items are scored on a 4-point scale (‘strongly disagree’ to ‘strongly agree’) and includes items from three subscales: behavioural self-soothing, recognising signs of anger and self-talk.The Anger Management scale is a 12-item subscale of the Personal Relationships Profile, a validated measure developed by Straus and colleagues . This scds trial , and fouThe Brief Irritability Test  is a 5-iThe Alcohol Use Disorders Identification Test (AUDIT)  is a 10-Caring Dads program (T2) and three months after the end of the Caring Dads program (T3)  to obtain demographic data about participants, as well as program data, such as ‘dose’ level of expanded service, referral sources, other services involved with clients and accountability arrangements (e.g. court orders).Program participants and professionals will be assigned a unique research ID to ensure anonymity. Outcome measures will be completed online and stored in a secure Qualtrics database. Other research data will be stored in the University of Melbourne OneDrive system with access restricted to members of the research team. Interviews and focus groups will be audio-recorded with participant consent. If these are conducted online, recording will be through Zoom and saved directly into OneDrive. Recordings of face-to-face interviews and focus groups will also be uploaded onto OneDrive. Password protection on files will be used as required.Intent to treat analyses will be used so that all service users are included in the primary analysis, allowing for variations in levels of participation in KODY and including those who drop out prematurely. Assuming differences between groups, supplementary analyses will examine the extent to which intervention effects vary according to whether families received a higher or lower ‘dose’ of expanded service. Recognizing that fathers are not being randomly assigned to the intervention and comparison group, care will be taken to compare time 1 characteristics of groups and, if necessary, use propensity analyses in post-group and follow-up comparisons. Each component of the study will be analysed separately in the first instance, then brought together to triangulate data collection and analysis, before a final synthesis.Caring Dads standard’ (the comparison group).This paper outlines the protocol for a quasi-experimental trial of the innovative KODY program, an all-of-family response to co-occurring issues of DV and AOD. A quasi-experimental design was deemed most appropriate for the trial, as randomized allocation to trial and comparison groups is impracticable and potentially unethical in the context of the program and the service system in which it operates. Primary outcomes for the study are the increased safety and wellbeing of mothers and children whose (ex)partners and fathers respectively participate in KODY , when compared with ‘Through an analysis of both outcomes measures and file reviews to provide ‘dose’ level information, the study aims to build the evidence for program responses that increase the safety and wellbeing of women and children impacted by men who use DV and AOD. The innovative aspects of the program—addressing the co-occurrence of DV and AOD with a specific focus on fathering, providing support to all members of the family and centring the needs of children—are all drawn from both academic evidence and practitioner expertise. The innovative nature of the program may result in small sample sizes in the early years of implementation, which may limit the power of analyses, but the authors’ experience with the evaluation of other innovative programs suggests that sample sizes will increase with program longevity. Operating within the DV and AOD service sectors respectively, the two partner organisations—Kids First and Odyssey House Victoria—have a shared focus on the needs of vulnerable children and train their staff in a common framework (Safe and Together) which brings a shared language and conceptualisation of the intersection of AOD and DV . In the


Cost-effectiveness analysis is widely adopted as a means to inform policy and decision makers in setting priorities for healthcare resource allocation. In resource-constrained settings, decision makers are confronted with healthcare resource reallocation decisions, e.g., moving funds from one or more existing healthcare programs to fund new healthcare programs. The decision-making plane (DMP) has been developed as a means to graphically present the results of reallocating available healthcare resources when healthcare program costs and effects are uncertain. Mapping a value function over the DMP allows the analyst to value all possible combinations of net costs and net effects that may result from reallocating available healthcare resources under conditions of uncertainty. In this paper, we extend this approach to include a change in portfolio risk, stemming from a change in the portfolios of funded healthcare programs, as an additional source of uncertainty, and demonstrate how this can be incorporated into the value function over net costs and net effects for a risk-averse decision maker. The methodology presented in this paper is of particular interest to decision makers who are risk averse, as it will help to better incorporate their preferences in the process of deciding how to best allocate scarce healthcare resources. Resource scarcity occurs in different contexts, i.e., with fixed, shrinking or increasing budgets, and so DMs must decide where additional resources will come from to fund new healthcare programs ,2,3. In rificed” .Cost-effectiveness analysis (CEA) is presented and used as a means to maximize the health benefits produced from available resources . The anaWhile this alternative decision rule is straightforward in the absence of uncertainty associated with program costs and effects, in reality, both costs and effects are stochastic. The decision-making plane (DMP) graphically presents the distribution of all potential net cost and net effect pairs that may result from reallocating resources between programs under uncertainty . FurtherThe present paper is different from the current stream of literature addressing risk aversion in cost-effectiveness analysis ,17,18 beIn the Methods section, we review the alternative decision rule under certainty and uncertainty. We then introduce the idea of increasing or decreasing the program portfolio variance in costs and effects associated with reallocating resources. In the Results section, we show how a change in the program portfolio variance in costs and effects can be incorporated in and how this affects the value function with regard to net costs and net effects. Finally, we discuss areas for possible future research.The rationale behind the alternative decision rule is described in detail elsewhere ,2,8,19. program A, which will replace program a, we need to find a program B, which will be replaced by program b, such that both (1) and (2) hold:A − Ca and ΔC(B) = CB − Cb, and ΔE(A) = EA − Ea and ΔE(B) = EB − Eb.To fund program A, which requires an additional USD 60,000,000 compared to the current program a. Program A generates 60 Life Years (LYs) more than program a for the same patient group. Assume that we identify programs B and b, which meet the requirements of Equations (1) and (2). Program B, if cancelled and replaced by b, releases USD 70,000,000, but reduces health outcomes by 20 LYs. Therefore, removing program B from the program portfolio to fund the additional costs of program A leads to a net gain of 40 LYs without the need to increase resources. As a hypothetical example, program A may reflect treating patients with lung cancer with surgery and radiochemotherapy, while program a may reflect treating the same patients with radiochemotherapy only. Moreover, as a hypothetical example, program B may reflect treating patients with Hepatitis C infection with antiviral drugs, while program b may represent not treating the same patients with antiviral drugs.Consider the example of four healthcare programs, as shown in A, a, B and b. Finally, we refer to portfolio costs and effects to denote the costs and effects of the jointly funded programs before and after reallocating resources, i.e., Pf(A + b) and Pf(a + B), respectively.The results of this deterministic resource reallocation policy are displayed on the DMP . The DMPprogram B to program A may lead to net costs and net effects that do not fall within Quadrant I of the DMP [Because the costs and effects of healthcare programs are uncertain, the alternative decision rule may lead to a non-zero probability that reallocating available resources from  the DMP ,20,21. Cprogram A has a lower standard deviation for both costs and effects than existing program a, indicating that uncertainties in costs and uncertainties in effects are reduced by replacing program a with program A  with portfolio Pf(A + b)  and a 30.3% probability that the available resources will be insufficient to support this reallocation, but will result in a net health gain  a. The joprogram A, we cannot determine whether the uncertainty of the (new) program portfolio Pf(A + b) has been increased or reduced compared to the (old) portfolio Pf(a + B) or by how much. This is because a wide variety of possible combinations of variances for program costs and effects may lead to the same program B and program a has a larger variance than a portfolio with program A and program b  of net costs and net effects on the DMP reflects the uncertainty associated with program costs and program effects of all four programs, i.e., those that are replaced and those that are supported by available resources. As such, the variances of the normal distributions displayed on the DMP are defined, in general, asDifferent healthcare programs may exhibit different levels of uncertainty. For example, antiretroviral treatment for HIV infection has been found to be variance increasing for both costs and effects . Before program A and program b now have a higher standard deviation for costs and effects than program B and program a. Therefore, by allocating available resources to program A and b, the overall uncertainty of the portfolio Pf(A + b) is increased compared to Pf(a + B)  and third (Q3) quartiles of a distribution for the costs or effects of the portfolio of funded programs and is estimated asAlthough there may be different approaches to presenting and valuing changes with regard to portfolio risk, in this paper, we present a simple but robust approach using the quartile coefficient of dispersion (QCD), which is less sensitive to skewness . The QCDA higher QCD indicates greater dispersion, and can thus be used to compare the degree of uncertainty between programs descriptively. There are other metrics that may be used to describe dispersion, such as the coefficient of variation, commonly used in finance, which is estimated by taking the ratio of the standard deviation to the mean. However, in a highly skewed distribution, the standard deviation is strongly affected by the tail of the distribution and outliers. The QCD is less sensitive to skewness as quartiles of a distribution are used for estimating the QCD, and differences between the extreme values in the tail are disregarded. Pf(A + b) and Pf(a + B). Consider Scenario 1 (Pf(a + B) for portfolio costs and portfolio effects before implementing program A is 0.069 and 0.089, respectively  for costs and effects, before reallocating available resources to A and b, is 0.027 and 0.034, respectively  for costs and effects is increased to 0.074 and 0.070, respectively  and net effects (DMPE) on the DMP follows a BVN distribution, and the quadrant probabilities can be derived by integration of the BVN. The probabilities of net costs and net effects falling in each quadrant of the DMP plane as a result of reallocating resources may also be calculated analytically. In situations where a disease model can be evaluated analytically, this would be the preferred approach to describe the structure of the system. However, in more complicated disease models, analysts usually resort to simulation for evaluating a cost-effectiveness model. As shown in For computational purposes, it is convenient to standardize the variables on the DMP, as follows. The x- and y-axes in the resultant standardized BVN are:x and y.In order to calculate the probability The probabilities for the other three quadrants are as follows:x and y distributions are given by Equations (3) and (4). In order to implement these calculations, one requires the input values for the parameters program A and, similarly, for programs a, B and b. The correlation required in (8)–(11) can be derived as:Numerical evaluation of the integrals, such as in (8), will provide accurate values for the quadrant probabilities of the joint distribution displayed in A related issue here is that the parameters may need to be estimated from sample data and, therefore, unless sample sizes are very large, there will be sample variation that affects the parameter estimates and, hence, the estimated quadrant probabilities. As such, there will be additional uncertainty in the positions and sizes of the ellipses in f and gain g functions for each quadrant of the DMP may be defined as shown in 1 and α2 represent the decision maker’s valuation of resource reallocation outcomes x and y and reflect a progressively increasing (if >1) or decreasing (if <1) rate of gain or loss as one moves away from the origin of the DMP [1 and α2 are specific to a DM (here we will assume α1 = α2 = 2), and can be interpreted as a risk aversion parameter [Decision makers may not give equal value to every potential outcome of a reallocation of available resources on the DMP. Gafni et al.  suggeste the DMP . The exparameter . β1 and β2; β1 < 1 and β2 > 1), since this may represent a disutility to a risk-averse DM. On the other hand, if moving from portfolio Pf(a + B) to portfolio Pf(A + b) is variance decreasing, and hence represents an increase in utility to the DM, then the value of net gains (and net losses) on the DMP may be increased (or decreased) . If reallocating available resources between programs is neither variance increasing nor variance decreasing, then β1 = β2 = 1. Using the net gain and loss functions defined above (β1 = 1.1 and β2 = 0.9 for portfolio risk reduction and β1 = 0.9 and β2 = 1.1 for an increase in portfolio risk (1Cβ and effects 1Eβ for gains (and β2C and β2E for losses). However, for demonstration purposes, we assume 1Cβ = 1Eβ = β1. We emphasize that these are hypothetical estimates and are presented to demonstrate how a change in portfolio variance may affect the value function. In reality, one would need to elicit the respective preferences from the DM . In the present paper, we introduce further exponents  due to an increase in uncertainty for both costs and effects  and the net gain is then increased to 33.04. In our example, net losses are reduced to 0.69 for the variance-reducing scenario and increased to 1.03 for the variance-increasing scenario. That is, the ranking of the reallocation decision changes from being indifferent between Scenario 1 and Scenario 2, when ignoring the change in portfolio risk, to preferring Scenario 1 over Scenario 2 when we include the change in portfolio risk in the value function for a risk-averse decision maker.To estimate expected net gain and expected net loss using the value function defined above , we drewIn this paper, we extended the previous work dealing with the uncertainty of the net costs and net effects when reallocating resources to include the risk aversion of the DM towards an increase in portfolio risk. In the present paper, we take a different approach than other current studies, which address risk aversion and affordability within the context of cost-effectiveness analysis. In a recent paper, for example, Lomas suggested a modification of the net benefit approach using a threshold ratio as a decision rule, which is adjusted depending on the budget impact of a new program . HoweverHow changes in portfolio risk translate into a value function depends on the preferences of a decision maker. In this paper, we assume that a DM prefers less risk, and this is reflected in a smaller variance, ceteris paribus. Other value functions, such as a preference for min–max reduction, would also be possible. We would then seek to reduce the spread of the costs and effects of the portfolio after resource reallocation. A mean–variance utility function may also be appropriate to model the trade-off between expected return and risk ,28,29. IIn the present paper, we assumed risk aversion towards both costs and effects. However, this approach could be extended to only include risk aversion towards effects and risk neutrality towards costs (or vice versa). In that case, we would limit the value function for gains and losses to health effects only and consider a resource reallocation as long as expected costs do not exceed the anticipated budget level. Furthermore, the value functions used for estimating gains and losses could be modified depending on the preferences of a decision maker, e.g., if a decision maker would prefer to weigh uncertain gains in health outcomes more than potential losses, the exponents of the value function could be adjusted accordingly. Changes in the before–after variance of a portfolio of healthcare programs may represent important components of portfolio risk that need to be assessed and included in the value function of net gains and net losses over the DMP. Empirical research is needed to evaluate how DMs value such changes in portfolio risk. In addition, future research should focus on defining and linking a preference function, such as a utility function or prospect theory value function, to the change in the variance of the costs and effects of the portfolio of funded programs. The methodology presented in this paper is of particular interest to DMs who are risk averse, as it will help to better incorporate their preferences in the process of deciding how to best allocate scarce healthcare resources.


As a main challenge in paramedical faculties of medical sciences, freshmen lose interest in their academic field of study and then job motivation. Lack of developed knowledge about their academic field and unfamiliarity with their future job’s tasks and roles contribute to freshmen’s job motivation loss. Various interventional programs have been implemented to improve students’ job motivation by familiarizing them with their future job’s duties and responsibilities.This was one-group pretest-posttest design study in 2019–2020. Students grouped into pairs of a freshman (shadowee) with a senior (shadower) in a clinical setting during shadowing program. This program helps freshmen to comprehend and discover realities of their academic field and can help them change their perspectives regarding their future job’s duties and responsibilities. The shadowees’ main task was reflective observation on operating room events and interactions and to be active in the program, several tasks e.g., how to wear gloves, guns, and disinfect equipment were assigned to them exclusively under the supervision of senior students. The Hackman and Oldham’s Job Diagnostic Survey (JDS) questionnaire and a novel Job Motivation Survey (JMS) questionnaire were distributed among participants.P < 0.001). Moreover, the overall pre-test and post-test scores of the job motivation survey questionnaire were 25.16 (± 9.75) and 39.80 (±5.18), respectively; this score difference was statistically significant (P < 0.001).Fifty freshmen majoring in operating room participated in the shadowing program from November 2019 to January 2020. Before and after the program, they completed Hackman and Oldham’s job diagnostic survey and researcher-made job motivation survey questionnaires. Results were indicative of a significant difference in job diagnostic survey questionnaire score, where overall pre-test and post-test scores before and after the intervention were 57.78 (±9.78) and 68.58 (±5.02), respectively; the score difference was statistically significant (Shadowing program improved freshmen’s realistic perception of their future job’s duties and responsibility, and hence enhancing their job motivation and job recognition. As future work, in various disciplines, further studies need to evaluate the impact of such interventional programs in providing early insights for freshmen as well as in providing guidance on their plans for education, and future job. Clinical education phase plays a major role in developing professional competencies in student majoring in paramedical field, including Operating Room (OR) . The OR Studies show that a large gap exists between preclinical and clinical training for OR students , 3. CurrAccording to previous studies, several factors may affect students’ perspectives regarding their future profession; changes in educational environment, different teaching methods in clinical settings, learning in unpredictable situations, and lack of knowledge regarding their future profession’s duties and responsibilities in clinical environment –12. VaniShadowing programs has been implemented by reputable medical education centers to provide early clinical experiences for students and to increase their participation in learning process , 21. Thei) enhancing the participatory learning climate; (ii) empowering senior students to teach; (iii) improving communication skills among students of different levels; (v) consolidating the rudimentary knowledge of pre-clinical students; and, (vi) application of theoretical knowledge in practice [In most shadowing program, by linking pre-clinical students with near-peers, prepares them for their future job and changes their job perspective . During practice , 29. Accpractice , 31. Howpractice .To the best of our knowledge, shadowing programs have not yet been implemented for OR students; and hence, this study evaluates a shadowing program for OR students, examining whether this program can increase OR freshmen’s job diagnostic and job motivation. Previous studies have not used valid and reliable tools to measure the impact of this program on student’s future job diagnostic and job motivation, nor have they focused on the implementation process of this program as well as its effectiveness from participants’ perspectives , 33. ConThis was one-group pretest-posttest design study in 2019–2020 on paramedical faculty’s OR students. After receiving approval for the research plan and code of ethics (IR.ZUMS.REC.1398.300) from Zahedan University of Medical Sciences, samples were taken from OR freshmen using convenience sampling. Informed consent of the participants was obtained. Probationary students and any from other universities were excluded from this study.The study was carried out in Zabol and Zahedan Universities of Medical Sciences, Iran. OR curriculum has been delivered as a traditional Flexnerian 4-year undergraduate bachelor program. The undergraduate bachelor of OR curriculum in Iran is divided into two phases; including preclinical (three semesters) and clinical phases (five semesters). Preclinical phase of curriculum has been based on the model of teaching that engage students in classrooms setting. In the clinical phase, OR students are exposed to the operating room environment, during which they become acquainted with the realities of the chosen major in the clinical environment and apply theory in practice.A multi-aspect evaluation of the shadowing program was performed before, during and after the program implantation.To select clinical students (shadowers), the Hackman and Oldham’s Job Diagnostic Survey (JDS) questionnaire and a novel Job Motivation Survey (JMS) questionnaire, designed for this study, were distributed among juniors and seniors. Moreover, to select appropriate shadowers, total GPAs of 36 candidate students were inquired from the university. Fourteen out of thirty-six students with the highest JDS scores and total GPA were selected as shadowers, and an orientation session was held for them. In this session, the shadowers were introduced to the program and were informed about the program’s goals, process, and tasks. In this session, on how to introduce operating room tools, equipment, and different parts of the operating room, to present the duties of operating room staffs, to create an attitude towards the job to students and encourage them to rethink the expected professional roles was emphasized. After completing the program, a cooperation certificate and a gift card were awarded as incentives to the shadowers.ii) introductions of OR team members; and, (iii) the need to communicate effectively with other OR team members. The shadowees were requested to refrain from any involvement in patient treatment and were only allowed to observe the clinical environment reflectively as well as identify upperclassmen’s role and tasks in OR carefully. After the sessions, the coordinator randomly assigned a shadowee to a shadower while considering schedule compatibility in clinical rotations. To coordinate the presence of students in OR environment, necessary moral and legal approvals were formerly obtained, and administrative correspondence was formally established between the teaching hospitals and the university.This program was as voluntarily part of the educational course for OR freshmen as shadowee. Shadowee were newcomers to the undergraduate operating room curriculum. Before the program, the shadowees were required to complete JDS and JM questionnaires. Then, for the shadowees, briefing sessions on the following were held: (i) professionalism and etiquette in OR;  were assigned to them exclusively under the supervision of senior students when not interfering with their duties and those of other OR team members. At the end of each encounter with operating room, shadowee can discuss with his/her shadower on their observations and reflections of the experience in reflection time. Furthermore, coordinator meet with students in two smaller groups for reflection. This component was added to the program because there was enough time for discussion about the shadowees’ experiences. Coordinator of the program guide the session by posing reflective questions addressing feelings, experiences, and views of shadowees.One week after the end of the program, JDS and JMS questionnaires were manually distributed and filled by the shadowees. If the completed JDS questionnaires were not received, reminders were sent a week later to collect them all.In this study, a 15-item shortened version of Hackman and Oldham’s JDS questionnaire was used . HackmanA novel questionnaire was designed for this study to examine OR students’ motivation to pursue profession related to their academic field of study. This questionnaire consisted of five general questions rated on a scale from completely agree (9) to completely disagree (1). In order to evaluate the questionnaire’s psychometric properties, we checked content-related evidence for validity  by 10 experts qualitatively . Expertsp = 0.05. Respectively, Effect Size (ES) above 0.8, 0.5, and 0.2% was indicative of strong, moderate, and weak effectiveness of this program.Statistical analysis was performed using SPSS software (version 26) . Paired-P < 0.001). In all items of the job motivation survey questionnaire, the intervention program was strongly effective (ES ≥ 0.8) . Out of 50 participants, there were 27 (54%) female and 23 (45%) male. Before and after of the program, total pre-test score of job motivation survey questionnaire was a mean of 25.1 with 9.75 standard deviation, whereas the mean of the total post-test score of job motivation survey questionnaire was 39.8 (5.18 standard deviation). OR freshmen’s job motivation significantly increased considering pre-test and post-test scores of job motivation survey questionnaire (P < 0.001) (Table The mean (std. dev.) of pre-test and post-test total scores of JDS questionnaire were 57.78 (9.78) and 68.58 5.02), respectively. Pre-test and post-test job diagnostic level was significantly different before and after the intervention . In addition, the post-test total scores of both JDS and JMS questionnaire was 0.18, which was not statistically significant (P = 0.2).The Pearson correlation coefficient between the pre-test total scores of both JDS and JMS questionnaires was 0.64, which was statistically significant  Design and implementation of shadowing program for OR students, (ii) Evaluation of this program’ impact on increasing freshmen’s job diagnostic and motivation; (iii) Purposeful selection of the shadowers; and, (iv) The empowerment shadowers through holding orientation program. A limitation of this study is that the program’s implementation and maintenance can be challenging, especially in critical environments such as OR. Paramedical faculties may act differently in their teaching hospitals, management policies, or availability of administrative support, all of which may affect the implementation of this model elsewhere. Another limitation is that the program evaluation was conducted as a cross-sectional survey and the results of the intervention were measured over a short period of time. Therefore, it is suggested that future studies be designed longitudinally or by considering a longer follow-up period to consider the effect of time on the impact of the intervention, as well as future longitudinal studies on the impact of the intervention on other job variables, including responsibility should be designed.The selection process of shadowers and monitoring proper implementation of the program are significant responsibilities to ensure that the guidelines are correctly followed and that the newcomer students (shadowees) are satisfied. Nevertheless, shadowing should be included in the formal curriculum of OR students before entering the clinical phase to help them plan their professional future. As future work, the impact of the shadowing program on the job motivation in the longer term should be evaluated though follow-up studies in various fields of study.In shadowing program, through students’ exposure to OR and familiarity with OR-related profession, they gain realistic insights on their academic field of study and their future profession. In shadowing program, using the opportunity to be present in the operating room environment and the real space of the clinic by accompanying senior students, it had an effect on students’ initial knowledge of the operating room and increase their motivation.


The past decade has seen growing interest in interventions that build resilience as a complementary practice to trauma-informed care. From school-based programs focused on self-regulation and academic success to programs that support the well-being of disadvantaged populations or healthcare workers at risk of burnout, the concept of resilience is being used most commonly for programming that builds the capacity of individuals to adapt under conditions of adversity. Critiques have raised concerns that resilience-promoting programs demonstrate bias toward changing individual-level factors such as cognitions , behavior , or attachments  which help people adapt to socially toxic situations without changing access to the resources they require to overcome exposure to adverse psychosocial factors. This trend belies advances to the theory of resilience which support a more contextualized, multisystemic understanding of how external protective factors (resources) enhance individual qualities (ruggedness) and vice versa. Building on a multisystemic description of resilience, the R2 Resilience Program© was developed and piloted with six different populations ranging from clients of urban social services to workers in a long-term care facility, managers in the health care sector, staff of a Fortune 500 corporation, students in a primary to grade 12 school, and adult volunteers affiliated with an international NGO. Focused on building both individual ruggedness and enhancing people’s resources (the two Rs), the program provides contextualized content for each population by selecting from 52 resilience promoting factors with a strong evidence base to create training curricula that enhance the personal qualities and social, physical, and institutional resources most likely to support resilience. This paper reviews the justification for a multisystemic approach to designing resilience interventions and then explains the process of implementation of the R2 program. Preliminary findings are reported, which suggest the program is experienced as effective, with evaluations ongoing. There are a growing number of programs to build resilience designed to complement trauma-informed interventions which have a narrower focus on alleviating symptoms after exposure to stress or adversity. This focus on resilience shifts attention from the factors that cause mental illness  or family, community, or institutional dysfunction (at a systemic level) to how people survive and thrive despite the challenges they experience.Different conceptualizations of resilience over the years have influenced programs designed to improve coping capacity. Early perspectives likened resilience to a largely static, dispositional personality traits which were thought to explain why some individuals emerged relatively unscathed from disadvantage or trauma . Such pePrograms that emphasize individual change may produce short-term benefits, but adaptations are likely to decrease their impact on well-being over time unless an individual’s environment is also transformed  in ways In this paper, we explore the justification for a multisystemic approach to designing resilience interventions and then describe a novel resilience program that utilizes this approach. The program is outlined, principles and processes are described, and early evaluation findings are reported, which suggests the program is experienced as effective.ruggedness may be sufficient to support well-being under conditions of normal stress, the greater the barriers to functioning experienced by an individual, the more important resources become.The bifurcation of fields of practice into micro and macro systemic interventions is being challenged by the emerging science of multisystemic resilience . To illuWe term this multisystemic perspective of resilience “R2” . In practice, individuals with more internal capacities  tend to be more likely to take advantage of opportunities for relationships and to exploit opportunities for financial or academic success e.g., , while iResearch supports this dual focus on ruggedness and resources. In a study of Danish schoolchildren, The R2 Resilience Program© was designed with principles of implementation science  to ensurThe second design flaw common to interventions that promote resilience is that programming may target change at the wrong systemic level, producing little or no sustainable experience of well-being. For example, there is evidence that bullying among school-aged children is best addressed by changes to school policy (making the school a safe space), or by helping children who are bullied find same-age peers to befriend them  rather than by empowering individual children to resist bullying on their own . While sThere are many programs and interventions already in use that purport to build resilience. Some are aimed at the general public and target common adversities like stress  that have been well studied and which are known to be associated with resilience. These factors are applicable to multiple populations and conditions of adversity. A complete list of all 52 factors is included in The R2 Resilience Program© is a curriculum-based approach to enhancing the resilience of populations experiencing atypical stress or adversity. The program is contextualized to fit the specific needs of an organization, educational institution, or business setting. Ideally, every implementation of the R2 approach accomplishes the following:•Explores the general and specific risks faced by individuals in each work or service setting. Whether these risks are related to individual factors like psychological trauma, institutional conditions like corporate restructuring, or a major event like a natural disaster, R2 responds to the issues that are most pressing locally. To tailor the intervention, R2 leaders meet with members of a senior management team and those responsible for the health and well-being of staff/students/clients to ensure the program is adapted to the specific challenges and opportunities people experience daily.•Identifies the range of resilience factors that are right for each setting. Different organizations, institutions, and businesses need different protective factors to support their staff/students/clients.•Identifies the right audience for the program. Content is created that focuses on the needs of individuals and groups seeking to build their resilience, as well as those holding positions of responsibility for others. The R2 curriculum can be adapted to meet the needs of individual staff, customers, and students, or adapted to include case material to support the work of organizational leaders.•Formats the delivery of the program to ensure it fits each organizational setting and the time and resources each has to build resilience. R2 curriculum is intentionally designed to be turned into workshops and online resources. Delivery can vary from a series of short interactive seminars delivered face-to-face to day-long events, websites, and apps, depending on what each organization needs. The program can be offered in different mediums, such as face-to-face coaching, group workshops, or online, and can be facilitated by an R2 expert, an internally trained trainer, or self-directed by participants through direct access to learning materials. This flexibility is accomplished through the modular formatting of the content and repurposing curriculum as new applications are requested.•Makes the training materials easily accessible to ensure the program is sustainable. Once an R2 program is developed, organizations continue to access these resources at a minimum cost. As the program grows, changes can be reflected in the materials that are shared.•Supports the design of an evaluation to measure outcomes. Evaluations can range from brief and minimally intrusive to far larger, multisite longitudinal studies of outcomes depending on the capacity of each setting and available funding. In most cases, evaluation tools are built into the delivery of the R2 curriculum through pre- and post-tests and self-assessment exercises embedded in each module that explores a different resilience factor. In this way, areas of intended change are measured and data can be made easily accessible to end-users.To date, R2 has been piloted with six different populations ranging from clients of urban social services, to workers in a long-term care facility, managers in the health care sector, staff of a Fortune 500 corporation, students in a primary to grade 12 school, and adult volunteers affiliated with an international NGO. Focused on building both individual ruggedness and enhancing people’s resources (the two Rs), the program provides contextualized content for each population. Specific goals for the intervention include: engaging participants in meaningful conversations about the many (multisystemic) factors that nurture and sustain resilience; providing practical strategies for improving individual ruggedness and access to, and use of, social, built, and natural resources; and when required, adding to the competencies of professionals tasked with enhancing the resilience of individuals and communities. To create and then implement the content to fulfill these goals, the program proceeds through four phases of intervention; contextualization, offsite program development, implementation, and evaluation.A Delphi process  prior toTo conduct the Delphi survey, members of an organization that are best placed to help determine which of the factors would form a priority R2 program are invited to rank order the 52 resilience factors. Respondents may include a mixture of senior management, program administrators, or even recipients of the forthcoming program (if the target group is not the whole of the organization) who may be the “experts by experience” . These i•Something that individuals are in need of;•Something not being adequately addressed elsewhere by the organization, institution, or business;•Something that individuals will find important ;•Something that a program could realistically improve.Participants are encouraged to distinguish resilience factors that are important and already being addressed by their organization, institution, or business, and those that they would like to see new programming address. In this way, R2 extends the capacity of an organization for resilience rather than simply replicating effective programming already in place.If this process is conducted during a live session , responses are automatically pooled and are then presented back to the group immediately. The group is reminded that these are only results from a first round and that the highest ranked choices are a potential shortlist for the program. Time is then allocated for discussion, where participants are invited to discuss the outcome of the first round, which may involve reflecting on the appropriateness of those items currently at the top of the lists or arguing for the importance of factors that were not very highly ranked. We also contribute to these discussions where relevant; for instance, noting factors that have been successfully included in similar contexts.A second round of ranking then takes place. If the outcome is substantially different from the first round, further discussion takes place to review these changes. Two to three rounds are normally required to reach a general consensus . Once acWhile the Delphi process helps to ensure that the most relevant factors related to resilience are the focus of programming, results may be skewed by who was selected to complete the ranking with those most in need of resilience training but less motivated to participate silenced by the process. While consensus can be achieved, this does not necessarily mean that Delphi conclusions reflect consensus across an entire organization. Results should therefore be considered cautiously and in light of who was selected as an expert and whether additional areas of expertise are required to refine results.The R2 team uses the information gathered during Phase 1 to develop a tailored version of the R2 program that fits the need of the host organization, institution, or business. The time required to develop the program will vary depending on the extent of the adaptation needed but typically can be accomplished within 4–6 weeks. An evaluation plan and evaluation materials are also developed at the same time. The short timeframe helps to maintain the momentum of the work already done. Each module contains common elements. These include:1.An introduction to the module and what participants can expect to learn. The material, while theoretically sound, is meant to be accessible and brief, with a heavy emphasis on the activities R2 Resilience Program© participants can do to integrate new resilience-enabling patterns of behavior into their lives.2.A brief overview of the resilience factor, as well as the typical risk factors which threaten individuals and make the factor more or less important in different contexts. This section of the module includes a plain language scientific summary that reviews the available evidence for the effectiveness of each resilience factor.3.Validated measures from the literature that can be used to help participants reflect on the impact each resilience factor may have on their lives. Where non-validated measures are recommended , anecdotal evidence of their performance as assessment tools is provided.4.A series of sensitizing questions that can be used in place of standardized measures to facilitate self-reflection and group discussion of the module’s content. These sensitizing questions are starting prompts to help participants consider the wide range of implications each resilience factor has on their lives and the lives of others across their organization, institution, or business.5.Case studies that illustrate how the module’s content reflects actual stories of successful recovery, adaptation and transformation in each organization, institution, or business. These case illustrations are developed through an appreciative inquiry process in which those attached to a specific setting are asked to tell stories of past successes developing resilience and to identify the constellation of personal qualities and external supports that they received which made their success possible.6.Sample lesson plans for facilitating workshop content specific to each resilience factor. Where available, visual materials like PowerPoint slides and links to online videos may be included. These lesson plans are provided in the format requested by the host organization, institution, or business but are usually malleable so that the program can be easily adapted to different groups of participants with different learning needs or the amount of time they have to devote to the training.7.The expected outcomes from completion of the module. Participants are provided with a summary of the changes they can expect from their use of the module with specific details about potential benefits to mental health and one’s ability to cope in social contexts that produce stress. Specific learning objectives are also developed for each setting, which are tailored to the selected protective factors required for the program. For example, a program involving self-efficacy will include learning objectives related to conceptual understanding of the concept and application of activities tailored for the selected setting that improve self-efficacy.With the modules developed, the R2 team returns to the host setting, this time presenting a general introduction to R2 to as many staff and other stakeholders as possible. The R2 experts then either (a) work with a small group of individuals to conduct a trial of the program  or (b) train a group of individuals within the organization or business who will be the R2 trainers, supporting them after the training is completed and implementation of the R2 curriculum begins.Whenever possible, an evaluation of outcomes is conducted by either R2 experts or by those with this expertise already employed by the organization, institution, or business. Typically, evaluations occur at regular intervals . Evaluation staff at the Resilience Research Centre are available to help compile and analyze the data, produce a report on the findings, and make recommendations for further tailored implementation of the R2 program.By using these four phases, R2 can be offered as a tailored intervention that is contextually and culturally relevant. Over time, however, and to make access to the program cost-effective, several versions of the program intended for similar populations are being aggregated into “standardized” versions of R2. Implementation of standardized versions still includes a period of contextualization where case examples are adapted and pedagogical techniques are matched to the needs of each setting’s participants, though the selection of the factors and much of the content of each module remains constant.To illustrate this process, a version of the R2 program was developed for use by a Fortune 500 company conducting a corporate social responsibility campaign to enhance the resilience of young adults with social anxiety. Through a Delphi process with stakeholders, 16 items were selected during Phase 1 of implementation see .In developing the modules to explain each factor, efforts were made to demonstrate the interconnections between them. Reflecting the science of resilience, factors, and processes associated with resilience interact such that every rugged feature depends on resources to facilitate growth, just as every resource requires the development of individual qualities to cope with atypical stressors. This dynamic model see  ensures M = 25.1, SD = 3.1 and M = 25.4, SD = 3.5, respectively). These results should be interpreted in light of the challenges brought about by the COVID-19 pandemic and the associated stay-at-home orders and restrictions people experienced. Although the resilience of the volunteers did not increase following their participation in the R2 program, stability in resilience levels at this time of great psychosocial distress should be interpreted as a positive outcome. Finally, when volunteers were asked if, overall, the R2 pilot program sessions were helpful in showing them how to build their personal resilience, 80% of those surveyed said the sessions were indeed helpful or very helpful. Similarly, 80% found the sessions helpful for generating ideas to improve their community’s resilience, and 86.7% stated that the sessions and the materials that were provided were easy or very easy to understand and follow. Among the components of the R2 pilot program that volunteers enjoyed the most, they listed the breakout sessions that gave participants a chance to discuss resilience issues in a group setting with other participants and the integration of case examples. Results from other trials are showing similar trends in outcomes.The R2 Resilience Program© is an evidence-informed approach to building resilience that purposefully provides participants with the opportunity to tailor the content. Variation in the number of factors chosen during each application and the factors that are chosen suggests that the goal of matching the program to different contexts and cultures has been effective. While results are still being assessed, initial evaluation data suggest the program is helping participants identify strategies that are useful when dealing with exposure to significant threats to their mental well-being. For example, 65 volunteers with an international service club participated in a trial version of R2 in April 2021. Between pre- and post-intervention, participants experienced a reduction in anxiety and depression symptoms, as well as perceived stress. Volunteers’ anxiety symptoms, assessed using the seven-item Generalized Anxiety Disorder (GAD-7) scale  decreasePedagogically, the approach to building resilience supported by R2 provides program participants with the opportunity to select from a menu of options the factors at multiple systemic levels most likely to produce positive outcomes. This approach, like other effective resilience interventions that adapt across cultures see  shows prWhile the content of R2 adapts when implemented in different settings, this strength in contextual relevance can undermine the program’s fidelity, requiring facilitators to ensure each application adheres to a set of principles  while tolerating unique expressions of the program content. Differences in the length of the program and variations in pedagogical techniques also raise questions about the dosage effect of the program and whether participants are choosing to design the program in the most efficacious way. We expect that with more trials and more evaluations, we will be able to guide organizations, institutions, and businesses better with regard to how they offer the R2 Resilience Program© in each setting.Despite this challenge, resilience programs (like R2) are most likely to be effective when individuals are given access to the resources they require and when these resources are provided in ways that are contextually and culturally meaningful. This means, for example, that persons in positions of power should be involved in programs to help facilitate access to important resources. For instance, if safety and security are a concern in a given organizational context, then in addition to person-centered activities such as encouraging recognition of safe or unsafe practices and articulating safety needs, senior staff need to be engaged in creating the mechanisms for employees to access the supports required to remain safe. Therefore, given an understanding of resilience as process instead of trait, programs like R2 depend on both individual action and the actions of other co-occurring systems to create an optimal context for personal and collective development.The original contributions presented in the study are included in the article/supplementary material; further inquiries can be directed to the corresponding author.Ethical review and approval were not required for the study on human participants in accordance with the local legislation and institutional requirements. The ethics committee waived the requirement of written informed consent for participation.MU led the development of the R2 program and drafted 70% of the manuscript. PJ contributed significantly to the development of the R2 curriculum and the implementation of the program and wrote the remainder of the manuscript. Both authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


They evaluated the acceptability of a digital training module and program by indicating the level of agreement with 15 and 18 statements, respectively, using 5-point Likert scales. The average item scores ranged from 3.0 to 4.2. All but one item, including student engagement, alignment to the curriculum and intent for reuse of the program, had a rounded average or median score of 4. The level of training intensity did not impact the teacher acceptability ratings. In conclusion, the teacher acceptability was good, and additional F2F training does not add value above the solely digital training of the teachers.The teacher-led implementation of healthy eating programs in schools is cost-effective and potentially impactful. Teacher acceptability is important for uptake; however, process evaluations are scarce. This study evaluated the effect of two intensities of teacher training on the evaluation of a vegetable education program for Australian primary schools by teachers. The teachers ( Schools provide an important and equitable opportunity to support healthy eating amongst students ,2. From The evaluations of nutrition and other health-promoting programs most often focus on impact and/or effect evaluations, whereas process evaluations are less frequently undertaken ,9,10. PrAcceptability (appreciation) is perceived to be amongst the most important indicators of process evaluations . To maxiEffect and teacher acceptability evaluations were undertaken on a newly developed vegetable education resource for primary schools to increase children’s vegetable acceptance and willingness to try ,17. ChilInformation from the pilot study on the effect and evaluation of teacher acceptability was used to refine the vegetable education program and its supporting materials. Considerable attention was given to minimise the preparation efforts and specifying the produce quantities needed for each lesson to manage the teacher’s expectations; this was done by calculating the minimal required quantities needed for tastings and translate those to the amount of vegetables needed on a classroom level  and reducing the variety of foods offered in lessons where a vegetable meal was prepared. Other changes included a reduction in the content density of the lessons, whilst simultaneously ensuring that the content retained both a behavioural change focus, as well as strong curriculum alignment. The 5E pedagogical framework  was chanA cluster randomised controlled trial (cluster RCT) amongst 25 schools involving 1639 students was subsequently undertaken to measure the effects on the behavioural outcomes [The current study was a process evaluation undertaken as part of the cluster RCT, which focused on the teacher acceptability of the program. The aims were two-fold: (1) to compare the effect of a low- and high-intensity training program on the teacher acceptability of a vegetable education resource and (2) to compare the teacher acceptability results from the modified version of the vegetable education resource with the previous version of the resource, to determine whether the content changes affected the teacher acceptability. The results will be used to support implementation on a larger scale and identify if there are areas for further improvement.n = 10) or a low (n = 9)-intensity version of teaching training prior to implementation of the program. Additionally, 6 control schools received no training and continued to implement their regular school curriculum. Teachers in both intervention arms were eligible to take part in the teacher evaluation reported here.Eligible participants in this study were primary school teachers who implemented the vegetable education program Taste & Learn™ in their classroom as part of a cluster randomised controlled trial to measure behavioural outcomes on students . The cluResearch was undertaken in two state capital cities: Sydney, New South Wales (NSW) and Adelaide, South Australia (SA) to determine whether any differences existed between states in teacher appreciation of the intervention due to variations in curriculum implementation and health policy by states. Until recently, each state in Australia had its own school curriculum, with state differences in the curricula having historical, geographical or demographic roots . AustralEthical approval for this study was provided by the CSIRO Human Research Ethics Committee (HREC24/2016), the NSW Department of Education and Communities (SERAP2017036) and the SA Department for Education (2018-0032). This trial was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12620000392965).The vegetable education program Taste & Learn™ was described in the Introduction. Detailed lesson plans were provided to teachers, which included suggested vegetables for each lesson. Schools were responsible for sourcing the vegetables themselves, and they were reimbursed upon the production of receipts. Further details of this program have been reported elsewhere .Prior to implementing the program in their classrooms, the teachers received one of two forms of training: (1) Low-intensity training: teacher received written lesson materials and an implementation manual, as well as an individual link to a Learning Management System (LMS) to undertake an online training module, which took around 20 min to complete. Adherence was monitored through the LMS platform. The implementation manual and online training module both covered the objectives of the program, theoretical information on the senses and the development of food acceptance in children and practical information to implement the program. The implementation manual also contained detailed information on alignment to the Australian curriculum. (2) High-intensity training: teachers received lesson materials, manual and online training modules, as in the low-intensity training, but additional interactive face-to-face (F2F) training was provided. F2F training (45 min) was delivered by research staff involved in the study and contained information on the same elements as delivered through the online training and written resources. In addition, implementation plans for their school were discussed with the staff. Adherence to the intervention was monitored through phone contact with the ‘champion’ for the study in each intervention school (both low and high) and through the reimbursement of costs for materials to implement the program.Participants were sent a link by email to take part in an online survey (SurveyGizmo) shortly after they implemented the vegetable education program in their classroom. NSW teachers implemented the program in school term 2 (April–June 2018) and were sent the survey link at the end of June 2018. SA teachers implemented the program in school term 3 (July–mid-September 2018) and were sent the survey link mid-September 2018. This was at the same time that the post-test student behavioural data  were colThe vegetable education program was evaluated using 18 statements , coverinp < 0.05 was used as a measure for statistical significance.Data analysis was conducted using SPSS . A value of U) tests were applied to the single-item ratings, as these variables had a skewed distribution .For the online training module and vegetable education program separately, first, internal consistency of the items pertaining to the same construct  were calculated using Cronbach’s alpha. An average score was calculated for constructs with sufficient internal consistency . Where the internal consistency was lower, the individual items were retained and median values reported because of skewed distributions. To determine if there were differences in the responses between teachers from different intervention arms and states, the univariate analysis of variance (ANOVA) was conducted with the dimension ratings as dependent variables and with the training intensity (low/high) and state (NSW/SA) as the independent factors. Nonparametric  completed the survey, which was a response rate of 78% of the eligible teachers. A total of 73% of teachers in the high-intensity training arm completed the survey and 88% of teachers in the low-intensity training arm. Feedback from teachers from 17 out of 19 intervention schools was received, with an average of 3.8 ± 2.5 teachers per school. The teachers covered all the year levels in primary school, with 23% of teachers who had taught lower primary school classes (5–8 year olds), 40% middle primary school classes (8–10 year olds) and 15% upper primary school classes (10–12 year olds), and 22% taught classes from multiple stages of primary school.n = 51; state: 51% NSW, 49% SA; training intensity: 59% high, 41% low) indicated having conducted the online training module. A total of 75% of teachers in the low-intensity training arm conducted the online training module and 82% of teachers in the high-intensity training arm. There was good internal consistency for the dimensions of content quality, learning goals, motivation, presentation design and reusability  or median score of 4. There were no statistically significant differences in ratings as a factor of the intervention arm (training intensity) or state (NSW/SA) .Open comments provided positive feedback ; comments related to accessing the materials  and comments related to the content. On the latter, two teachers wished that the module provided detailed training on a lesson-by-lesson basis, whereas another teacher commented that the module was not needed, as sufficient background information was given in the lessons themselves.The item scores for the vegetable education program ranged from 3.0 to 4.2 . There wThe statistical analysis showed that the level of intensity of training  did not affect any of the acceptability ratings of the vegetable education program by the teachers . DiffereSuggestions for improvements related to the time/duration of the lessons in relation to content density, with some teachers suggesting breaking up the material into smaller lessons or reducing the amount of material. Preparation time involved for the practical aspects was also mentioned and the involvement of others suggested . Some teachers also suggested adding a recording element for the students . There were two teachers who commented that students in their first year of schooling found it difficult to come up with describing words and suggested buddy classes with older students.p = 0.01) and the preparation efforts needed for the program   , with no < 0.01) .The current study aimed to compare the effects of a low- and a high-intensity training program on the teacher acceptability of a vegetable education resource, Taste & Learn™, and compared it with the acceptability evaluations of a previous version of the program. The results showed that the vegetable education resource had good acceptability amongst the teachers, regardless of the type of training, but SA teachers were less positive about a small number of aspects of the program. Compared to a previous version of the program, the teachers evaluated the preparation efforts and materials more positively.Acceptability of the Taste & Learn™ program by the teachers was good. Not many process evaluations of the comparable programs have been undertaken, but the acceptability of the Taste & Learn™ program was similar to the teacher acceptability of a Dutch sensory education program Taste Lessons . The mosThe current study showed that the revision of the resource materials positively contributed to the teacher acceptability of the program. Notably, the response to the statement “The amount of preparation for this program is reasonable” changed from a score of 2 (“Slightly disagree”)  to 3 (“NLive Life Well @School) [Although there were no differences between states for the majority of the aspects, the teachers in NSW rated the program higher than SA on several aspects. The potential reasons for these differences are unclear. At the time of the study, the NSW government had an active framework of promoting healthy eating programs in schools (@School)  whereas @School) .An effect evaluation of the vegetable education program showed that the level of intensity of teacher training did not affect the student outcomes . The curTwo statements in the current study measured the teacher’s perceptions of specific impacts of the program on the students, i.e., positively influencing student’s vegetable knowledge and student’s vegetable acceptance. The teachers had high agreement to both statements. The intervention had positive behavioural outcome effects on the students, including knowledge and vegetable acceptance  consisteThe response rate of the eligible teachers was 78%. The eligibility criterion for taking part in the survey was that the teachers implemented the vegetable education program in their classroom; therefore, any teachers or schools that dropped out before implementing the vegetable education program were not eligible to take part in this survey.There were 23 out of 106 classes (22%) in the intervention schools that dropped out after the baseline student data were collected . FifteenThis study demonstrated good teacher acceptability of the vegetable education program and its supporting resources. It also highlighted some further development opportunities. The online module was well-received in all aspects, but there were some technical difficulties with accessing the materials. It would also be recommended to undertake a further process evaluation of the program when the full implementation begins to determine its reach and impact on a larger scale—for example, using the RE-AIM framework .